Look in the SQL Reference for the correct syntax of the UPDATE statement.
Your statement should look like:
update BorrowHist set (NumNoticesSent,c_price) = ((BorrowHist.NumNoticesSent + 1) , (select t11.c_price from borrowhist,session.tt11 t11 where BorrowHist.borrowerid =t11.borrowerid and BorrowHist.accession_no = t11.accession_no and BorrowHist.LIBRARYNAME =t11.lib_name))
HTH
Andy