If you try to lock a table in exclusive mode, there must be no lock on any row at all. If there are such locks, DB2 waits for the time specified in the lock timeout, hoping that the locks are released. If that doesn't happen, you get the error message you saw.
So what you have to figure out is which other connections are holding locks on the table and then resolve the issue. You can either make sure that those other connections release the locks earlier, e.g. commit/rollback more often, or you may have to increase the lock timeout, or you have to work with row-based locks instead of the table lock.