Hello

I am getting the following error message in db2diag.log file from yesterday night. Is there anybody know how to track out this issue?

2010-07-06-13.28.37.442749+060 E56440596E583 LEVEL: Error
PID : 8338 TID : 47455219476800PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-38563 APPID: *N0.bculinux.100706122841
AUTHID : BCULINUX
EDUID : 40456 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, DRDA Application Server, sqljsTermAgentReply, probe:10
MESSAGE : DIA5000C A DRDA AS token "AGENT TERMINATING" was detected. The
diagnostic data returned is (SRVDGN): "SQLERRPQL09053
SQLCODE:-1224".

2010-07-06-13.28.37.442987+060 E56441180E834 LEVEL: Error
PID : 8338 TID : 47455219476800PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-38563 APPID: *N0.bculinux.100706122841
AUTHID : BCULINUX
EDUID : 40456 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, DRDA Application Server, sqljsTermAgentReply, probe:11
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -1224 sqlerrml: 0
sqlerrmc:
sqlerrp : SQL09053
sqlerrd : (1) 0x00000000 (2) 0x00000000 (3) 0x00000000
(4) 0x00000000 (5) 0x00000000 (6) 0x00000000
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:

=> SQL1224N The database manager is not able to accept new requests, has terminated all requests in progress, or has terminated your particular request due to an error or a force interrupt.

=> The same error has been repeating 29 times in db2diag.log for last 6-7 days
=> There is always users connecting to this machine and not sure what is causing this error message.

There is no any other error message is getting generated in same timestamp.

Kindly provide if any clue about the same....

Regards,
Eldho Mathew

</snippet>

2010-07-09-10.11.29.265236+060 I113857934E477 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, base sys utilities, sqleMergeSqlca, probe:20
MESSAGE : Received sqlcode -911 for request 80000004 from node number 5

2010-07-09-10.11.29.265353+060 I113858412E846 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, trace services, sqlt_logerr_data (secondary logging func, probe:0
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -911 sqlerrml: 2
sqlerrmc: 68
sqlerrp : SQLRL02B
sqlerrd : (1) 0x80100044 (2) 0x00000044 (3) 0x00000000
(4) 0x00000000 (5) 0x00000000 (6) 0x00000005
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:

2010-07-09-10.11.29.310398+060 I113859259E478 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, base sys utilities, sqleMergeSqlca, probe:20
MESSAGE : Received sqlcode -911 for request 80000004 from node number 13

2010-07-09-10.11.29.310576+060 I113859738E846 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, trace services, sqlt_logerr_data (secondary logging func, probe:0
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -911 sqlerrml: 2
sqlerrmc: 68
sqlerrp : SQLRL02B
sqlerrd : (1) 0x80100044 (2) 0x00000044 (3) 0x00000000
(4) 0x00000000 (5) 0x00000000 (6) 0x0000000D
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:

2010-07-09-10.11.29.310685+060 I113860585E478 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, base sys utilities, sqleMergeSqlca, probe:20
MESSAGE : Received sqlcode -911 for request 80000004 from node number 15

2010-07-09-10.11.29.310815+060 I113861064E846 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, trace services, sqlt_logerr_data (secondary logging func, probe:0
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -911 sqlerrml: 2
sqlerrmc: 68
sqlerrp : SQLRL02B
sqlerrd : (1) 0x80100044 (2) 0x00000044 (3) 0x00000000
(4) 0x00000000 (5) 0x00000000 (6) 0x0000000F
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:

2010-07-09-10.11.29.319769+060 I113861911E478 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, base sys utilities, sqleMergeSqlca, probe:20
MESSAGE : Received sqlcode -911 for request 80000004 from node number 28

2010-07-09-10.11.29.319916+060 I113862390E846 LEVEL: Error
PID : 8338 TID : 47454405781824PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41868 APPID: *N0.bculinux.100709091103
AUTHID : BCULINUX
EDUID : 154568 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, trace services, sqlt_logerr_data (secondary logging func, probe:0
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -911 sqlerrml: 2
sqlerrmc: 68
sqlerrp : SQLRL02B
sqlerrd : (1) 0x80100044 (2) 0x00000044 (3) 0x00000000
(4) 0x00000000 (5) 0x00000000 (6) 0x0000001C
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:
--------------------------* Error message *----------------------------------------
2010-07-09-10.11.48.413983+060 E113863237E583 LEVEL: Error
PID : 8338 TID : 47449209039168PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41927 APPID: *N0.bculinux.100709091150
AUTHID : BCULINUX
EDUID : 154799 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, DRDA Application Server, sqljsTermAgentReply, probe:10
MESSAGE : DIA5000C A DRDA AS token "AGENT TERMINATING" was detected. The
diagnostic data returned is (SRVDGN): "SQLERRPQL09053
SQLCODE:-1224".

2010-07-09-10.11.48.414193+060 E113863821E834 LEVEL: Error
PID : 8338 TID : 47449209039168PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41927 APPID: *N0.bculinux.100709091150
AUTHID : BCULINUX
EDUID : 154799 EDUNAME: db2agent (BCUDB) 0
FUNCTION: DB2 UDB, DRDA Application Server, sqljsTermAgentReply, probe:11
DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
sqlcaid : SQLCA sqlcabc: 136 sqlcode: -1224 sqlerrml: 0
sqlerrmc:
sqlerrp : SQL09053
sqlerrd : (1) 0x00000000 (2) 0x00000000 (3) 0x00000000
(4) 0x00000000 (5) 0x00000000 (6) 0x00000000
sqlwarn : (1) (2) (3) (4) (5) (6)
(7) (8) (9) (10) (11)
sqlstate:
-----------------------------********* That's all ***********-------------------------------
2010-07-09-10.11.55.564295+060 I113864656E481 LEVEL: Warning
PID : 8338 TID : 47455143979328PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41871 APPID: 172.16.10.160.6958.100709091155
AUTHID : ETLUSER
EDUID : 154776 EDUNAME: db2agent (BDWLP01) 0
FUNCTION: DB2 UDB, database utilities, sqluvtld_route_in, probe:837
DATA #1 : <preformatted>
Starting LOAD operation (S) (1) (R).


2010-07-09-10.11.55.567899+060 I113865138E511 LEVEL: Warning
PID : 8338 TID : 47449150318912PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41871 APPID: 172.16.10.160.6958.100709091155
AUTHID : ETLUSER
EDUID : 154812 EDUNAME: db2lcata 0
FUNCTION: DB2 UDB, database utilities, DIAG_NOTE, probe:0
DATA #1 : String, 84 bytes
LOADID: 154776.2010-07-09-10.11.55.564284.0 (4;16)
Catalog subagent is running. 0, 0

2010-07-09-10.11.56.317552+060 I113865650E539 LEVEL: Warning
PID : 8338 TID : 47454175095104PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41871 APPID: 172.16.10.160.6958.100709091155
AUTHID : ETLUSER
EDUID : 154696 EDUNAME: db2linit 0
FUNCTION: DB2 UDB, database utilities, DIAG_NOTE, probe:0
DATA #1 : String, 111 bytes
LOADID: 154776.2010-07-09-10.11.55.564284.0 (4;16)
Init SA is running, (status = 2) (action = 5) (op = R). 0, 0

2010-07-09-10.11.56.321111+060 I113866190E539 LEVEL: Warning
PID : 8338 TID : 47454175095104PROC : db2sysc 0
INSTANCE: bculinux NODE : 000 DB : BDWLP01
APPHDL : 0-41871 APPID: 172.16.10.160.6958.100709091155
AUTHID : ETLUSER
EDUID : 154696 EDUNAME: db2linit 0
FUNCTION: DB2 UDB, database utilities, DIAG_NOTE, probe:0
DATA #1 : String, 111 bytes
LOADID: 154776.2010-07-09-10.11.55.564284.0 (4;16)
Init SA is running, (status = 4) (action = 5) (op = R). 0, 0

There is no clues in which situation it's happening.... even though there are lock messages prior to the error... the EDUID is different.

Not sure how to track out what is generating this error and when?
db2 trace is not possible in this situation. This error is not occurring frequently and always more than 100 application connects to this server that makes the root cause difficult. I know IBM can easily give some clue.... however, I want to make sure it is impossible for a DBA to get into a solution.... Kindly provide if you have any ideas....