I am facing a strage error in DB2 V9.7 which I noticed in two different databases. I am on AIX 6.1 . I am seeing one of the application getting struck in executing phase. Its not getting terminated through force application command as well. Nothing gets changed in the application snapshot. If I take an application snapshot now and another one after one hour and do a diff, the only difference is the snapshot timestamp.

This process is in executing phase from two days. Let me know if anyone had gone through this type of issue or any ways to trouble shoot this issue.

When this issue happened first time, I tried to bounce the instance with db2stop force. It didn't work as the stop force command tries to force off all the applications before it stops the instance and as this application didn't get forced off, db2stop force was just hanging. I had to issue db2_kill and restart the instance after which this applicaton has gone away. But I am now seeing the same issue again.

Any suggestions ?

The best is to collect several call stacks to see what's happening. For this problem, I'd suggest to open a pmr.

Thanks Bella. I'll do that.

This application which got struck in executing phase was holding the dirty pages in the logs and resulted in throwing these warning messages db2diag.log. I posted in the below thread.

error messages in db2diag.log

Finally it turns out to be an issue with IOCP on AIX 6.1 TL4.
Who ever is using AIX 6.1 TL4, please be aware of application getting hung in executing phase and doesn't terminate through force application. Have to issue db2_kill.

This issue is because of IOCP on AIX 6.1 TL4. I believe AIX PMR team has already been engaged by DB2 PMR team to work on this issue.

Work around is to stay in AIX 6.1 TL3 or lower version

OR

Turn IOCP usage off in db2. If you are in DB2 V9.5, IOCP usage is turned off by default. But if you are in DB2 V9.7 IOCP need to be exclusively turned off -> db2set DB2_USE_IOCP=OFF as the default in V9.7 is 'ON'. With this workaround, one can still stay in AIX 6.1 TL4 but the problem is you cannot take advantage of the new page cleaning behaviour which uses IOCP.

AIX apar for this issue is IZ74508 . AIX has finally released an ifix, IZ74508.100416.epkg.Z, for this issue which can be applied on AIX6.1- TL04 SP1, TL04 SP2 and TL04 SP3

Just want to add that this AIX APAR is HIPER and the info about it will be added to the following doc soon:
IBM - Known issues for DB2 on AIX 5.2, 5.3, and 6.1