I have a db running on Suse Linux, DB2 V10.1 FP 3. On occasion we hit an issue where a db2stop force will hang, normally we wait around 20 minutes or so, check the db2diag.log to see if there issue is readily apparent (ie. log archiving etc) and if the issue can not be found we issue a db2_kill. For some reason on one of our systems the db2_kill is not working (twice so far), when I run a ps- aux I can see the process is in a uninterruptible sleep state (see below):
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
db2snr 49911 0.0 0.1 709668 52576 ? R 11:48 0:00 /db2/db2snr/sqllib/adm/db2stop2 NOMSG SN
root 50593 0.0 0.1 279676 37356 ? D 12:05 0:00 db2stop
db2snr 51124 0.0 0.0 100744 13244 ? D 12:08 0:00 db2level
db2snr 54887 0.0 0.0 101324 13916 ? D 14:13 0:00 ipclean -h
db2snr 57022 0.0 0.0 101324 13916 ? D 14:28 0:00 ipclean -h
The only way we have been able to get out of this is to reboot the server, which is really strange as in the past 10+ years I can't recall a case of db2_kill not working. If I manually try a kill -9 this does not work as well since the process is in a sleep state.
I can't run a db2support since the shutdown is in progress, this just gets queued up as well. Any ideas?
Thanks for the info Mark. I will try that next time this happens. Unfortunately we could not lave the system down any longer so we ended up rebooting. Database restarted without any issue, did not even require a crash recovery. Really strange issue.