Our DB V7.2 runs on Aix V4.3

Before refreshing the Database we noticed the following message in the db2diag.log


2004-05-03-17.35.20.979394 Instance:db200001 Node:000
PID:9776(db2) Appid:
command_line_process clp_send_bpreq Probe:4

Error returned from SQLOWQUE: ffff f63e

From Troubleshooting guide - f63e means "Bad Queue Handle".


F63E -902 62 Bad queue handle


To obtain detailed information after the database refresh we changed the dbm cfg parameter DIAGLEVEL to 4
but it started flooding the same message in log file which caused the (/home/db200001/sqllib/db2dump(instance directory))
file system full .To avoid the 100% used scenario we allocated more space to the above file system

To reduce the intensity of logging we changed the DIAGLEVEL to default 0.
removed the db2diag , Deactivated the DB ,stopped and started the instance
but we found that same error was been logged in diag file when we expected it would not log any error in the diag file
also the space was not being freed even after removing the diag file

Finally we reset the diaglevel to 3
To make sure that no commands are hung ,we ran the utility ipclean to remove DB2 engine and clients IPC resources
We also tried to remove defunct interprocess communication segments ,semaphore segments ,message queue segments
We found that one of the memory segment was getting killed but was coming up again automatically
this reduced the intensity of logging by 60% but the same error persisted

Any help in this regard will be appretiated

with regards

ashaq

Seems there is a problem with the clp talking to the db2bp process, do you see it even when there are no db2bp's out there?

I would suggest a db2 terminate, but seems like you've already killed and cleaned everything... does this effect your ability to issue commands?

Try killing and cleaning everything, then check what db2bp processes are out there... make sure it's not dying when you start issuing commands against the database.

Probably one the lab would have to look at...

petruk,
here is the output from ps -ef
UID PID PPID C STIME TTY TIME CMD
root 1 0 0 Aug 22 - 3:19 /etc/init
root 2694 1 0 Aug 22 - 0:00 /usr/lib/errdemon
root 2888 1 0 Aug 22 - 1095:31 /usr/sbin/syncd 60
root 3418 3652 0 Aug 22 - 59:41 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-URdAqa
root 3652 1 0 Aug 22 - 0:00 /usr/dt/bin/dtlogin -daemon
db200001 4036 57734 0 21:30:06 - 13:19 db2agent (NC3DB)
root 4150 3652 0 Aug 22 - 0:00 dtlogin <:0> -daemon
root 4728 31604 0 Jan 22 - 0:01 amqzllp0 -mTESTMQM ?
db200001 4914 57734 0 21:30:04 - 0:32 db2agent (idle)
root 6226 7766 0 Aug 22 - 0:24 /usr/sbin/syslogd
root 6452 1 0 Aug 22 - 0:00 /usr/ccs/bin/shlap
root 6828 7766 0 Aug 22 - 0:07 /usr/sbin/rpc.statd
root 6998 7766 0 Aug 22 - 2:19 sendmail: accepting connections on port 25
root 7274 7766 0 Aug 22 - 0:03 /usr/sbin/portmap
db200001 7626 57734 0 21:30:01 - 7:37 db2agent (idle)
root 7766 1 0 Aug 22 - 0:00 /usr/sbin/srcmstr
root 7998 7766 0 Aug 22 - 0:02 /usr/sbin/inetd
root 8256 7766 0 Aug 22 - 3:34 /usr/sbin/snmpd
root 8514 7766 0 Aug 22 - 0:00 /usr/sbin/dpid2
root 9550 7766 0 Aug 22 - 0:00 /usr/sbin/biod 6
dbautil 9776 1 1 Apr 27 - 2719:34 db2 -v
root 10066 7766 0 Aug 22 - 0:46 /usr/sbin/nfsd 3891
root 10336 7766 0 Aug 22 - 0:46 /usr/sbin/rpc.lockd
root 10582 7766 0 Aug 22 - 0:00 /usr/sbin/rpc.mountd
root 10850 1 0 Aug 22 - 2:13 /usr/sbin/cron
root 11106 1 0 Aug 22 - 0:00 /usr/sbin/uprintfd
root 11362 7766 0 Aug 22 - 0:00 /usr/sbin/writesrv
root 11628 7766 0 Aug 22 - 0:06 /usr/sbin/qdaemon
root 12648 1 0 Aug 22 - 0:00 /usr/lpp/diagnostics/bin/diagd
db200001 12846 57734 0 21:30:04 - 0:17 db2agent (idle)
root 13158 1 0 Aug 22 - 0:00 /usr/bin/AIXPowerMgtDaemon
db200001 13778 25380 0 21:30:06 - 0:02 db2agent (idle)
imnadm 14190 1 0 Aug 22 - 0:00 /usr/IMNSearch/httpdlite/httpdlite -r /etc/IMNSearch/httpdlite/httpdlite.conf
root 14450 1 0 Aug 22 - 0:00 /usr/sbin/getty /dev/console
root 14712 1 0 Aug 22 - 0:00 imqsmdem imqsrv.ini /etc/IMNSearch/dbcshelp/
root 14968 1 0 Aug 22 lft0 0:00 /usr/sbin/getty /dev/lft0
db200001 15212 57734 0 21:30:04 - 8:09 db2agent (idle)
db200001 15590 25380 0 21:30:04 - 4:33 db2agent (idle)
db200001 15788 57734 0 21:30:06 - 0:15 db2agent (idle)
root 16010 4150 1 Aug 22 - 726:31 dtgreet
root 16290 1 0 Aug 22 - 0:00 /esm/bin/aix-rs6k/esmd -fv
db200001 16546 49088 0 20:45:37 - 1:20 db2agent (NC3DB)
db200001 16674 57734 0 21:30:02 - 7:46 db2agent (idle)
db200001 17078 28714 0 20:45:54 - 0:00 db2pfchr
db200001 17312 49088 0 20:45:37 - 0:05 db2agent (NC3DB)
db200001 18108 49088 0 20:45:37 - 0:00 db2agent (NC3DB)
db200001 18238 28714 0 20:45:54 - 0:01 db2pfchr
db200001 18486 28714 0 20:45:54 - 0:49 db2pfchr
root 18702 1 0 Jan 22 - 1:29 amqzxma0 -m MQMGOM
db200001 19358 57734 0 21:30:06 - 0:03 db2agent (idle)
db200001 20258 25380 0 21:30:06 - 0:06 db2agent (idle)
db200001 21154 57734 0 20:55:04 - 0:14 db2agent (NC3DB)
db200001 21512 25380 0 21:30:02 - 0:05 db2agent (idle)
db200001 22204 49838 0 21:00:01 - 11:13 db2agent (idle)
db200001 23006 25380 0 21:30:03 - 1:42 db2agent (idle)
db200001 23234 28714 0 20:45:32 - 0:00 db2spmlw
db200001 23668 28714 0 20:45:54 - 0:02 db2pfchr
db200001 24878 49088 0 20:45:30 - 0:00 db2spmrm
db200001 25380 49088 0 20:45:29 - 0:00 db2tcpcm
db200001 25682 28714 0 20:45:54 - 0:03 db2pfchr
root 25880 18702 0 Jan 22 - 0:01 amqzllp0 -mMQMGOM ?
db200001 26406 49838 0 20:47:59 - 0:03 db2agent (NC3DB)
dbautil 26950 47020 0 Apr 28 pts/0 0:00 -ksh
db200001 27480 28714 0 20:46:00 - 0:12 db2pclnr
db200001 28024 28714 0 20:45:54 - 0:11 db2pfchr
db200001 28714 49088 0 20:45:29 - 0:00 db2gds
db200001 29092 25380 0 21:30:04 - 6:36 db2agent (idle)
db200001 29384 49088 0 20:45:37 - 0:00 db2agent (NC3DB)
db200001 29990 57734 0 20:53:45 - 0:00 db2agent (NC3DB)
root 30406 7998 0 12:49:34 - 0:00 telnetd -a
root 30690 51274 0 Jan 22 - 0:17 /usr/mqm/bin/amqzfuma -m SALOME
root 30940 1 0 20:45:29 - 0:00 db2wdog
root 31180 18702 0 Jan 22 - 0:01 amqhasmx MQMGOM /var/mqm
db200001 31378 28714 0 20:46:00 - 0:12 db2pclnr
root 31604 1 0 Jan 22 - 1:31 amqzxma0 -m TESTMQM
db200001 31940 28714 0 20:45:54 - 0:00 db2pfchr
db200001 32234 25380 0 21:30:05 - 3:16 db2agent (idle)
shussain 33056 30406 0 12:49:39 pts/1 0:00 -ksh
db200001 33734 57734 0 21:30:02 - 5:05 db2agent (idle)
db200001 33910 25380 0 21:05:07 - 0:00 db2agent (NC3DB)
db200001 35260 28714 0 20:45:54 - 0:05 db2pfchr
db200001 35436 25380 0 21:30:04 - 22:42 db2agent (NC3DB)
db200001 35704 25380 0 21:30:06 - 0:13 db2agent (idle)
db200001 37296 25380 0 21:30:06 - 0:00 db2agent (idle)
db200001 37378 57734 0 21:30:03 - 9:43 db2agent (idle)
db200001 37768 25380 0 21:30:01 - 17:48 db2agent (NC3DB)
db200001 37914 25380 0 21:30:06 - 0:03 db2agent (idle)
db200001 38294 28714 0 20:45:54 - 0:02 db2pfchr
db200001 38554 25380 0 21:30:02 - 2:38 db2agent (idle)
root 38824 51274 0 Jan 22 - 0:01 amqzllp0 -mSALOME ?
db200001 39164 25380 0 20:55:20 - 0:20 db2agent (NC3DB)
root 39592 18702 0 Jan 22 - 0:17 /usr/mqm/bin/amqzfuma -m MQMGOM
db200001 39744 28714 0 20:45:54 - 0:29 db2pfchr
db200001 39980 57734 0 20:47:23 - 0:08 db2agent (NC3DB)
db200001 40360 25380 0 21:30:07 - 12:38 db2agent (NC3DB)
db200001 40754 25380 0 21:30:05 - 2:54 db2agent (idle)
db200001 41284 57734 0 20:47:24 - 0:00 db2agent (NC3DB)
root 41990 31604 0 Jan 22 - 0:17 /usr/mqm/bin/amqzfuma -m TESTMQM
db200001 42816 28714 0 20:45:54 - 0:08 db2pfchr
db200001 43120 25380 0 20:48:23 - 0:01 db2agent (NC3DB)
db200001 43948 25380 0 21:30:05 - 0:39 db2agent (idle)
db200001 44252 25380 0 21:30:02 - 0:01 db2agent (idle)
db200001 45168 28714 0 20:45:54 - 0:00 db2pfchr
db200001 45534 25380 0 21:30:04 - 11:31 db2agent (NC3DB)
db200001 45934 57734 0 20:47:18 - 0:00 db2agent (NC3DB)
db200001 46088 25380 0 20:59:08 - 0:00 db2agent (NC3DB)
root 46810 51274 0 Jan 22 - 0:01 amqhasmx SALOME /var/mqm
root 47020 7998 0 Apr 28 - 0:00 telnetd -a
root 47288 7998 0 Sep 26 - 0:00 rpc.ttdbserver 100083 1
root 48088 31604 0 Jan 22 - 0:02 amqhasmx TESTMQM /var/mqm
db200001 49088 30940 0 20:45:29 - 0:00 db2sysc
db200001 49304 25380 0 21:30:02 - 3:40 db2agent (idle)
db200001 49838 49088 0 20:45:29 - 0:00 db2ipccm
db200001 50152 28714 0 20:45:54 - 0:08 db2dlock (NC3DB)
db200001 50312 25380 0 20:52:15 - 0:00 db2agent (NC3DB)
db200001 51064 25380 0 21:30:05 - 0:29 db2agent (idle)
root 51274 1 0 Jan 22 - 1:32 amqzxma0 -m SALOME
root 52580 51274 0 Jan 22 - 0:19 amqzlaa0 -mSALOME -fip1
db200001 52828 57734 0 21:30:02 - 0:48 db2agent (idle)
db200001 53712 49088 0 20:45:37 - 0:00 db2agent (NC3DB)
db200001 53828 25380 0 21:30:06 - 2:17 db2agent (idle)
db200001 54232 28714 0 20:46:00 - 0:12 db2pclnr
shussain 54376 33056 4 15:42:35 pts/1 0:00 ps -ef
db200001 54654 28714 0 20:46:00 - 0:12 db2pclnr
db200001 55152 25380 0 21:30:00 - 9:37 db2agent (idle)
db200001 55444 25380 0 20:47:18 - 0:00 db2agent (NC3DB)
db200001 55648 28714 0 20:45:54 - 0:00 db2loggr (NC3DB)
db200001 55908 25380 0 21:30:01 - 6:19 db2agent (NC3DB)
db200001 56238 57734 0 21:30:02 - 7:49 db2agent (idle)
db200001 56364 25380 0 21:30:06 - 0:26 db2agent (idle)
db200001 57098 28714 0 20:45:30 - 0:00 db2resyn
db200001 57590 25380 0 20:52:16 - 0:00 db2agent (NC3DB)
db200001 57734 49088 0 20:45:29 - 0:00 db2tcpcm
db200001 58356 25380 0 21:30:02 - 13:55 db2agent (NC3DB)
db200001 58408 25380 0 21:30:05 - 6:13 db2agent (idle)
db200001 58684 28714 0 20:45:54 - 0:15 db2pfchr
db200001 59096 57734 0 21:30:03 - 0:01 db2agent (idle)
root 59626 51274 0 Jan 22 - 0:14 /usr/mqm/bin/runmqchi -m SALOME
db200001 59666 28714 0 20:45:54 - 0:23 db2pfchr
db200001 60104 28714 0 20:45:54 - 0:04 db2pfchr
db200001 60512 28714 0 20:45:30 - 0:00 db2srvlst
db200001 60780 28714 0 20:45:54 - 1:04 db2pfchr
db200001 60968 57734 0 21:04:55 - 0:00 db2agent (NC3DB)
db200001 61266 25380 0 20:53:22 - 0:00 db2agent (NC3DB)

the only db2bp ,
[shussain@nc3testdb#]ps -ef|grep db2bp
shussain 44556 33056 0 15:47:37 pts/1 0:00 grep db2bp
[shussain@nc3testdb#]

I could not find any db2bp process ,If this is the error what you suggest me next ,and also want to know the use of db2bp.
how to bring db2bp up and stop it from dying when issuing commands against DB

thanks for your reply

ashaq

Why not stop the instance and do a db2iupdt ....

I have seen some problems with db2bp process which went away when I did this ...

Cheers
sathyaram

Sathyaram,
if we execute the db2iupdt without applying fixpack ,will it lead to any other
problem ,also do you think using the following command can help me
$ /usr/sbin/slibclean (to clean db2memory)..thanks

db2iupdt will not create problems even if you run it without applying fixpacks ... If there is a problem, it has to do with the environment corruption and not with db2iupdt ...

slibclean should hopefully help

Cheers

Sathyaram,
The problem is solved now ,I repeated the same process as metioned earier
After shutting down instance ,I used ipclean and db2_kill which left two process .One of them was writing to the diag file .Killed it and it released the whole memory ..Problem is solved but the cause of the problem is not yet known..thanks for ur help