I had to install DB2 LUW V8.1 to recrete a production environment. I installed DB2 DB2 v8.1.1.0 on AIX 5.3 with DPF and manged to start DB2 with a single partiton. But When I updated AIX /etc/services , db2nodes.cfg and .rhost files to enable DPF for 6 partitions and issued db2start then I get SQL1042C , 'unexpected system error' (for all 6 nodes).
SQL1032N No start database manager command was issued. SQLSTATE=57019


Could someone shade some light on this please ?

extract from db2diag.log:
2010-06-07-23.17.26.6748461 Instance:db2test1 Node:000
PID:2490484(db2star2) TID:1 Appid:none
oper system services sqloqwait2 Probe:80

Unexpected system error 0x23 has occurred.
This has been mapped to ZRC 0x83000023.

PID:2490484 TID:1 Node:000 Title: SYSTEM ERROR DESCRIPTION
No message of desired type
2010-06-07-23.17.26.6797011 Instance:db2test1 Node:000
PID:2490484(db2star2) TID:1 Appid:none
oper system services sqloqwait2 Probe:90

Message number 0 is unavailable.

PID:2490484 TID:1 Node:000 Title: sqloqwait WaitPostQueueId:
0050 00be .P..

PID:2490484 TID:1 Node:000 Title: sqloqwait waitpost ptr:
0780 0000 0001 80a8 ........

PID:2490484 TID:1 Node:000 Title: sqloqwait waitpost:
1000 0000 0001 0000 0000 0000 0000 0002 ................
0000 0000 0000 0000 ........

PID:2490484 TID:1 Node:000 Title: sqloqwait timeout:
0000 0000 ....

PID:2490484 TID:1 Node:000 Title: sqloqwait global Waitlist ptr:
0000 0000 0000 0000 ........

2010-06-07-23.17.26.6921801 Instance:db2test1 Node:000
PID:2490484(db2star2) TID:1 Appid:none
base sys utilities startdbm Probe:500

db2sysc exited prematurely. sqlcode :8300 0023 ...#

2010-06-07-23.17.26.6948941 Instance:db2test1 Node:000
PID:2490484(db2star2) TID:1 Appid:none
base sys utilities startdbm Probe:510

Check for any leftover processes / IPC resources owned by this instance and then updating the instance using db2iupdt command.

Hi, Thanks for the help. Do I need to run db2iupdt with any switches after killing any left over processes of the instance ?

Can you please tell me why we need to run db2iupdt? Does db2diag.log give any indication? I couldn't read much from that.

In case of db2start failing with sql1042c, the first thing to do is to make sure that there are no leftover db2 resources (use commands - ipclean, ps/kill, ipcs/ipcrm) and then update the instance. You don't need to specify any option, just "db2iupdt db2test1" using root. This will refresh the files under instance's .../sqllib dir and it usually helps (in case some permissions were modified or some inconsistency between the installed libraries and local executables under the .../sqllib, etc...). If this doesn't help, then some additional tracing may be needed. You can also just try dropping and recreating the instance.

Hi Bella Thanks. I first cleaned up the leftover db2 process using ipclean, kill and ipcrm and run the "db2iupdt db2test1" however I'm getting the same error.

Then I drop the db2test1 instance and recreate it as below:
"/usr/opt/db2_08_01/instance/db2icrt -a SERVER -s ese -u db2fenc1 -w 64 -p db2c_db2test1 db2test1"

The command completd successfully.

Then I issued db2start/db2stop and was successful as previously (per single node). After that I updated the db2nodes.cfg for 6 nodes and issued db2start getting the original error - SQL1042C "An unexpected system error occurred"
I installed db2 using the same imaged used to install prodution server. Can you please help me to move this forward ?

Since you are attempting to use DPF, there are a lot of other things you must do in addition to changing db2nodes.cfg. For example, you must set up ssh/rsh keys, and some other things. You also need a license for DPF (unless you have a 90 day try and buy license.

Please check the following:

- instance owner can write to /dev/null
- you don't have some security software that is killing db2sysc (not sure why this would only happen with more than one partition... but still check):
IBM - DB2START fails with SQL1042C error and ENOMSG (35) reported in the db2diag.log
- filesystem/mount point/etc... is configured ok for the instance owner


If this (and what Marcus suggested) doesn't help, modify db2nodes.cfg to only contain two partitions, delete/move all files in the db2dump dir, perform ipclean/kill.. and issue db2start. Then, collect db2support using "db2support . -s -g" and attach it here. In addition, please provide:


- .rhosts
- /etc/hosts
- grep -i <instance> /etc/services
- ls -l /dev/null
- errpt -a (db2support doesn't always capture this info)

I can't do a lot of debugging here but will take a look at your config.

Hi, Thanks for both of you, I would like to clarify and eliminate the licencing issue first.

Marcus, I have tested the rsh and it's properly setup and working. Also I have updated the services file to configure tcp/ip for all 6 nodes and also believe updated the .rhosts and hosts file correctly. But I am not sure about the license. I thought db2setup will install the DPF licence by default when I select multipartition install. This is the output from db2licm -l .

Product Name = "DB2 Enterprise Server Edition"
Product Password = "DB2ESE"
Version Information = "8.1"
Expiry Date = "Permanent"
Registered Connect User Policy = "Disabled"
Number Of Entitled Users = "5"
Enforcement Policy = "Soft Stop"
Number of processors = "4"
Number of licensed processors = "1"
Annotation = ""
Other information = ""

Is there any way that I can confirm the issue is due to DPF licence? I expect if it's the case some error message on db2diag.log. For example db2diag.log has warning msg for number of processors (SQL8017W).

Bella, I will be looking at the things that you have suggested and thanks again for both of you.

Hi, I don't think this a license issue if it's the case there should be some error/warning message in the log.

I am just wondering somthing I have forgotten to mention about the this server is the cause.

Oracle run on the same AIX server with very heavy work load utilising 98 - 100 % of the CPU.

If this is the case how can I verify this ? (without stopping Oracle)

I don't really see how high CPU usage is related to db2start failing with sql1042c.

Apologies, it's 98 -100 % memory usage by Oracle. I'm not sure how Oracle release this memeory for other non-Oracle processors like DB2. What's the situation if DB2 doesn't have enough memory to facilitate multipartition communication ? I have attached a memory snapshot. Thanks again Bella.

I'd have expected some other errors logged in the db2diag.log if there was a problem with allocating FCM resources. Do you see any errors in the AIX error log?

Thanks Bella, Finally it has been resolved. You are correct it was due to a security softawre. SeOS policy had to chage to allow db2sysc for instance owner and retrusted.

Now db2 start without an error. But when I issue db2_all (eg db2_all date), I'm getting "Permission denied".

Thanks again for your help