Results 1 to 13 of 13
  1. #1
    Join Date
    Jun 2010
    Posts
    39

    Unanswered: DB2 AIX error - db2sysc exited prematurely

    I had to install DB2 LUW V8.1 to recrete a production environment. I installed DB2 DB2 v8.1.1.0 on AIX 5.3 with DPF and manged to start DB2 with a single partiton. But When I updated AIX /etc/services , db2nodes.cfg and .rhost files to enable DPF for 6 partitions and issued db2start then I get SQL1042C , 'unexpected system error' (for all 6 nodes).
    SQL1032N No start database manager command was issued. SQLSTATE=57019


    Could someone shade some light on this please ?

    extract from db2diag.log:
    2010-06-07-23.17.26.6748461 Instance:db2test1 Node:000
    PID:2490484(db2star2) TID:1 Appid:none
    oper system services sqloqwait2 Probe:80

    Unexpected system error 0x23 has occurred.
    This has been mapped to ZRC 0x83000023.

    PID:2490484 TID:1 Node:000 Title: SYSTEM ERROR DESCRIPTION
    No message of desired type
    2010-06-07-23.17.26.6797011 Instance:db2test1 Node:000
    PID:2490484(db2star2) TID:1 Appid:none
    oper system services sqloqwait2 Probe:90

    Message number 0 is unavailable.

    PID:2490484 TID:1 Node:000 Title: sqloqwait WaitPostQueueId:
    0050 00be .P..

    PID:2490484 TID:1 Node:000 Title: sqloqwait waitpost ptr:
    0780 0000 0001 80a8 ........

    PID:2490484 TID:1 Node:000 Title: sqloqwait waitpost:
    1000 0000 0001 0000 0000 0000 0000 0002 ................
    0000 0000 0000 0000 ........

    PID:2490484 TID:1 Node:000 Title: sqloqwait timeout:
    0000 0000 ....

    PID:2490484 TID:1 Node:000 Title: sqloqwait global Waitlist ptr:
    0000 0000 0000 0000 ........

    2010-06-07-23.17.26.6921801 Instance:db2test1 Node:000
    PID:2490484(db2star2) TID:1 Appid:none
    base sys utilities startdbm Probe:500

    db2sysc exited prematurely. sqlcode :8300 0023 ...#

    2010-06-07-23.17.26.6948941 Instance:db2test1 Node:000
    PID:2490484(db2star2) TID:1 Appid:none
    base sys utilities startdbm Probe:510
    Last edited by stj20; 06-23-10 at 11:41.

  2. #2
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    Check for any leftover processes / IPC resources owned by this instance and then updating the instance using db2iupdt command.

  3. #3
    Join Date
    Jun 2010
    Posts
    39
    Hi, Thanks for the help. Do I need to run db2iupdt with any switches after killing any left over processes of the instance ?

    Can you please tell me why we need to run db2iupdt? Does db2diag.log give any indication? I couldn't read much from that.

  4. #4
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    In case of db2start failing with sql1042c, the first thing to do is to make sure that there are no leftover db2 resources (use commands - ipclean, ps/kill, ipcs/ipcrm) and then update the instance. You don't need to specify any option, just "db2iupdt db2test1" using root. This will refresh the files under instance's .../sqllib dir and it usually helps (in case some permissions were modified or some inconsistency between the installed libraries and local executables under the .../sqllib, etc...). If this doesn't help, then some additional tracing may be needed. You can also just try dropping and recreating the instance.

  5. #5
    Join Date
    Jun 2010
    Posts
    39
    Hi Bella Thanks. I first cleaned up the leftover db2 process using ipclean, kill and ipcrm and run the "db2iupdt db2test1" however I'm getting the same error.

    Then I drop the db2test1 instance and recreate it as below:
    "/usr/opt/db2_08_01/instance/db2icrt -a SERVER -s ese -u db2fenc1 -w 64 -p db2c_db2test1 db2test1"

    The command completd successfully.

    Then I issued db2start/db2stop and was successful as previously (per single node). After that I updated the db2nodes.cfg for 6 nodes and issued db2start getting the original error - SQL1042C "An unexpected system error occurred"
    I installed db2 using the same imaged used to install prodution server. Can you please help me to move this forward ?

  6. #6
    Join Date
    May 2003
    Location
    USA
    Posts
    5,737
    Quote Originally Posted by stj20 View Post
    After that I updated the db2nodes.cfg for 6 nodes and issued db2start getting the original error - SQL1042C "An unexpected system error occurred"
    I installed db2 using the same imaged used to install prodution server. Can you please help me to move this forward ?
    Since you are attempting to use DPF, there are a lot of other things you must do in addition to changing db2nodes.cfg. For example, you must set up ssh/rsh keys, and some other things. You also need a license for DPF (unless you have a 90 day try and buy license.
    M. A. Feldman
    IBM Certified DBA on DB2 for Linux, UNIX, and Windows
    IBM Certified DBA on DB2 for z/OS and OS/390

  7. #7
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    Please check the following:

    - instance owner can write to /dev/null
    - you don't have some security software that is killing db2sysc (not sure why this would only happen with more than one partition... but still check):
    IBM - DB2START fails with SQL1042C error and ENOMSG (35) reported in the db2diag.log
    - filesystem/mount point/etc... is configured ok for the instance owner


    If this (and what Marcus suggested) doesn't help, modify db2nodes.cfg to only contain two partitions, delete/move all files in the db2dump dir, perform ipclean/kill.. and issue db2start. Then, collect db2support using "db2support . -s -g" and attach it here. In addition, please provide:


    - .rhosts
    - /etc/hosts
    - grep -i <instance> /etc/services
    - ls -l /dev/null
    - errpt -a (db2support doesn't always capture this info)

    I can't do a lot of debugging here but will take a look at your config.

  8. #8
    Join Date
    Jun 2010
    Posts
    39
    Hi, Thanks for both of you, I would like to clarify and eliminate the licencing issue first.

    Marcus, I have tested the rsh and it's properly setup and working. Also I have updated the services file to configure tcp/ip for all 6 nodes and also believe updated the .rhosts and hosts file correctly. But I am not sure about the license. I thought db2setup will install the DPF licence by default when I select multipartition install. This is the output from db2licm -l .

    Product Name = "DB2 Enterprise Server Edition"
    Product Password = "DB2ESE"
    Version Information = "8.1"
    Expiry Date = "Permanent"
    Registered Connect User Policy = "Disabled"
    Number Of Entitled Users = "5"
    Enforcement Policy = "Soft Stop"
    Number of processors = "4"
    Number of licensed processors = "1"
    Annotation = ""
    Other information = ""

    Is there any way that I can confirm the issue is due to DPF licence? I expect if it's the case some error message on db2diag.log. For example db2diag.log has warning msg for number of processors (SQL8017W).

    Bella, I will be looking at the things that you have suggested and thanks again for both of you.

  9. #9
    Join Date
    Jun 2010
    Posts
    39
    Hi, I don't think this a license issue if it's the case there should be some error/warning message in the log.

    I am just wondering somthing I have forgotten to mention about the this server is the cause.

    Oracle run on the same AIX server with very heavy work load utilising 98 - 100 % of the CPU.

    If this is the case how can I verify this ? (without stopping Oracle)

  10. #10
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    I don't really see how high CPU usage is related to db2start failing with sql1042c.

  11. #11
    Join Date
    Jun 2010
    Posts
    39
    Apologies, it's 98 -100 % memory usage by Oracle. I'm not sure how Oracle release this memeory for other non-Oracle processors like DB2. What's the situation if DB2 doesn't have enough memory to facilitate multipartition communication ? I have attached a memory snapshot. Thanks again Bella.
    Attached Files Attached Files

  12. #12
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    I'd have expected some other errors logged in the db2diag.log if there was a problem with allocating FCM resources. Do you see any errors in the AIX error log?

  13. #13
    Join Date
    Jun 2010
    Posts
    39
    Thanks Bella, Finally it has been resolved. You are correct it was due to a security softawre. SeOS policy had to chage to allow db2sysc for instance owner and retrusted.

    Now db2 start without an error. But when I issue db2_all (eg db2_all date), I'm getting "Permission denied".

    Thanks again for your help

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •