Results 1 to 6 of 6
  1. #1
    Join Date
    Mar 2014
    Posts
    26

    Answered: DB2 Instance Frequently Disconnected

    Dear Friends

    Need your assistance, currently I`m having an issue database suddenly disconnected . this issue happens frequently almost every day

    Seek your advice base from below log.

    I cant open PMR to IBM due this database absolute no longer supported by IBM

    DB2 Version :- 8.1
    OS Version :- AIX 5

    Im very appreciate your advice and assistance



    2016-07-28-08.13.03.749492+480 E26267C438 LEVEL: Error (OS)
    PID : 880800 TID : 1 PROC : db2agent (CC_DB)
    INSTANCE: prdinst1 NODE : 000 DB : CC_DB
    APPHDL : 0-12 APPID: AC0F0529.B5AB.160728001039
    FUNCTION: DB2 UDB, oper system services, sqlockpw_int, probe:50
    CALLED : OS, -, unspecified_system_function
    OSERR : EINTR (4) "A system call received an interrupt."

    2016-07-28-08.13.03.750262+480 I26706C249 LEVEL: Severe
    PID : 880800 TID : 1
    FUNCTION: DB2 Common, Security, Users and Groups, secLogMessage, probe:20
    DATA #1 : String, 44 bytes
    check password failed with rc = -2097151996


    2016-07-28-08.13.03.757563+480 I26956C484 LEVEL: Severe
    PID : 880800 TID : 1 PROC : db2agent (idle)
    INSTANCE: prdinst1 NODE : 000 DB : CC_DB
    APPID : AC0F0529.B5AB.160728001039
    FUNCTION: DB2 UDB, oper system services, sqloEDUSIGCHLDHandler, probe:50
    DATA #1 : String, 146 bytes
    Detected the abnormal termination of an EDU with process id 454874
    The waitpid() status of the abnormally terminated child process was 0x00000100

    2016-07-28-08.13.03.767281+480 I27441C334 LEVEL: Warning
    PID : 1114206 TID : 1 PROC : db2loggr (CC_DB)
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, data protection, sqlpghck, probe:1390
    MESSAGE : ExtNum 27904, state 401, baselsn 0000003661824000 nextlsn
    00000036618242C3

    2016-07-28-08.13.03.792181+480 E27776C575 LEVEL: Severe
    PID : 880800 TID : 1 PROC : db2agent (idle)
    INSTANCE: prdinst1 NODE : 000 DB : CC_DB
    APPID : AC0F0529.B5AB.160728001039
    FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:10
    MESSAGE : ADM0503C An unexpected internal processing error has occurred. ALL
    DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN.
    Diagnostic information has been recorded. Contact IBM Support for
    further assistance.

    2016-07-28-08.13.03.792538+480 E28352C675 LEVEL: Severe
    PID : 880800 TID : 1 PROC : db2agent (idle)
    INSTANCE: prdinst1 NODE : 000 DB : CC_DB
    APPID : AC0F0529.B5AB.160728001039
    FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:20
    DATA #1 : Signal Number Recieved, 4 bytes
    6
    DATA #2 : Siginfo, 64 bytes
    0x2FF1A9DC : 0000 0006 0000 0000 0000 0009 0000 0000 ................
    0x2FF1A9EC : 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0x2FF1A9FC : 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0x2FF1AA0C : 0000 0000 0000 0000 0000 0000 0000 0000 ................

    2016-07-28-08.13.03.850224+480 I29028C432 LEVEL: Severe
    PID : 499794 TID : 1 PROC : db2ipccm
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, oper system services, sqloEDUSIGCHLDHandler, probe:50
    DATA #1 : String, 159 bytes
    Detected the death of an EDU with process id 880800
    The signal number that terminated this process was 6
    Look for trap files (t880800.*) in the dump directory

    2016-07-28-08.13.03.880193+480 E29461C510 LEVEL: Severe
    PID : 499794 TID : 1 PROC : db2ipccm
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:10
    MESSAGE : ADM0503C An unexpected internal processing error has occurred. ALL
    DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN.
    Diagnostic information has been recorded. Contact IBM Support for
    further assistance.

    2016-07-28-08.13.03.880661+480 E29972C610 LEVEL: Severe
    PID : 499794 TID : 1 PROC : db2ipccm
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:20
    DATA #1 : Signal Number Recieved, 4 bytes
    6
    DATA #2 : Siginfo, 64 bytes
    0x2FF1B44C : 0000 0006 0000 0000 0000 0009 0000 0000 ................
    0x2FF1B45C : 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0x2FF1B46C : 0000 0000 0000 0000 0000 0000 0000 0000 ................
    0x2FF1B47C : 0000 0000 0000 0000 0000 0000 0000 0000 ................

    2016-07-28-08.13.03.933079+480 I30583C361 LEVEL: Severe
    PID : 856234 TID : 1 PROC : db2sysc
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler, probe:15
    MESSAGE : DiagData
    DATA #1 : Hexdump, 15 bytes
    0x10008AF0 : 416E 2045 4455 2063 7261 7368 6564 2E An EDU crashed.

    2016-07-28-08.13.03.933203+480 I30945C349 LEVEL: Severe
    PID : 856234 TID : 1 PROC : db2sysc
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler, probe:16
    MESSAGE : DiagData
    DATA #1 : Hexdump, 4 bytes
    0x2FF21244 : 0007 A052 ...R

    2016-07-28-08.13.03.933264+480 I31295C349 LEVEL: Severe
    PID : 856234 TID : 1 PROC : db2sysc
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler, probe:17
    MESSAGE : DiagData
    DATA #1 : Hexdump, 4 bytes
    0x2FF21250 : 0000 0101 ....

    2016-07-28-08.13.03.933322+480 I31645C349 LEVEL: Severe
    PID : 856234 TID : 1 PROC : db2sysc
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler, probe:18
    MESSAGE : DiagData
    DATA #1 : Hexdump, 4 bytes
    0x2FF21254 : FFFF FFFF ....

    2016-07-28-08.13.03.933380+480 I31995C349 LEVEL: Severe
    PID : 856234 TID : 1 PROC : db2sysc
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleChildCrashHandler, probe:19
    MESSAGE : DiagData
    DATA #1 : Hexdump, 4 bytes
    0x2FF21240 : FFFF FC88 ....

    2016-07-28-08.13.03.933438+480 I32345C339 LEVEL: Severe
    PID : 856234 TID : 1 PROC : db2sysc
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleSysCtlr, probe:10
    MESSAGE : DiagData
    DATA #1 : Hexdump, 4 bytes
    0x2FF21A84 : 0000 0001 ....

    2016-07-28-08.13.03.955412+480 E32685C503 LEVEL: Severe
    PID : 745684 TID : 1 PROC : db2wdog
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, oper system services, sqloRunInstance, probe:490
    MESSAGE : ADM0503C An unexpected internal processing error has occurred. ALL
    DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN.
    Diagnostic information has been recorded. Contact IBM Support for
    further assistance.

    2016-07-28-08.13.03.955873+480 I33189C350 LEVEL: Severe
    PID : 745684 TID : 1 PROC : db2wdog
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, oper system services, sqloRunInstance, probe:500
    MESSAGE : DiagData
    DATA #1 : Hexdump, 8 bytes
    0x2FF21B68 : 0000 0102 0000 0009 ........

    2016-07-28-08.13.03.955992+480 I33540C347 LEVEL: Severe
    PID : 745684 TID : 1 PROC : db2wdog
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, base sys utilities, sqleCleanupResources, probe:5
    MESSAGE : DiagData
    DATA #1 : Hexdump, 4 bytes
    0x2FF21AC8 : 0000 0102 ....

    2016-07-28-08.13.03.956211+480 I33888C393 LEVEL: Warning
    PID : 745684 TID : 1 PROC : db2wdog
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, routine_infrastructure, sqlerKillAllFmps, probe:5
    MESSAGE : Bringing down all db2fmp processes as part of db2stop
    DATA #1 : Hexdump, 4 bytes
    0x2FF21930 : 0000 0000 ....

    2016-07-28-08.13.03.957534+480 I34282C283 LEVEL: Severe
    PID : 917724 TID : 772 PROC : db2fmp (C)
    INSTANCE: prdinst1 NODE : 000
    FUNCTION: DB2 UDB, common communication, sqlccipcdarihandshake, probe:3
    RETCODE : ZRC=0x83000024=-2097151964

    2016-07-28-08.13.03.957782+480 I34566C317 LEVEL: Warning
    PID : 917724 TID : 772 PROC : db2fmp (C)
    INSTANCE: prdinst1 NODE : 000
    MESSAGE : Thread of db2fmp terminated with nonzero rc
    DATA #1 : Hexdump, 4 bytes
    0x300EE9D8 : FFFF FB95

  2. Best Answer
    Posted by db2mor

    "Post the output of the 'db2level' command (you can run this as user prdinst1).

    If you cannot upgrade to a currently supported version of DB2 then consider using the final fixpack for the V8.x version.

    What are the file-permissions for ~prdinst1/sqllib/security/db2ckpw ?
    Have they changed recently?

    The code -2097151996 means "unexpected operating system error" which is consistent with "unspecified_system_function".

    Look for more details in the AIX message log / syslog, ask your AIX sysadmin for help.

    Determine the date when DB2 was working correctly and you were not seeing this error. Then use that date to work out what AIX changes happened after that date.
    For example, if any AIX changes happened after that date, or any configuration changes that can impact DB2 such as changes to the authentication mechanism or encryption algorithm for passwords, or permissions changes on files and directories used by DB2. Your AIX sysadmin may have records of changes.

    If you have another AIX hostname with the exact same DB2 version and it is working OK then compare the working and failing machines for the AIX security-config settings."


  3. #2
    Join Date
    Apr 2012
    Posts
    1,035
    Provided Answers: 18
    Post the output of the 'db2level' command (you can run this as user prdinst1).

    If you cannot upgrade to a currently supported version of DB2 then consider using the final fixpack for the V8.x version.

    What are the file-permissions for ~prdinst1/sqllib/security/db2ckpw ?
    Have they changed recently?

    The code -2097151996 means "unexpected operating system error" which is consistent with "unspecified_system_function".

    Look for more details in the AIX message log / syslog, ask your AIX sysadmin for help.

    Determine the date when DB2 was working correctly and you were not seeing this error. Then use that date to work out what AIX changes happened after that date.
    For example, if any AIX changes happened after that date, or any configuration changes that can impact DB2 such as changes to the authentication mechanism or encryption algorithm for passwords, or permissions changes on files and directories used by DB2. Your AIX sysadmin may have records of changes.

    If you have another AIX hostname with the exact same DB2 version and it is working OK then compare the working and failing machines for the AIX security-config settings.

  4. #3
    Join Date
    Mar 2014
    Posts
    26
    Dear db2mor

    thank you for assitance and advice .

    recently we do DR activity where we swing Production database into DR environment .
    This issue happen after we swing back into Production environment . Now i faced third time database disconnected for this week .
    Im really no idea how to resolve this issue due limited resources and info regarding this issue.

    Base from IBM Portal and forum , i need to apply latest fixpack for this database .
    While, this database end off support is it possible for me to get latest fixpack ? Due i cant download for IBM portal base from search.

    below db2level result :-
    $ db2level
    DB21085I Instance "prdinst1" uses "32" bits and DB2 code release "SQL08021"
    with level identifier "03020106".
    Informational tokens are "DB2 v8.1.1.80", "s041221", "U800400", and FixPak "8".
    Product is installed at "/usr/opt/db2_08_01".

    file permission ~prdinst1/sqllib/security/db2ckpw currnelty root . I think no changes has been made .

    For others related with AIX , currenlty im working with Sys Admin .


    Thank you
    Amy Badri

  5. #4
    Join Date
    May 2010
    Location
    India
    Posts
    84
    Provided Answers: 2
    What is the value for DB2_NUM_CKPW_DAEMONS? There were some issues related to password authentication and AIX 5. Fix was to change the value for DB2_NUM_CKPW_DAEMONS. I do not remember details. Check your current value and find out whether there are any issues with that setting for your configuration (DB2 version and AIX version).

    Satya...

  6. #5
    Join Date
    Jul 2016
    Location
    Moscow
    Posts
    108
    Provided Answers: 13
    Install aix apar iy71365
    or
    Install db2 8.1.11 at least (it's better to install the latest fp18, of course) and set the following variable:
    db2set DB2_NUM_CKPW_DAEMONS=0
    Regards,
    Mark.

  7. #6
    Join Date
    Mar 2014
    Posts
    26
    this issue resolved after bounce server . this issue caused most probably because of hung process ( batch) file process where this file using root access


    Thank you

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •