Hi,

I have started the db through db2start.The database start up but after few seconds it automatically shut down.

Pleae help me us

2009-05-16-13.25.34.535294+330 E1444174A512 LEVEL: Severe
PID : 16427 TID : 1 PROC : db2pclnr 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:10
MESSAGE : ADM0503C An unexpected internal processing error has occurred. ALL
DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN.
Diagnostic information has been recorded. Contact IBM Support for
further assistance.

2009-05-16-13.25.34.535433+330 E1444687A1617 LEVEL: Severe
PID : 16427 TID : 1 PROC : db2pclnr 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:20
DATA #1 : Signal Number Recieved, 4 bytes
6
DATA #2 : Siginfo, 256 bytes
0xFFFFFFFF7FFFCCA0 : 0000 0006 FFFF FFFF 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCCB0 : 0000 402B 0000 0000 0000 0068 0000 0000 ..@+.......h....
0xFFFFFFFF7FFFCCC0 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCCD0 : 0009 AE8F 0000 0016 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCCE0 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCCF0 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD00 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD10 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD20 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD30 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD40 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD50 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD60 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD70 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD80 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0xFFFFFFFF7FFFCD90 : 0000 0000 0000 0000 0000 0000 0000 0000 ................

2009-05-16-13.25.39.469530+330 E1446305A917 LEVEL: Severe
PID : 16397 TID : 1 PROC : db2gds 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, oper system services, sqloEDUSIGCHLDHandler, probe:50
DATA #1 : <preformatted>
Detected the death of an EDU with process id 16439.
The signal number that terminated this process was 6.
Look for trap files (t16439.*) in the dump directory.
CALLSTCK:
[0] 0xFFFFFFFF7AF74C54 sqloSetNonCriticalEDUFlag + 0x4D4
[1] 0xFFFFFFFF787D2798 _thr_slot_offset + 0x5B0
[2] 0xFFFFFFFF787C6A04 sema_post + 0x628
[3] 0xFFFFFFFF787BD2F8 _msgrcv + 0x78
[4] 0xFFFFFFFF787C50FC msgrcv + 0x6C
[5] 0xFFFFFFFF7AF75F4C sqloTermEDUServices + 0x4A4
[6] 0xFFFFFFFF7AF7578C sqloInitEDUServices + 0x3EC
[7] 0xFFFFFFFF7AF713C4 __1cHDBGTerm6F_i_ + 0x19EC
[8] 0xFFFFFFFF7AF7256C sqloRunInstance + 0x5E4
[9] 0x0000000100006444 main + 0x924

2009-05-16-13.25.39.488101+330 I1447223A336 LEVEL: Event
PID : 16397 TID : 1 PROC : db2gds 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, trace services, pdInvokeCalloutScript, probe:10
START : Invoking /db2/db2bwd/sqllib/bin/db2cos from oper system services sqloEDUCodeTrapHandler

2009-05-16-13.25.39.488530+330 I1447560A803 LEVEL: Error (OS)
PID : 16397 TID : 1
FUNCTION: DB2 Common, OSSe, OSSHFile::write, probe:40
MESSAGE : ECF=0x90000028 No free space on the file system
CALLED : OS, -, write OSERR: ENOSPC (28)
ARG #1 : Hex, 4 bytes
0x4
ARG #2 : String, 430 bytes
2009-05-16-13.25.39.487997
PID : 16397 TID : 1 PROC : db2cos
INSTANCE: db2bwd NODE : 0
APPHDL : APPID: none
FUNCTION: oper system services, sqloEDUCodeTrapHandler, probe:999
EVENT : Invoking /db2/db2bwd/sqllib/bin/db2cos from oper system services sqloEDUCodeTrapHandler

ARG #3 : Hex, 8 bytes
0x1AE

2009-05-16-13.25.42.518179+330 I1448364A297 LEVEL: Event
PID : 16397 TID : 1 PROC : db2gds 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, trace services, pdInvokeCalloutScript, probe:20
STOP : Completed invoking /db2/db2bwd/sqllib/bin/db2cos

2009-05-16-13.25.42.521402+330 E1448662A510 LEVEL: Severe
PID : 16397 TID : 1 PROC : db2gds 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, oper system services, sqloEDUCodeTrapHandler, probe:10
MESSAGE : ADM0503C An unexpected internal processing error has occurred. ALL
DB2 PROCESSES ASSOCIATED WITH THIS INSTANCE HAVE BEEN SHUTDOWN.
Diagnostic information has been recorded. Contact IBM Support for
further assistance.

Regards,
Manju

You can't tell why the crash happened from the information you posted. You really need to open a pmr. If this is a brand new instance/db, then check that your OS has all the necessary patches as per IBM/DB2 recommendations and may also consider upgrading to the latest or more current fixpak.

Hi,

IF UNIX:
--------
I would like to know your file system spaces, i found that there is no space in your system, check the file system where you want to create the instance, also check the /tmp and other temporary file systems.

IF WINDOWS:
-------------
Check your drive space !!

THE MESSAGE FROM YOUR DB2DIAG.LOG:
---------------------------------------
2009-05-16-13.25.39.488530+330 I1447560A803 LEVEL: Error (OS)
PID : 16397 TID : 1
FUNCTION: DB2 Common, OSSe, OSSHFile::write, probe:40
MESSAGE : ECF=0x90000028 No free space on the file system
CALLED : OS, -, write OSERR: ENOSPC (28)
ARG #1 : Hex, 4 bytes
0x4
ARG #2 : String, 430 bytes
2009-05-16-13.25.39.487997
PID : 16397 TID : 1 PROC : db2cos
INSTANCE: db2bwd NODE : 0
APPHDL : APPID: none
FUNCTION: oper system services, sqloEDUCodeTrapHandler, probe:999
EVENT : Invoking /db2/db2bwd/sqllib/bin/db2cos from oper system services sqloEDUCodeTrapHandler

ARG #3 : Hex, 8 bytes
0x1AE

Hi,

I have enough disk space,The /db2/BWD/db2dump filesystem reaches 100% utilisation.If i delete the files in db2dump fiflesystem again it reaches 100% utilisation.I think database crashed only.I have online and offline backup.Th log archive file imported for which backup online or offline backup?

Please kindly help me..




Filesystem size used avail capacity Mounted on
/ 13G 7.8G 4.9G 62% /
/db2/BWD 148G 98G 48G 68% /db2/BWD
/db2/BWD/db2dump 9.8G 9.8G 0K 100% /db2/BWD/db2dump
/db2/BWD/log_dir 9.8G 4.6G 5.2G 47% /db2/BWD/log_dir
/dev 13G 7.8G 4.9G 62% /dev
/sapinstall 4.9G 4.6G 263M 95% /sapinstall
/sapmnt/BWD 4.9G 1.9G 3.0G 39% /sapmnt/BWD
/userlogs 4.9G 7.1M 4.9G 1% /userlogs
/usr/sap/BWD 20G 7.3G 12G 38% /usr/sap/BWD
/usr/sap/trans 15G 6.0G 8.7G 41% /usr/sap/trans
proc 0K 0K 0K 0% /proc
ctfs 0K 0K 0K 0% /system/contract
mnttab 0K 0K 0K 0% /etc/mnttab
objfs 0K 0K 0K 0% /system/object
swap 105G 376K 105G 1% /etc/svc/volatile
fd 0K 0K 0K 0% /dev/fd
swap 105G 4.7M 105G 1% /tmp
swap 105G 40K 105G 1% /var/run

Regards,
Manj

Hi,
The signal 6 shows the reason of crached.But i am unable to understand the reason.I gave the details.Please look into this and tell me the reason why the crached happened?

Awating for your reply...


2009-05-16-12.04.27.011106+330 E856460A917 LEVEL: Severe
PID : 20875 TID : 1 PROC : db2gds 0
INSTANCE: db2bwd NODE : 000
FUNCTION: DB2 UDB, oper system services, sqloEDUSIGCHLDHandler, probe:50
DATA #1 : <preformatted>
Detected the death of an EDU with process id 20912.
The signal number that terminated this process was 6.
Look for trap files (t20912.*) in the dump directory.
CALLSTCK:
[0] 0xFFFFFFFF7AF74C54 sqloSetNonCriticalEDUFlag + 0x4D4
[1] 0xFFFFFFFF787D2798 _thr_slot_offset + 0x5B0
[2] 0xFFFFFFFF787C6A04 sema_post + 0x628
[3] 0xFFFFFFFF787BD2F8 _msgrcv + 0x78
[4] 0xFFFFFFFF787C50FC msgrcv + 0x6C
[5] 0xFFFFFFFF7AF75F4C sqloTermEDUServices + 0x4A4
[6] 0xFFFFFFFF7AF7578C sqloInitEDUServices + 0x3EC
[7] 0xFFFFFFFF7AF713C4 __1cHDBGTerm6F_i_ + 0x19EC
[8] 0xFFFFFFFF7AF7256C sqloRunInstance + 0x5E4
[9] 0x0000000100006444 main + 0x924
Regards,
Manj

Hi,

----------------------------------------------------------
Signal #6
uname: SunOS R:5.10 V:Generic_127127-11 M:sun4u N:ndflbwd
process id: 20912
thread id : 1 (0x1)
</Header>
<SignalDetails>
<Siginfo_t length="256">
00000006 FFFFFFFF 00000000 00000000
000051B0 00000000 00000068 00000000
00000000 00000000 00000000 00000000
0009AE8F 00000016 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000
</Siginfo_t>
Signal #6 (SIGABRT): si_code is -1 (SI_LWP:user generated signal via lwp_kill()) si_pid is 20912, si_uid is 104, si_value is 00000000
</SignalDetails>
<ResourceLimits>
Data seg top [sbrk(0)] = 0x00000001001BC220
Cur data size (bytes) = 0xFFFFFFFFFFFFFFFD
Cur stack size (bytes) = 0x0000000000800000
Cur core size (bytes) = 0x0000000077359400
</ResourceLimits>
<Registers>
UCONTEXT: addr=ffffffff7fffc9b0 flags=000000000000002f
next=0000000000000000 stack=ffffffff7f800000
regset: ccr= 0000000000000098 pc= ffffffff787d3894
npc= ffffffff787d3898 gwins=0000000000000000
g: 0000000000000000 00000000000000a3 ffffffff76f16160 ffffffff7e167478
0000000000000000 0000000000000000 0000000000000000 ffffffff79100200
o: 0000000000000000 0000000000000006 0000000000000000 ffffffff788fbe3c
0000000000000005 ffffffff788fbe38 ffffffff7fffc591 ffffffff78770ec8
l: ffffffff788f6b94 0000000000000000 0000000000000020 0000000000000000
0000000000000001 0000000000000005 0000000000000000 0000000000000000
i: 0000000000000006 0000000000000006 0000000000000005 ffffffff788f8180
ffffffff79100200 0000000000000005 ffffffff7fffc641 ffffffff7874a5a4
SP==o6+2047==ffffffff7fffcd90 FP==i6+2047==ffffffff7fffce40 SavePC=i7
PC location: _lwp_kill + 0x8
Object file: /lib/sparcv9/libc.so.1 (offset 0xd3894)
</Registers>
<StackTrace>
##### Object: /lib/sparcv9/libc.so.1
_lwp_kill(0x6,0x6,0x5,0xffffffff788f8180,0xfffffff f79100200,0x5) + 0x8
raise(0x6,0x6,0x5,0xffffffff788f8180,0xffffffff791 00200,0x5) + 0x10
abort(0x1,0x0,0xffffffff7af9a4e0,0x1a1b20,0x0,0x0) + 0xd0
##### Object: /opt/IBM/db2/V9.1/lib64/libdb2e.so.1
sqloExitEDU(0x1,0xffffffffffffffff,0x0,0x0,0x0,0x1 8780000) + 0x580
__1cKsqle_panic6F_v_(0x92e0,0xdb2cafe,0x0,0x9000,0 x1a,0xffffffff7d7f1700) + 0x140
__1cXsqlbClnrAsyncWriteSetup6FpvpnLSQdDLO_IO_REQdD __v_(0x2207890e8,0xffffffff76b06130,0x18100001,0x1 810000a,0xc00,0x18100000) + 0xa1c
__1cUSQdDLO_LIO_HANDLE_DATATsqloDispatchNBlocks6Mp pnLSQdDLO_IO_REQdD_LpknWSQdDLO_LIO_CALLBACK_INFO_p nYSQdDLO_LIO_DISPATCH_STATUS_nQSQdDLO_LIO_OP_TYPE_ _i_(0xffffffff76b04600,0xffffffff76b04640,0x20,0x2 20789128,0x18780000,0x0) + 0xe0
__1cXsqlbClnrDispatchSomeAIO6FpnMSQdDLB_CLNR_CB_pp nLSQdDLO_IO_REQdD_L_v_(0x0,0x0,0xe40,0x220788fd8,0 xffffffff7d7f1700,0xffffffff7d8856f0) + 0x98
__1cSsqlbClnrEntryPoint6FpCI_v_(0xffffffff76b04640 ,0x2207890e8,0x220788fd8,0x0,0xffffffffffffffff,0x 2) + 0x584
__1cNsqloCreateEDU6FpFpcI_v0LpnNSQdDLO_EDU_INFO_pi _i_(0xffffffff7a856990,0xffffffff7fffe068,0x10,0xf fffffff7e167478,0xffffffff7fffe04c,0xffffffff7fffe 290) + 0x228
sqloTermEDUServices(0x0,0xffffffff7dff6610,0x288,0 x18780000,0x0,0xffffffff7fffe050) + 0x590
sqloInitEDUServices(0xffffffff7dfe3718,0x0,0x2,0x0 ,0x12fd0,0xffffffff7d7f1700) + 0x3ec
__1cHDBGTerm6F_i_(0x1e00c0,0x100006600,0x100008960 ,0xc76,0x20004019c,0x2) + 0x19ec
sqloRunInstance(0x100006600,0x100008960,0x0,0x2000 41040,0x20004019c,0x0) + 0x5e4
##### Object: /db2/db2bwd/sqllib/adm/db2sysc
main(0x1,0x200041ac6,0xffffffff7ffffc35,0x188,0xff ffffff7e1473d8,0x10010db70) +Regards,
Manj

I would do exactly what the message tells you to do: "Contact IBM Support for further assistance."

Hi,

Did you performed any fixpack installation ?, if yes, was the fix pack applied successfully, can you issue ' errpt | more' and paste the command output, is the paging availability enough. The log will be available with online db backups only.

db2pclnr (page cleaner) is the process that received the signal. Some error was encountered when db2pclnr was in process of writing pages from the bufferpool to disk. As a result of this error, db2 called sqle_panic to shutdown itself, this is done to protect database integrity. So, what you need to check is the errors reported in the db2diag.log just prior to the crash (prior to ADM0503C). But the best is to open a pmr.