DB2 v220.127.116.11 special_27031 DPF 7 partitions for SAP BI system.
Linux RHEL 6 with vxfs db filesystems on Dell Compellent. Vendor has checked os and vxfs logs and there is no error on hardware level.
Yesterday we spotted few errors during data load - rewriting data between tables:
2012-10-29-18.104.22.1689813+060 I1069604752E3625 LEVEL: Severe
PID : 60811 TID : 140724284679936PROC : db2sysc 2
INSTANCE: db2rbp NODE : 002
EDUID : 1363 EDUNAME: db2pfchr (RBP) 2
FUNCTION: DB2 UDB, buffer pool services, sqlb_verify_page, probe:2
MESSAGE : ZRC=0x86020001=-2046689279=SQLB_BADP "page is bad"
DIA8400C A bad page was encountered.
...... rest in attachment.
I saw APAR: IBM IC83088: BAD PAGE ENCOUNTERED DURING REDO OF SQLD_LR_REFRESH_TCB PRECEDED BY ALTER TABLESPACE OPERATIONS - United States
but there are no trap files and no SQLD_LR_REFRESH_TCB entries.
Unfortunately we don't have possibility to report error directly to IBM as SAP OEM customer and SAP support is asking us to perform full db2dart scan - db has 1,44 TB and it will take ages to finish it .
Is there any other solution to find the reason or fix ?
You need to open a pmr. Get db2dart (offline) or inspect (online) for tablespace ID 22 on node 0. Find out the table name (based on tablespace ID 22 and table ID 143) and try to query it (select * from <table name>). Note that it can crash the instance again.
Two attempts to read a page failed - I suspect there is some problem on disk, use db2dart/inspect to verify. The trap file should be in /db2/RBP/db2dump/FODC_BadPage_2012-10-29-08.34.51.612226_0000. The partial stack is dumped into the db2diag.log (sqldTableStats - collects statistics). I don't think this issue is related to the APAR you found.
I've done db2 inspect yesterday ( 2 tables are affected )and they asked for db2dart for repair procedure. Don't know if this call will reach IBM, so if anyone want to look into i'm attaching results.
One of the files show data corruption for the object in your attached db2diag.log. The second file - you specified an invalid tablespace ID #2
The dumped page looks all zeroed out. IBM can provide suggestions on how to resolve this (ie. check if you have a good backup image to restore from, initialize a bad page...dumped page has no values based on this report). You need to get help from IBM.
Sorry for mess. I'm not so fluent in DPF and taken 000 instead of 002 and 003
Now you have proper ones.
And yes, we are asking SAP to pass issue to IBM, but it looks like SAP wants to do everything by himself
As a fellow IBM-er, this makes me little angry ....