I have a situation where one of my tablespaces were offline. The unix admin says he repaired the logical volumn and I should be able to used it now. However, when I tried to switch the tablespace online, I still got an error. I have verified that I can read from the container, by using the dd command. I have attached a snippet of the trace log below, up to the place where the error occurs. Can someone please suggest what maybe causing this error? Is it still a problem with the container? I'm not sure what steps the unix admin took to repair the logical volumn, so could it be the logical volumn is still damaged? The problem is with container /dev/rcdrslv15, which you will see below.

Thanks

4692 DB2 cei_data oper_system_services sqloopenp (1.25.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 2
2f64 6576 2f72 6364 7273 6c76 3134 0000 /dev/rcdrslv14..
007c 0000 0180 .|....

4693 DB2 cei_data oper_system_services sqloopenp (1.25.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 7
0000 004c ...L

4694 DB2 cei_retcode oper_system_services sqloopenp (1.21.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4695 DB2 cei_entry oper_system_services sqloseek (1.20.15.489)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4696 DB2 cei_data oper_system_services sqloseek (1.25.15.489)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 004c 0000 0000 0000 0200 0000 0000 ...L............

4697 DB2 cei_retcode oper_system_services sqloseek (1.21.15.489)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4698 DB2 cei_entry oper_system_services sqloread (1.20.15.110)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4699 DB2 cei_data oper_system_services sqloread (1.25.15.110)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 004c 0000 0200 ...L....

4700 DB2 cei_retcode oper_system_services sqloread (1.21.15.110)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4701 DB2 cei_entry oper_system_services sqlogins (1.20.15.58)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4702 DB2 cei_retcode oper_system_services sqlogins (1.21.15.58)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4703 DB2 cei_entry oper_system_services sqloclose (1.20.15.21)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4704 DB2 cei_data oper_system_services sqloclose (1.25.15.21)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 004c ...L

4705 DB2 cei_retcode oper_system_services sqloclose (1.21.15.21)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4706 DB2 cei_entry oper_system_services sqlofmblk (1.20.15.41)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4707 DB2 cei_data oper_system_services sqlofmblk (1.25.15.41)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
200a c838 ..8

4708 DB2 cei_retcode oper_system_services sqlofmblk (1.21.15.41)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4709 DB2 fnc_retcode buffer_pool_services sqlbDMSDoContainerOp (1.31.2.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4710 DB2 fnc_entry buffer_pool_services sqlbDMSDoContainerOp (1.30.2.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4711 DB2 cei_entry oper_system_services sqlogmblk (1.20.15.60)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4712 DB2 cei_data oper_system_services sqlogmblk (1.25.15.60)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 0000 0000 03ff 0000 0000 ............

4713 DB2 cei_entry oper_system_services MemPrivatePool (1.20.15.259)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4714 DB2 cei_retcode oper_system_services MemPrivatePool (1.21.15.259)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4715 DB2 cei_data oper_system_services sqlogmblk (1.25.15.60)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 250
200a c838 ..8

4716 DB2 cei_retcode oper_system_services sqlogmblk (1.21.15.60)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4717 DB2 cei_entry oper_system_services sqloopenp (1.20.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4718 DB2 cei_data oper_system_services sqloopenp (1.25.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 2
2f64 6576 2f72 6364 7273 6c76 3135 0000 /dev/rcdrslv15..
007c 0000 0180 .|....

4719 DB2 cei_data oper_system_services sqloopenp (1.25.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 7
0000 004c ...L

4720 DB2 cei_retcode oper_system_services sqloopenp (1.21.15.97)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4721 DB2 cei_entry oper_system_services sqloseek (1.20.15.489)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4722 DB2 cei_data oper_system_services sqloseek (1.25.15.489)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 004c 0000 0000 0000 0200 0000 0000 ...L............

4723 DB2 cei_retcode oper_system_services sqloseek (1.21.15.489)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4724 DB2 cei_entry oper_system_services sqloread (1.20.15.110)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4725 DB2 cei_data oper_system_services sqloread (1.25.15.110)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 004c 0000 0200 ...L....

4726 DB2 cei_retcode oper_system_services sqloread (1.21.15.110)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4727 DB2 cei_entry oper_system_services sqlogins (1.20.15.58)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4728 DB2 cei_retcode oper_system_services sqlogins (1.21.15.58)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4729 DB2 cei_entry oper_system_services sqloclose (1.20.15.21)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

4730 DB2 cei_data oper_system_services sqloclose (1.25.15.21)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 1
0000 004c ...L

4731 DB2 cei_retcode oper_system_services sqloclose (1.21.15.21)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 254
rc = 0

4732 DB2 fnc_entry buffer_pool_services sqlbLogErr (1.30.2.228)
pid 68978; tid 1; node 0; cpid 77878; sec 0; nsec 0; tpoint 0

Traces are tricky for non-IBM staff to read, since the trace points map to source code markers we don't have access to. Sometimes there's bits that give you clues.

What about the db2diag.log (using DIAGLEVEL 4), is there an errno entry or anything like that? What's the error you see when you try to access the tablespace?

Here is the db2diag.log. How can I tell if some of the db2 files are corrupt? This is after the unix admin had repaired /dev/rcdrslv15.

Thanks

2004-07-12-11.52.55.119975 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSDoContainerOp Probe:840 Database:RECORDS
DIA9999E An internal error occurred. Report the following error code :
"FFFFC11E".

2004-07-12-11.52.55.151362 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSDoContainerOp Probe:840 Database:RECORDS

Error checking container 2 (/dev/rcdrslv15) for tbsp 1. Rc = FFFFC11E

Data TitleQLB_CONTAINER_TAG PID:68978 Node:000
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................


2004-07-12-11.52.56.118764 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSStartPool Probe:800 Database:RECORDS
DIA9999E An internal error occurred. Report the following error code :
"FFFFC11E".

2004-07-12-11.52.56.143531 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSStartPool Probe:800 Database:RECORDS

Tablespace 1 (OCCSPACE2)


2004-07-12-11.52.56.243537 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbStartPools Probe:30 Database:RECORDS

Tablespace OCCSPACE2 (1) is in state x4000.
Starting it failed. rc=ffff c11e ÿÿÁ.


2004-07-12-11.53.23.317461 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSDoContainerOp Probe:840 Database:RECORDS
DIA9999E An internal error occurred. Report the following error code :
"FFFFC11E".

2004-07-12-11.53.23.343995 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSDoContainerOp Probe:840 Database:RECORDS

Error checking container 2 (/dev/rcdrslv15) for tbsp 1. Rc = FFFFC11E

Data TitleQLB_CONTAINER_TAG PID:68978 Node:000
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................
0000 0000 0000 0000 0000 0000 0000 0000 ................


2004-07-12-11.53.24.311352 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSStartPool Probe:800 Database:RECORDS
DIA9999E An internal error occurred. Report the following error code :
"FFFFC11E".

2004-07-12-11.53.24.336114 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbDMSStartPool Probe:800 Database:RECORDS

Tablespace 1 (OCCSPACE2)


2004-07-12-11.53.24.436113 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbSwitchTablespaceOnline Probe:881 Database:RECORDS

Tablespace OCCSPACE2 (1) will remain offline. State = 0x00004000.

2004-07-12-11.53.24.544691 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
buffer_pool_services sqlbSwitchTablespaceOnline Probe:881 Database:RECORDS
DIA9999E An internal error occurred. Report the following error code :
"FFFFC11E".

2004-07-12-11.53.24.569960 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
global_services sqlzerdm Probe:20 Database:RECORDS

ffff c11e ÿÿÁ.


2004-07-12-11.53.24.669948 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
global_services sqlzerdm Probe:30 Database:RECORDS

0000 0000 0000 0000 0000 0000 0000 0040 ...............@


2004-07-12-11.53.25.182154 Instance:db2fudd Node:000
PID:68978(db2agent (RECORDS)) Appid:*LOCAL.db2fudd.040712150744
access_plan_manager sqlra_compile_var Probe:20 Database:RECORDS
DIA9999E An internal error occurred. Report the following error code :
"FFFFC11E".

FFFFC11E just means not accesible.

But - the dump of the container tag would have me concerned, it's all zero'd out, so even though the lv is there is it possible the data that was originally there is not?

Is there any way to correct this ? I would be satisfied with just getting the tablespace online. Is there a way to drop the tables within the tablespace or the tablespace itself?

I believe you can drop the tablespace (and implicitly all tables in it), as long as there isn't any referential integrity between tables in it and other tablespaces. If there is, I believe db2 will prevent you from doing the drop, but I'd had to confirm that.

Best option would be to restore if possible, if you're still having trouble with that LV use a redirected restore to stick the data elsewhere.

It appears that having RI doesn't prevent the tablespace drop, it just deactivates the constraints.

try to use db2dart

regards,

mujeeb

It looks like I'm in a catch 22, because the tablespace has a table in it that has indexes in a different tablespace. I can't drop the offline tablespace because the table has parts in a different tablespace and I can't drop the indexes because the table's tablespace is offline. Is there anyway to get around this situation?