I'm running an old DB2 v7.1 AIX 4.3. I have 2 servers that are working one as primary and the other one secondary using data propagator.
On the secondary sever, since I'm still tunning up my data propagator, I used to 3 times a week restore full backup from the promary server (production) into it.
It has been working fine until last wednesday. Restored finished aparently normal (no outstanding messages) and when I try to apply logs (rollforward as I always do since we just have online backups) the log ended with error and dumps...
So I decided to start from scratch. I dropped the database on the secondary server, created one empty again ( I did this before 3 times and worked fine) and restored...
Before I dropped, recreated and restore again, I ran a DB2DART at database level and I got 2 errors with 2 tables regarding indexes...
so this is the db2dart report:
---------
Table inspection start: DB2OWNER.SOPS_OT_RESALE

Data inspection phase start. Data obj: 275 In pool: 4
Data inspection phase end.


Index inspection phase start. Index obj: 1035 In pool: 5
Scanning pages for unique index itoken(1) root page:83266p.
Scanning pages for index itoken(2) root page:89061p.
Scanning pages for index itoken(3) root page:113494p.
Index inspection phase end.

Table inspection end.

Table inspection start: DB2OWNER.SOPS_GROUP_T

Data inspection phase start. Data obj: 276 In pool: 4
Data inspection phase end.


Index inspection phase start. Index obj: 1036 In pool: 5
Scanning pages for unique index itoken(1) root page:85986p.
Scanning pages for index itoken(2) root page:85987p.
Scanning pages for index itoken(3) root page:85988p.
Scanning pages for index itoken(4) root page:85989p.
Scanning pages for index itoken(5) root page:104924p.
Error: last key on page 806764p,entries=127 is higher than the highkey on page=119196p,slot=62,rid=4294967295,entries=101.
Error: Index keys misordered, compare rc=1. Lvl(2) Page,slot,rid current:814952p,0,x303E0527 prev:806764p,126,x2D1BDC13.
2 keyErrs found in index itoken(5) root page:104924p.
Total key errors found: 2.
Error detected in index.
Index inspection phase end.

Table inspection end.

Table inspection start: DB2OWNER.STAR_POM_PRODUCTS

Data inspection phase start. Data obj: 1547 In pool: 4
Data inspection phase end.


Index inspection phase start. Index obj: 1302 In pool: 5
Scanning pages for unique index itoken(1) root page:585378p.
Error: last key on page 806933p,entries=83 is higher than the highkey on page=666707p,slot=21,rid=4294967295,entries=63.
Error: Index keys misordered, compare rc=1. Lvl(2) Page,slot,rid current:814148p,0,x2FB32E06 prev:806933p,82,x1814D602.
Error: last key on page 806926p,entries=97 is higher than the highkey on page=769061p,slot=19,rid=4294967295,entries=47.
Error: Index keys misordered, compare rc=1. Lvl(2) Page,slot,rid current:814149p,0,x3044843C prev:806926p,96,x2E0FFD46.
4 keyErrs found in index itoken(1) root page:585378p.
Total key errors found: 4.
Error detected in index.
Index inspection phase end.

Table inspection end.

-------

Well supposing I didn't apply the logs yet...this could be a problem in the original backup(production database)?

So I droppped, recreated and restored again...my db2diag.log during the restore (level 4) showed me this (didn't apply the logs yet).

---------------
007-08-03-10.04.55.032137 Instance:db2owner Node:000
PID:69676(db2bm.9678.0) Appid:none
oper_system_services sqloNLCKUnlock Probe:5

0000 0016 ....


2007-08-03-10.24.20.588770 Instance:db2owner Node:000
PID:45122(db2bp) Appid:*LOCAL.db2owner.070802180245
oper_system_services sqlo_ssem_set_v Probe:2

Data TitleQO semop=> errno: PID:45122 Node:000
0000 0016 ....


Data TitleQO semop=> pSSemID: PID:45122 Node:000
0000 0000 ....


Data TitleQO semop=> pSSemSetID: PID:45122 Node:000
002c 002d .,.-


2007-08-03-10.24.21.123819 Instance:db2owner Node:000
PID:45122(db2bp) Appid:*LOCAL.db2owner.070802180245
oper_system_services sqlo_ssem_set_v Probe:2

Data TitleQO semop=> errno: PID:45122 Node:000
0000 0016 ....


Data TitleQO semop=> pSSemID: PID:45122 Node:000
0000 0000 ....


Data TitleQO semop=> pSSemSetID: PID:45122 Node:000
002c 002d .,.-


Data Title:Abnormal connect termination; semV err OK PID:45122 Node:000

-----

This could be a problem with the backup files?
Could be a problem with my tablespaces (most of them are raw containers..should I delete them and recreate them before restoring)?
Could be a problem with my production database that generates bad backups?

I'm under a lot of pressure to make this work, since a whole Data Warehouse team uses the secondary database to ETL (informatica) and they are stuck waiting on me.
Also this is an unsupported version...so any help in how to fix, or debug I would truly appreciate..


Thanks

Have you tried upgrading to V8 or V9? V7 is out of service for more than 1.5 years already.

Hi,
can you write commands you have executed.
Grofaty

Hi,
not to be offensive, but this is classical "property software" problem. Constant migration is required although new functionalities are not required by the company. If someone has a problem with unsupported version of product this is his own problem... I don't wonder any more why open source programs are rising up in popularity.

There is also another problem: systems are becoming more and more complex to migrate and sometimes there is no way of simple migration. I have been a victim (in a soft meaning of the word) in one migration where software on multiple hardware computers had to be migrated to new version just because of migrating one software to new version. Very costly and time consuming... I just get filling that property software is becoming to much complex... or rephrase 'property software' has a complex solutions to complex problems, but open source can never have a complex solution, because it is just made up with small fractions of code from thousandth of different projects... so simple solutions to complex problems are required... What is even more interesting when you buy a property software you need to install it, fix it, patch it and read tones of documents to make it work. Open source software is just downloaded the last available version (witch already has all fixes included) installing it and it just works...
Grofaty

Thanks all of you.
I know that this DB2 is out of support and in my previous job I had couple v7.1 instances and many v8.1 and v8.2. I had a serious problem with one of the v7.1 and even paying for support IBM couldn't help me much (nothing actually).
So when I joined to this new job I knew what I was stepping on ...it is a thin ice ready to break...
The company that I work for is negotianing new contracts for upgrading to the latest version. I will be in charge to move from v7 to v8 and then v9..

Thanks again

I don't want to start a different topic, but your argument that open source is better in this respect doesn't hold much water either, as you well know.

p.s: I'm typing this on a Linux system (and I'm using Linux for 13 years now).

Did the problem get resolved? If so, how?