When i made one Node ( node number 2 non catalog node ) down and taking backups by giving rest of the partitionnums backup are failing intermittently
following it the db2diag content i am getting inresponse of failure.


2011-11-28-16.11.41.193133+330 E419091E511 LEVEL: Error\par
PID : 12485 TID : 47804865046848PROC : db2sysc 0\par
INSTANCE: db2inst1 NODE : 000 DB : AVAMAR1\par
APPHDL : 0-477 APPID: *N0.db2inst1.111128104139\par
AUTHID : DB2INST1\par
EDUID : 2803 EDUNAME: db2agent (AVAMAR1) 0\par
FUNCTION: DB2 UDB, database utilities, sqlubkdReceiveReply, probe:999\par
MESSAGE : ZRC=0x81580016=-2124939242=SQLKD_NODE_FAILURE\par
"Mapping for SQLKF_NODE_FAILED"\par
\par
2011-11-28-16.11.41.193275+330 E419603E509 LEVEL: Error\par
PID : 12485 TID : 47804865046848PROC : db2sysc 0\par
INSTANCE: db2inst1 NODE : 000 DB : AVAMAR1\par
APPHDL : 0-477 APPID: *N0.db2inst1.111128104139\par
AUTHID : DB2INST1\par
EDUID : 2803 EDUNAME: db2agent (AVAMAR1) 0\par
FUNCTION: DB2 UDB, database utilities, sqlubMPPRequestor, probe:583\par
MESSAGE : ZRC=0x81580016=-2124939242=SQLKD_NODE_FAILURE\par
"Mapping for SQLKF_NODE_FAILED"\par
\par
2011-11-28-16.11.41.193348+330 E420113E522 LEVEL: Error\par
PID : 12485 TID : 47804865046848PROC : db2sysc 0\par
INSTANCE: db2inst1 NODE : 000 DB : AVAMAR1\par
APPHDL : 0-477 APPID: *N0.db2inst1.111128104139\par
AUTHID : DB2INST1\par
EDUID : 2803 EDUNAME: db2agent (AVAMAR1) 0\par
FUNCTION: DB2 UDB, database utilities, sqlubMPPRequestor, probe:1184\par
MESSAGE : SQL1170N The operation cannot continue because database partition\par
"2" is not available.\par


It seems DB2 trying to connect to node number 2 also although i didn't give the nodenumber 2 while firing the backup.
As it's happening intermittently Please let me know what can be the cause for this.

in latest version of DB2 if backup db is started against catalog node, all nodes will do backup. see doc for specifications..
you can still take backup from individual node(s) - also see doc...

yes! all node can take backup! but i want to take the backup for specific nodes only !
i forcefully made one node down! and firing backup for rest of the partitions
N.B. catalog node is Active. And i am firing backup from catalog node only.

yes this is possible
take a look in infocenter
IBM DB2 9.5 Information Center for Linux, UNIX, and Windows
for details : backup command...

yes i know this is possible ! and i am firing for selected node only (nodes which are not down ) but it's failing with error please read the first thread for full information.

We are supporting IBM DB2 9.5, and 9.7 in our backup software.

We are having the following DB2 setup:

OS: Red Hat Enterprise Linux 5 x86_64

DB2: DB2 v9.5.0.0

We have configured a DPF setup with two hosts, total 4 database partitions
- 0,1,2 and 3.

Node 0 is the catalog node.

Host 1 is having partitions 0 and 1
Host 2 is having partitions 2 and 3

We have 1 database - AVAMAR1

We have brought down partition 3 using - db2stop NODENUM 3.

After this we tried to take ONLINE backup of remaining partitions 0,1 and 2 of AVAMAR1 database from command line.

db2 'backup db AVAMAR1 on all dbpartitionnums except dbpartitionnum (3) online to /tmp'

I tried the same with following command also:

db2 'backup db AVAMAR1 on dbpartitionnum(0,1,2) online to /tmp';

We observed that backup failed for AVAMAR1 and observed the many errors in db2diag.log

However, the below error is of particular interest to us:

MESSAGE : SQL1170N The operation cannot continue because database partition
"3" is not available.

It seems DB2 is trying to connect to partition number 3, although we didn't give the partition 3 while firing the backup.


Could you please point to what might be going wrong here.

We are observing the same behavior with DB2 9.7 on RHEL 5 x86_64 as well.

N.B. : LOGRETAIN is ON for this database , and offline backup is already taken on all the partitions.

Start the online backup of nodes 0 1 2, when all db nodes are up and running.

I tried to backup a db in a DPF env (3 logical partitions on one host) with one of the partitions stopped and it worked for me. The backup command has to be executed from the catalog partition. I was using v9.7 FP5.


bk@host /home/bk/SSV > db2 "backup db ssv on dbpartitionnums (1,2,3)"
Part Result
---- ------------------------------------------------------------------------
0001 DB20000I The BACKUP DATABASE command completed successfully.
0002 DB20000I The BACKUP DATABASE command completed successfully.
0003 DB20000I The BACKUP DATABASE command completed successfully.

Backup successful. The timestamp for this backup image is : 20111205204524



bk@host /home/bk/SSV > db2 "backup db ssv on dbpartitionnums (1,2)"
Part Result
---- ------------------------------------------------------------------------
0001 DB20000I The BACKUP DATABASE command completed successfully.
0002 DB20000I The BACKUP DATABASE command completed successfully.

Backup successful. The timestamp for this backup image is : 20111205204600



bk@host /home/bk/SSV > db2stop nodenum 3
12/05/2011 20:46:25 3 0 SQL1064N DB2STOP processing was successful.
SQL1064N DB2STOP processing was successful.



bk@host /home/bk/SSV > db2 "backup db ssv on dbpartitionnums (1,2)"
Part Result
---- ------------------------------------------------------------------------
0001 DB20000I The BACKUP DATABASE command completed successfully.
0002 DB20000I The BACKUP DATABASE command completed successfully.

Backup successful. The timestamp for this backup image is : 20111205204631



bk@host /home/bk/SSV > db2start nodenum 3
12/05/2011 20:46:52 3 0 SQL1063N DB2START processing was successful.
SQL1063N DB2START processing was successful.



bk@host /home/bk/SSV > db2stop nodenum 2
12/05/2011 20:47:02 2 0 SQL1064N DB2STOP processing was successful.
SQL1064N DB2STOP processing was successful.


bk@host /home/bk/SSV > db2 "backup db ssv on dbpartitionnums (1,3)"
Part Result
---- ------------------------------------------------------------------------
0001 DB20000I The BACKUP DATABASE command completed successfully.
0003 DB20000I The BACKUP DATABASE command completed successfully.

Backup successful. The timestamp for this backup image is : 20111205204709

Thanks Prashant,
We already tried case suggested by you and that is always working!
We want to check the status for the case when on node is down, as mention in the problem.

db2girl:
I am facing this issue intermittently for me chances are 10:1, i observe the same behavior on another setup also.(You may hit this problem if you try it for some time)

Are you sure the partition is still up when you receive the error? Use ps to check if db2sysc is running for that partition. If you think it's a bug, open a pmr.

db2girl, the OP is doing online backups. Your tests are offline.

Pankaj, Can you set your diaglevel to 4 to see if you can any other useful info.

db2girl: can you please try online backup with same setup!