Environment:

Solaris 8
IBM Informix Dynamic Server Version 9.40.FC6
Netbackup 4.5 FP_6

I'm trying to recover a production database in a test environment.

My backup has no problems but when I try to restore in bar_act.log I have the following problem.

************************************************** ******
2009-04-14 10:29:44 16858 16856 Begin cold level 0 restore rootdbs (Storage Mannager copy ID: 1 1237915000).
2009-04-14 10:29:44 16858 16856 ASSERT: file bar_unix.c line 1253 - contact product support
2009-04-14 10:29:44 16858 16856 See also: /tmp/core.16858
2009-04-14 10:31:53 16858 16856 Process 16858 received signal 11. Process will exit after cleanup.
************************************************** *******

But the log in Netbackup shows me that the recover of rootdbs was succesfull and continue restoring all dbspaces.

But when I tried to recover logs it shows me the following error
************************************************** *****
2009-05-20 20:00:10 5575 5573 Unable to start the logical log restore: plogdbs which contains a logical log or physical log file
is being restored and rootdbs has not been physically restored.
If the ROOT DBspace or a DBspace containing the physical log or a logical
log file is restored, then the ROOT DBspace and all DBspaces containing
a logical log file or physical log file must be physically restored..
2009-05-20 20:00:10 5575 5573 /informix/bin/onbar_d complete, returning 131 (0x83)
************************************************** ***


Thanks in advance for your help

Cecilia

It looks like.

IC51992: ONBAR CAN CRASH WITH UNFAVOURABLE GL_DATETIME SETTING

APAR status
Closed as program error.

Error description

Any GL_DATETIME setting that yields more than the standard 19
character length datetime string can cause onbar to run into a
segmentation violation and/or write a core dump.
Sample:
$ export GL_DATETIME="%Y-%m-%d %H:%M:%S%F3"
$ onbar -b -l
$ onbar -m
...
2007-02-27 15:07:25 13535 13533
/sqldists/10.00.FC5W4/bin/onbar_d -b -l
2007-02-27 15:07:27 13535 13533 Begin backup logical log 186.
2007-02-27 15:07:27 13535 13533 Successfully connected to
Storage Manager.
2007-02-27 15:07:30 13535 13533 ASSERT: file bar_unix.c line
1261 - contact product support
2007-02-27 15:07:30 13535 13533 See also: /tmp/core.13535
2007-02-27 15:07:30 13535 13533 (-43080) Process 13535
received signal 11. Process will exit after cleanup.

Local fix

Put a line at the beginning of the $INFORMIXDIR/bin/onbar script
:
unset GL_DATETIME

Problem summary

"Using ON-Bar and non-default GLS settings, especially GL_DATETI

Problem conclusion

"Problem first fixed in IDS 10.00.xC7."

And did you point to right XBSA librariy?

Hi,

Thanks for your answer.
I have already read this post, but I don´t use that environment variable.
Actually I didn´t find in Internet the exact error that bar_act.log is sending. I look for similar errors, (bar_unix.c with different line programs) like the post you are sending me but no one gives a real solution.




I don´t understand exactly what you mean with your last post. Could you explain me what exactly I have to check?

Again I really appreciate your help.

Thanks

Ana Cecilia

Post onconfig's from your production server and test server.

Sorry for delay, I was trying a fresh environment but the error it's same.

Production and test onconfig are the same. I made a cold backup of /informix and restore it in test environment.

I did this procedure 3 years ago and works. I'm completely lost





# Root Dbspace Configuration

ROOTNAME rootdbs # Root dbspace name
ROOTPATH /informix/links/rootdbs01 # Path for device containing root dbspace
ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes)
ROOTSIZE 512000 # Size of root dbspace (Kbytes)

# Disk Mirroring Configuration Parameters

MIRROR 0 # Mirroring flag (Yes = 1, No = 0)
MIRRORPATH # Path for device containing mirrored root
MIRROROFFSET 0 # Offset into mirrored device (Kbytes)

# Physical Log Configuration

PHYSDBS plogdbs # Location (dbspace) of physical log
PHYSFILE 256000 # Physical log file size (Kbytes)

# Logical Log Configuration

LOGFILES 104 # Number of logical log files
LOGSIZE 30000 # Logical log size (Kbytes)

# Diagnostics

MSGPATH /informix/online.log # System message log file path
CONSOLE /dev/console # System console message path

# To automatically backup logical logs, edit alarmprogram.sh and set
# BACKUPLOGS=Y
ALARMPROGRAM /informix/etc/alarmprogram.sh # Alarm program path
TBLSPACE_STATS 1 # Maintain tblspace statistics

# System Archive Tape Device

TAPEDEV /dev/null # Tape device path
TAPEBLK 32 # Tape block size (Kbytes)
TAPESIZE 40000000 Maximum amount of data to put on tape (Kbytes)

# Log Archive Tape Device

LTAPEDEV /libre/logs # Log tape device path
LTAPEBLK 16 # Log tape block size (Kbytes)
LTAPESIZE 10240 # Max amount of data to put on log tape (Kbytes)
# Optical

STAGEBLOB # Informix Dynamic Server staging area

# System Configuration

SERVERNUM 0 # Unique id corresponding to a OnLine instance
DBSERVERNAME baan_srv # Name of default database server
DBSERVERALIASES sfodbc # List of alternate dbservernames
#NETTYPE ipcshm,5,100,CPU # Configure poll thread(s) for nettype
NETTYPE ontlitcp,7,250,NET # Configure poll thread(s) for nettype
DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed env.
RESIDENT -1 # Forced residency flag (Yes = 1, No = 0)

MULTIPROCESSOR 1 # 0 for single-processor, 1 for multi-processor
NUMCPUVPS 10 # Number of user (cpu) vps
SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps to one

NOAGE 0 # Process aging
AFF_SPROC 0 # Affinity start processor
AFF_NPROCS 0 # Affinity number of processors

# Shared Memory Parameters

LOCKS 2000000 # Maximum number of locks
BUFFERS 2000000 # Maximum number of shared buffers
NUMAIOVPS 10 # Number of IO vps
PHYSBUFF 128 # Physical log buffer size (Kbytes)
LOGBUFF 128 # Logical log buffer size (Kbytes)
CLEANERS 20 # Number of buffer cleaner processes
SHMBASE 0x10A000000L # Shared memory base address
SHMVIRTSIZE 5000000 # initial virtual shared memory segment size
SHMADD 512000 # Size of new shared memory segments (Kbytes)
SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL 1800 # Check point interval (in sec)
LRUS 20 # Number of LRU queues
LRU_MAX_DIRTY 2 # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 1.5 # LRU percent dirty end cleaning limit
TXTIMEOUT 300 # Transaction timeout (in sec)
STACKSIZE 64 # Stack size (Kbytes)


DYNAMIC_LOGS 2
LTXHWM 50
LTXEHWM 60

# System Page Size
# BUFFSIZE - OnLine no longer supports this configuration parameter.
# To determine the page size used by OnLine on your platform
# see the last line of output from the command, 'onstat -b'.


# Recovery Variables
# OFF_RECVRY_THREADS:
# Number of parallel worker threads during fast recovery or an offline restore.
# ON_RECVRY_THREADS:
# Number of parallel worker threads during an online restore.

OFF_RECVRY_THREADS 10 # Default number of offline worker threads
ON_RECVRY_THREADS 1 # Default number of online worker threads

# Data Replication Variables
DRINTERVAL 30 # DR max time between DR buffer flushes (in sec)
DRTIMEOUT 30 # DR network timeout (in sec)
DRLOSTFOUND /informix/etc/dr.lostfound # DR lost+found file path

# CDR Variables
CDR_EVALTHREADS 1,2 # evaluator threads (per-cpu-vp,additional)
CDR_DSLOCKWAIT 5 # DS lockwait timeout (seconds)
CDR_QUEUEMEM 4096 # Maximum amount of memory for any CDR queue (Kbytes)
CDR_NIFCOMPRESS 0 # Link level compression (-1 never, 0 none, 9 max)
CDR_SERIAL 0 # Serial Column Sequence
CDR_DBSPACE # dbspace for syscdr database
CDR_QHDR_DBSPACE # CDR queue dbspace (default same as catalog)
CDR_QDATA_SBSPACE # List of CDR queue smart blob spaces

# CDR_MAX_DYNAMIC_LOGS
# -1 => unlimited
# 0 => disable dynamic log addition
# >0 => limit the no. of dynamic log additions with the specified value.
# Max dynamic log requests that CDR can make within one server session.

CDR_MAX_DYNAMIC_LOGS 0 # Dynamic log addition disabled by default

# Backup/Restore variables
BAR_ACT_LOG /informix/bar_act.log # ON-Bar Log file - not in /tmp please
BAR_DEBUG_LOG /informix/bar_dbug.log # ON-Bar Debug Log - not in /tmp please
BAR_DEBUG=7
BAR_MAX_BACKUP 0
BAR_RETRY 1
BAR_NB_XPORT_COUNT 10
BAR_XFER_BUF_SIZE 31
RESTARTABLE_RESTORE ON
BAR_PROGRESS_FREQ 0

# Informix Storage Manager variables
ISM_DATA_POOL ISMData
ISM_LOG_POOL ISMLogs

# Read Ahead Variables
RA_PAGES 100 # Number of pages to attempt to read ahead
RA_THRESHOLD 94 # Number of pages left before next group


#RA_PAGES 336 # Number of pages to attempt to read ahead
#RA_THRESHOLD 146 # Number of pages left before next group



DBSPACETEMP tempdbs,tempdbs1,tempdbs2,tempdbs3,tempdbs4,tempdb s5
# Default temp dbspaces

# DUMP*:
# The following parameters control the type of diagnostics information which
# is preserved when an unanticipated error condition (assertion failure) occurs
# during OnLine operations.
# For DUMPSHMEM, DUMPGCORE and DUMPCORE 1 means Yes, 0 means No.

DUMPDIR /tmp # Preserve diagnostics in this directory
DUMPSHMEM 1 # Dump a copy of shared memory
DUMPGCORE 0 # Dump a core image using 'gcore'
DUMPCORE 0 # Dump a core image (Warning:this aborts OnLine)
DUMPCNT 1 # Number of shared memory or gcore dumps for
# a single user's session

FILLFACTOR 90 # Fill factor for building indexes

# method for OnLine to use when determining current time
USEOSTIME 0 # 0: use internal time(fast), 1: get time from OS(slow)

# Parallel Database Queries (pdq)
MAX_PDQPRIORITY 100 # Maximum allowed pdqpriority
DS_MAX_QUERIES # Maximum number of decision support queries
DS_TOTAL_MEMORY # Decision support memory (Kbytes)
DS_MAX_SCANS 1048576 # Maximum number of decision support scans
DATASKIP OFF
# OPTCOMPIND
# 0 => Nested loop joins will be preferred (where
# possible) over sortmerge joins and hash joins.
# 1 => If the transaction isolation mode is not
# "repeatable read", optimizer behaves as in (2)
# below. Otherwise it behaves as in (0) above.
# 2 => Use costs regardless of the transaction isolation
# mode. Nested loop joins are not necessarily
# preferred. Optimizer bases its decision purely
# on costs.
OPTCOMPIND 0 # To hint the optimizer

DIRECTIVES 1 # Optimizer DIRECTIVES ON (1/Default) or OFF (0)

ONDBSPACEDOWN 2 # Dbspace down option: 0 = CONTINUE, 1 = ABORT, 2 = WAIT
OPCACHEMAX 0 # Maximum optical cache size (Kbytes)

# HETERO_COMMIT (Gateway participation in distributed transactions)
# 1 => Heterogeneous Commit is enabled
# 0 (or any other value) => Heterogeneous Commit is disabled
HETERO_COMMIT 0

SBSPACENAME # Default smartblob space name - this is where blobs
# go if no sbspace is specified when the smartblob is
# created. It is also used by some datablades as
# the location to put their smartblobs.
SYSSBSPACENAME # Default smartblob space for use by the Informix
# Server. This is used primarily for Informix Server
# system statistics collection.

BLOCKTIMEOUT 3600 # Default timeout for system block
SYSALARMPROGRAM /informix/etc/evidence.sh # System Alarm program path

# Optimization goal: -1 = ALL_ROWS(Default), 0 = FIRST_ROWS
OPT_GOAL 0

ALLOW_NEWLINE 0 # embedded newlines(Yes = 1, No = 0 or anything but 1)

DD_HASHSIZE 20
DD_HASHMAX 511
DS_HASHSIZE 503
DS_POOLSIZE 500

ceciliapadillac, is this onconfig from production server?
And see here Configuring ISM

Yes, This is the onconfig from production but also is onconfig for test server. I did a full backup of /informix and restore it in my test environment. The onconfig is same for both servers.

Also I studied about XBSA library, let me explain what I found.

From Informix Manual
"By default, ON–Bar looks for the XBSA shared library in $INFORMIXDIR/lib/libbsa.s[ol] on UNIX. To specify a different name or location of the XBSA shared library, use the BAR_BSALIB_PATH configuration parameter. You can also make $INFORMIXDIR/lib/ibsad001.s[ol] a symbolic link to the correct library."

In both production and test environment I have the following files
root@xxxx # pwd
/informix/lib
root@xxxx # ls -l libb*
-r-xr-xr-x 1 informix informix 2289072 Dec 8 2004 libbsa.so
root@xxxx # ls -l ibsa*
lrwxrwxrwx 1 root other 37 May 27 19:15 ibsad001.so -> /usr/openv/netbackup/bin/infxbsa.so64

Variable BAR_BSALIB_PATH is not defined in my original onconfig file because the link ibsad001.so points to XBSA library from Netbackup


In the manual Installing Netbackup for Informix says
"Make sure /usr/lib/ibsad001.s0 is linked to Netbackup infxbsa.so"

lrwxrwxrwx 1 root other 37 Jun 1 11:44 /usr/lib/ibsad001.so -> /u
sr/openv/netbackup/bin/infxbsa.so64

-r-xr-xr-x 1 root other 1459008 Oct 9 2003 /usr/openv/netbackup/bin/infxbsa.so64

This link doesn't exist in my test environment, I changed it but I have the same error.

Now, I found a log in /opt/openv/netbackup/logs/infxbsa

When I restore rootdbs, I have the following messages:

11:58:24.990 [10962] <4> closeApi: INF - server EXIT STATUS = 0: the requested operation was successfully completed
11:58:24.990 [10962] <4> close_image: INF - restore completed SUCCESSFULLY
11:58:24.990 [10962] <4> close_image: INF ---- end of Restore ---
11:58:24.990 [10962] <4> VxBSAEndTxn: INF - entering VxBSAEndTxn.
11:58:24.990 [10962] <4> VxBSAEndTxn: INF - Transaction being ABORTED.
11:58:24.990 [10962] <4> VxBSATerminate: INF - entering VxBSATerminate.
11:58:24.990 [10962] <4> VxBSAGetEnv: INF - entering GetEnv -BBSA_DEBUGFD
11:58:24.990 [10962] <4> VxBSAGetEnv: INF - returning -
************************************************** *****8
When I restore the following dbspaces I have the following:

13:59:58.953 [2320] <4> closeApi: INF - EXIT STATUS 0: the requested operation was successfully completed
14:00:03.950 [2320] <4> closeApi: INF - server EXIT STATUS = 0: the requested operation was successfully completed
14:00:03.950 [2320] <4> close_image: INF - restore completed SUCCESSFULLY
14:00:03.950 [2320] <4> close_image: INF ---- end of Restore ---

14:00:03.950 [2320] <4> VxBSAEndTxn: INF - entering VxBSAEndTxn.
14:00:03.950 [2320] <4> VxBSAEndTxn: INF - Transaction being COMMITED.14:00:09.645 [2320] <4> VxBSATerminate: INF - entering VxBSATerminate.
14:00:09.645 [2320] <4> VxBSAGetEnv: INF - entering GetEnv - NBBSA_DEBUGFD
14:00:09.645 [2320] <4> VxBSAGetEnv: INF - returning -

The difference is with rootdbs doesn't commit XBSA transaction but the restore is succesful. I understand now the message. rootdbs is restore but XBSA is not finishing the transaction and leaves rootdbs with a flag.

The question is... why is not committing the transaction?

Hi, someone in other forums asked me to verify my backup.

I execute onbar -v rootdbs and look what I found.

STATUS: Checking sysuser:sysauth
STATUS: Checking 0x00100044:
ERROR: bad extent number 0 -- only 0 extents.
ERROR: Unable to convert logical address to physical address.
STATUS: Checking onpload:ix100_1
STATUS: Checking 0x0010004E:
ERROR: bad extent number 0 -- only 0 extents.
ERROR: Unable to convert logical address to physical address.
STATUS: Checking 0x0010004F:
ERROR: bad extent number 0 -- only 0 extents.
ERROR: Unable to convert logical address to physical address.STATUS: Checking onpload:maps
STATUS: Checking onpload:ix104_1
STATUS: Checking onpload:mapitem
.................... exit omitted
STATUS: Table checks FAILED.STATUS: BLOBChunk checks PASSED.
TIME: [06/04/2009 13:04:31] Checking Completed.
STATUS: Tables/Fragments validated: 241
Archive Validation FAILED.

It seems that it's an Informix problem, not an XBSA problem, isn't it?

The funny think for me is that Informix log doesn't report it. I just look this error in this forum but I didn't find anything. I'm continue with research.

If someone knows about this new error, I really appreciate. All your comments you have make were very, very useful.