Unanswered: Informix Chunk is offline (IDS7 on Windows 2000)
one of our informix servers starting abnormally. the services on Widows starts ok but with no access to the database. one of the chunks
seem to be offiline..
The informix is running on Windows 2000 environment. I have checked to confirm but the group the 'informix' user belongs to is already added to the security tab with full acess rights to both the informix client folder on C Drive and the informix data folder in E Drive. Please note that the server and clientis on same machine currently.
Please find the errors below for your advice.
18:15:29 DR: DRAUTO is 0 (Off)
18:17:01 IBM Informix Dynamic Server Version 7.31.TD8 Software
Serial Number AAC#J993468
18:17:05 IBM Informix Dynamic Server Initialized -- Shared Memory
18:17:05 Physical Recovery Started.
18:17:05 Physical Recovery Complete: 1 Pages Restored.
18:17:05 Logical Recovery Started.
18:17:08 Logical Recovery Complete.
1 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks
18:17:09 Dataskip is now OFF for all dbspaces
18:17:09 KAIO: error in kaio_WRITE, kaiocbp = 2dac706c, errno = 3
18:17:09 fildes = 1056 (gfd 7), buf = 2e001000, nbytes = 4096, offset
18:17:09 KAIO: error in kaio_WRITE, kaiocbp = 2d9a886c, errno = 3
18:17:09 fildes = 1056 (gfd 7), buf = d8e9000, nbytes = 4096, offset
18:17:09 KAIO: error in kaio_WRITE, kaiocbp = 2d99d448, errno = 3
18:17:09 fildes = 1056 (gfd 7), buf = d8fb000, nbytes = 4096, offset
18:17:09 Assert Failed: I/O error, Primary Chunk 'e:\ifmxdata
\ol_ccbsrv\datadbs_dat.002' -- Offline
18:17:09 IBM Informix Dynamic Server Version 7.31.TD8
18:17:09 Who: Thread(43, flush_sub(2), 0, 3)
File: \tristarm\rsam\rsbuff.c Line: 4175
18:17:09 Results: Chunk is now unusable
18:17:09 Action: Repair and restore from mirror or archive
18:17:12 Releasing server from system block
18:17:16 See Also: af.4130125
18:17:16 I/O error, Primary Chunk 'e:\ifmxdata\ol_ccbsrv\datadbs_dat.
002' -- Offline
18:17:17 Assert Failed: Chunk 6 is being taken OFFLINE.
18:17:17 IBM Informix Dynamic Server Version 7.31.TD8
18:17:17 Who: Thread(43, flush_sub(2), 0, 3)
File: \tristarm\rsam\rsmirror.c Line: 1740
18:17:17 Results: Dynamic Server must abort
18:17:17 Action: Reinitialize shared memory
18:17:23 See Also: af.4130125
18:17:23 Chunk 6 is being taken OFFLINE.
18:17:24 Assert Failed: INFORMIX-OnLine Must ABORT
Critical media failure.
18:17:24 IBM Informix Dynamic Server Version 7.31.TD8
18:17:24 Who: Thread(43, flush_sub(2), 0, 3)
File: \tristarm\rsam\rsmirror.c Line: 1744
18:17:30 See Also: af.4130125
18:17:30 \tristarm\rsam\rsmirror.c, line 1744, thread 43, proc id
1360, INFORMIX-OnLine Must ABORT
Critical media failure..
18:17:30 PANIC: Attempting to bring system down
I am not sure the Kernel IO / DIRECT IO is accepted on windows, can you check in your documentation.
For the same purpose, SET the onconfig parameter DIRECT_IO to 0 and rebounce your engine.
Check your log message and send it again. The Kaio_write message should disappear, and probably the error.
This is the first step to check in the troubleshooting process. Can you please do this?
Thanks for the commnent. I've checkd the docuemntations, it says
"Windows platforms do not recognize the value of the DIRECT I/O configuration parameter, and direct I/O is used for dbspace chunks on Windows® platforms regardless of the value of the DIRECT_I/O configuration parameter." Also, I have checked the onconfig file and there is not existing paramter for DIRECT_IO. Perhaps, I should add it and set the value = 0.
btw, do u ve skype? my screen name is 'gassamaba'.
Questions ordered by priority:
1) Has this informix server ever worked correctly or is this the first time you start it?
2) as you confirmed, the KAIO_write messages do not have anything to do in the log file and point to an expected behaviour.
3) Is the user informix owner of the e:\ifmxdata\ol_ccbsrv\datadbs_dat.002 cooked file?
With unix this is a prerequisite, and I think it is also for windows.
4) do you have other servers working with the same environment?
5) 7.31 is very old and has no more support/patches . Can you consider migrating to the 11.70 version ?
1 & 5. We are using a new application on a different server. This IDS 7 server has some old information and reports we needed to run now. The server was runnign unattended, this error first occured in October according to the log file but we only relised it now when needed to run some stuff on it.
2. I have added the 'DIRECT_IO 0' paramter to onconfig file but it made no effect.
3. 'Informix' user owns it by inheretence. The owner of the folder is 'Administrators'. informix user belongs to 'informix-admin' group which also belongs to 'administrators' group. However, however, informix user has full access right on the folder and its contents of datafiles. ALSO, I have changed the owner to 'informix' user for the 'e:\ifmxdata\ol_ccbsrv\' directory and restarted the database, yet the error still as is.
I also tried to use use 'oncheck|onstat|onmode' utilities but kept getting permission error message "Could not initialize the security subsystem. Pease ensure that this account has the necessary privilages". According the error definitions; this kind of error occures if there are other DBSERVERALIASes but there are no other aliases on the server.
do u use have VNC client? drop me email email@example.com or skype :gassamaba
will really appreciate that.
the instance had received a Assert Failed errors days ago, mentionning a Kernel IO write error. We had possibility of a chunk physical or logical corruption, then a instance crash.
This Kernel IO error has been repeating for a number of times, systematically when rebouncing the instance.
While ausculting the system, we found out that some of the disk drives were running out of space, including the one hosting the Informix shared memory file.
The immediate effect was to prevent Informix to write this shared memory file at bounce time, thus preventing the instance to ever start.
A good files cleanup session has resolved the issue and now gassamba seems to be a happy man :-)
I agree that the KAIO error message has nothing to do with the Windows port, since KAIO is not supported on this platform. This "unexpected behaviour" allowed us to explore unuseful areas .....
But this was version 7.31, a 14 years old teenager version....
for the community :
the issue is resolved. The main reason that causes the initialization failure of the Shared Memory is the disk space of the data file directory. Perhaps, explicitly changing directory owner to 'informix' user rather than 'informix-admin' group to which 'informix' user is member may also have role. We were considering rebuilding the databse with 'dbimport', but this was lost of errors in the process and begooden-it has made a very good suggestions on that.
I am not sure, I think the KAIO error may be a bug as Windows is not using that parameter as oer thre IBM documentation.
I really appreciate the support begooden-it, in particular, provided to me. I am the happiest man now.
begooden-it has an excellent log analyser, I hope he'll share that with the community