12-13-02, 13:59 #1Registered User
- Join Date
- Dec 2002
Unanswered: dBase headers corrupting, invalid data in DBFs on network
I have dBase 5 for DOS and a Novell 5 network for up to 110 users. I have noticed 3 kinds of errors happening more often now, often with tables opened exclusively, not shared:
The most common is that invalid characters get stuffed into records, usually the newer ones. The worst is where the ASC(1-27) control characters get into fields that are indexed, then the index is bad. The data has to be fixed or removed, and then PACKed (since even deleted records can mess up an index). Since some of the data is just offset right or left, it looks like the buffers are getting scrambled. Other times it looks like part of the browse screen itself is there in the record, including the vertical bars!
On rare occasions I've even seen the text of a report that had been sent to the printer stuffed in the middle of a DBF!
I wonder if part of the problem is that much of the original code has the users editing the actual fields on the screen or by BROWSE. Perhaps using memory variables to add/edit may be better, but is not feasable to do in most cases.
Appending a record to a table can end up with the RECCOUNT() not matching the actual records. We now detect this by LISTing one of the table's fields with the console screen off, then comparing the "last" record's number (now at EOF() to the record count plus one. For example, you can LIST 50 records, but RECCOUNT() reports 60. The "missing" records really are there. If you GOTO 55, then DISPLAY or "?" the fields you see the data, but BROWSE says "End of field encountered".
I used to PACK the tables. Now I rename it to a temp name, recreate the table, and APPEND in the old records. By doing this I can salvage some of the newer records where the header says there are fewer records than there actually are.
Added note: I just scanned what was written to the disk and saw what seems to be index code in the middle of a record or two. Somehow the index data is getting dumped into the table. These should be discrete, separate file handles, so something is screwy here. Why would data from one area be written into another file handler area?
Similar to above, except the records appended never really append. The file size never enlarges but the record count does. PACKing the table (and REBOOTing the workstation is a must!) seems to fix this. But I've noticed that some tables are repeatedly getting corrupted this way by only a few certain users. (The workstations are NOT identical.) And once they corrupt one table, they can then corrupt some (but not all) tables opened later doing the same functions.
This has been a real problem in recent months.
Since the Novell 5 network does compress some files, we made sure that these DBFs never compress anymore. Looks like there may be fewer data errors now, but they still happen. Even once is too much!
Checking all DBFs every night catches these errors, but this cannot continue.
Last edited by Mark Amanns; 12-16-02 at 11:00.
02-25-03, 02:19 #2Registered User
- Join Date
- Feb 2003
- San Antonio, TX
Welcome to the world of archaic databases
Okay a couple of possibilities come to mind. If your system is, in effect, caching data when records are being modified/added, then having more than one user accessing the data can result in the kind of file-trashing you are describing. Another possibility is that the network is somehow "sharing" memory or packets and getting them confused.
In spite of the fact that DBase has often been sold as a networkable database, it really isn't all that well optimized or designed for use on a network. You may have reached a sort of saturation on your system . . . too many users accessing too many records too often.
In other words, I think you may need to give some serious thought to what your _next_ database is going to be and just how soon you can migrate to it!Ralph D. Wilson II
"Any sufficiently advanced technology is indistinguishable from magic." A.C. Clark
03-26-03, 19:15 #3Registered User
- Join Date
- Mar 2003
Say it with me "We love legacy"
We've been struggling with similar problems. One of the guys here found out it has something to do with clock speed on the newer PCs. The final solution was to identify PCs that would not cause the problem and tell the users that "This set of PCs are the only ones you can use to preform mass changes." By chance most of the PC were IPACs. We found that any time a newer PC (P3 or P4s) tried to append multiple records to one table the end of file marker was moved to make room for the new records but the data was not writen down to the table. This is exactly what you are discribing in point number 3.
Point 1 is odd. I've seen it every so often and always thought it was Novell just spraying junk all over the disk.
Point 3. Is is posible that end of file pointer is adjusted, the data is not getting written and you just see what *used* to be in that spot on the disk.
Either way give this a try. We added a field, called VALID, to all of our tables. It was 1 char in length. The idea is that any time a new record is created Clipper defaults all char fields with spaces. Then each night we look at that field on each record and if any of them are not ' ' we know there is a problem.
Good luck. GBA.Brian
(replace NOSPAM with hotmail)
05-30-03, 14:24 #4Registered User
- Join Date
- Dec 2002
Too late, we gave up...
I've seen several posts saying that IPX and TCP/IP protocols did not like to coexist together. I'm not sure, but it seems to be more of a workstation than a network conflict.
On another site I saw this suggestion, also too late for me to see if it worked: If you begin to experience index file corruption on large files routinely, check the setting "CLIENT FILE CACHING ENABLED". It should be set to OFF.
Sad to say, when we upgraded to new hardware (from 400MHz to 2.4GHz and 6 times as much hard drive storage) we also moved to Windows 2000 Server SP3. No more IPX. We are still using dBase 5, but the data issues went away on the new server.
Was it a hardware issue, network server issue, or workstation OS issue, or a combination? I'll never know. In the back of my mind, though, I wonder if Mr. Softie makes their new OS just incompatible enough with Novell's server to make users give up in frustration, not to mention their undercutting Novell's user license prices.