SE 5.10 UC1 Corruption
Posted in 1998
Hi All I have only posted a question here once before and have answered a few myself, but have been an avid reader of this newsgroup for some time now. I have a problem that I am sure someone must have seen before so I decided to post again. My client has the following installation in 108 sites. The same application, database and procedures (to a certain extent ;-)) are employed in all sites : IBM POWERPC's/RS6000 2 GB HDD's AIX 4.2 SE 5.10 UC1 4.11 fully compiled binaries We recently upgraded the operating systems to the above configuration from AIX 3.5.1 and AIX 4.1 as part of our Y2K rollout. Similarly, our database has moved from various flavours of SE 4 to 5.10. The databases vary in size between 700MB (for smaller sites) and just under 1GB (for larger sites). Lately I have noticed a trend where two of the most volatile tables in the system are being corrupted. This is also not an isolated incident and has occured on 6 seperate occasions in as many sites over the past few months. There does not seem to be a pattern as far as the platform is concerned since today made the the spread even. The symptoms are as follows : -Corruption is detected after a daily batch extract of the tables is done, the extract file does not pass routine checks and an exception is raised with my support team. -Bcheck confirms corruption -An ls on the .dat and .idx file sizes reports a dat file in excess of 2GB with a similar .idx file size. -Our logs report SQL error -564 (Cannot sort rows) -System error no 28 (No space left on device) - No surprise there -A df on the disk reports (on average) only 80% usage of the disk (since the database size has not increased dramatically in the last two years - this is not a major concern at this point) What does concern me is that the size of the disk is only 2GB. On two of the occasions, we were unable to save the tables due to the extent of the corruption. Since one of the tables contains daily sales transactions, the dataset as a whole must be restored or we must rollback with the most recent backup to preserve data integrity. This causes untold misery for my support team as a thousand provedures must be followed to recover the site. This is the reason for my posting. I know that database corruption manifests itself in many wierd and wonderful ways, however this seems rope the operating system in aswell. My questions are : -How can the op`erating system report a file of this size? -Since the size of the file reported by AIX is obvioulsy bogus, is AIX itself being corrupted by Informix? -We have had corruption of these tables before, but not with these symptoms, could this have something to do with our configuration? (Informix and IBM assured us we would have no compatability issues) Any comments, thoughts, suggestions or thumbsucks would be greatly apreciated. Thanks in advance David A good scapegoat is almost as good as a solution.