understanding oncheck -pp

Hi, i m trying to understand the following output from oncheck -pp on IDS9.40.FC5XF. DV11% oncheck -pp 0x0050004E 0addr stamp chksum nslots flag type frptr frcnt next prev 6:1016 93984 6cdf 0 4 FREE 24 2020 0 0 0:8 e c c c e c e c c e c a c 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 kindly let me know the meaning of chksum, flag, type. also pl explain what does e, c, a and 4 mean in the next line. what could be the other values in that field ? thanks. vartika.

The oncheck option you are using (-pp) prints logical page numbers from a specified tablespace's partnumber. You are looking at a bitmap page for a tablespace. The first page, or page 0. of every tablespace (tablespace = logical collections of all extents belonging to a single table or if fragmented, a single fragment) is a bitmap page. The bitmap page tracks the relative fullness or emptyness of every page in the tablespace. The values you are asking about are not important to us users, they are pretty much for internal Informix use. The chksum is a checksum value used for consistency checking, the flag and Free indicate that it is a bitmap page that you are looking at and the values you mentioned all represent pages in the tablespace. What was the purpose of dumping the page? It's cool to figure out some of the internal architecture for sure, but was there another reason besides curiosity? Mike

Thanks Mike, As you might have guessed , i got an error of bad page ie 'Assert Failed:bad current page' .. so just thought of drilling down. I couldn't find any elaborate doc about IDS oncheck utility in google, so had to post this issue in this forum. I also want to understand/interpret the output in /tmp/af.xxxxxxx file. I will be thankful, if anyone can help. vartika.

Vartika - You can post the portion of your assert failure that references the bad page here for the forum's review, but you are best served by calling Informix Technical Support. Much of the information in the af.XXXXX file is for internal use - most notably the stack trace of the offending thread. The folks in Lenexa (where Informix support is located) can assist you. Mike

Thanks Mike, Here is the excerpts from the af.xxxxxxx file - ***** 11:08:29 Assert Failed: Page Check Error in btcurrent:bad current page 11:08:29 Who: Session(16, informix@DV11, 3743, 10b013e98) Thread(45, sqlexec, 10afd38f0, 1) File: rsdebug.c Line: 1057 11:08:29 Results: Possible inconsistencies in 'mydbs:"informix".test' 11:08:29 Action: Run 'oncheck -cDI mydbs:"informix".test' 11:08:29 Stack for thread: 45 sqlexec base: 0x000000010b46c000 len: 69632 pc: 0x000000010087ecb4 tos: 0x000000010b47a451 state: running vp: 1 0x10087ecb4 oninit :: afstack + 0x34 sp=0x10b47ac50(0xb2fc518, 0xb2fcb58, 0x1000008, 0x0, 0xef40b8, 0xb47b1e0) 0x10087dd18 oninit :: afhandler + 0xaf8 sp=0x10b47af20 delta_sp=720(0xcd5200, 0xef6358, 0xef6158, 0x401, 0x1, 0xefe7a0) 0x10087d158 oninit :: affail_interface + 0x4c sp=0x10b47b640 delta_sp=1824(0xef5d58, 0xef6158, 0xef6358, 0x1, 0xe0bed0, 0x421 ) 0x1005ef45c oninit :: bffail + 0xe48 sp=0x10b47b700 delta_sp=192(0xcd2fb0, 0xef6158, 0xede020, 0xef5d58, 0xe67f1c, 0x0) 0x10055a8cc oninit :: btcurrent + 0x578 sp=0x10b47b7b0 delta_sp=176(0xe03c48, 0xede028, 0xede020, 0xef48f8, 0xafd38f0, 0x4) 0x10050adac oninit :: find_page + 0x2e4 sp=0x10b47b8b0 delta_sp=256(0xafd38f0, 0x0, 0xede020, 0xb47ba80, 0xafd3c18, 0xb47f048 ) 0x10050a3f0 oninit :: rsread + 0x66c sp=0x10b47b980 delta_sp=208(0xef48f8, 0xb47f028, 0xef4900, 0xede020, 0x2, 0xb47f048) 0x100911538 oninit :: fmread + 0x234 sp=0x10b47bad0 delta_sp=336(0xb47f548, 0x102, 0xede048, 0xb2ab598, 0xb47c010, 0x0) 0x10027fb1c oninit :: sqisread + 0x6c sp=0x10b47bf60 delta_sp=1168(0x2, 0xede048, 0x102, 0xb47ff38, 0x2, 0xcd0da0) 0x1002b70f4 oninit :: readseq + 0x610 sp=0x10b47c020 delta_sp=192(0xede048, 0x10000, 0xede040, 0xb47c5ee, 0x100, 0xb47fd48) 0x1002b671c oninit :: gettupl + 0x2d0 sp=0x10b47c490 delta_sp=1136(0xb47fd48, 0xede040, 0xede048, 0x2000, 0x8000, 0xb2a7a89) 0x1002b2ab8 oninit :: scan_next + 0x388 sp=0x10b47c5f0 delta_sp=352(0xb47fd48, 0xb47fd48, 0xede040, 0xb47fd48, 0x20202020, 0x 0) 0x1002c0e70 oninit :: getrow + 0x1ac sp=0x10b47c6b0 delta_sp=192(0xb468620, 0x78, 0xb489390, 0x0, 0x10000000, 0xb46b058) 0x1002c0ba8 oninit :: fetchrow + 0x200 sp=0x10b47c770 delta_sp=192(0xb468620, 0xede008, 0x1400, 0x152c, 0xede040, 0xb468620) 0x100497ba0 oninit :: exfetch + 0xa0 sp=0x10b47c820 delta_sp=176(0xede040, 0xa, 0xa, 0xb47c9ac, 0xb468620, 0x2) 0x1003e9ed0 oninit :: sq_nfetch + 0x538 sp=0x10b47c8d0 delta_sp=176(0xede040, 0xb482028, 0x22, 0xb484028, 0x82, 0x0) 0x10046e330 oninit :: sqmain + 0xdc0 sp=0x10b47c9e0 delta_sp=272(0x9, 0x1358, 0x1400, 0xede040, 0xede008, 0xcd0da0) 0x1008a6b0c oninit :: listen_verify + 0x5a4 sp=0x10b47caf0 delta_sp=272(0x0, 0xd28040, 0x2000, 0xcd0da0, 0x0, 0xb2a6998) 0x100854990 oninit :: startup + 0xfc sp=0x10b47ce50 delta_sp=864(0xefe7a0, 0x7, 0xed5078, 0xefe7a0, 0xef4558, 0xb29eec1) 11:08:29 See Also: /tmp/af.41504d4, shmem.41504d4.0 **** after this it gives the output from various onstat options, namely - /informix/bin/onstat -g ath: /informix/bin/onstat -g stk 45 light: /informix/bin/onstat -u: /informix/bin/onstat -g ses 16: /informix/bin/onstat -g sql 16: /informix/bin/onstat -c: /informix/bin/onstat -s: /informix/bin/onstat -k: /informix/bin/onstat -b: /informix/bin/onstat -t: /informix/bin/onstat -d: /informix/bin/onstat -l: /informix/bin/onstat -p: /informix/bin/onstat -x: /informix/bin/onstat -f: /informix/bin/onstat -h: /informix/bin/onstat -C: /informix/bin/onstat -F: /informix/bin/onstat -R: /informix/bin/onstat -g glo: (all mth options.) /informix/bin/onstat -g sym: list of Environment Variables: /informix/bin/oninit -V: /informix/bin/onstat -V: /informix/bin/oncheck -V: /informix/bin/onmode -V: uname -a: I agree its a tech support job but still i look forward to your input. :) vartika

Vartika - Have you ran the recommended oncheck yet? 'oncheck -cDI mydbs:"informix".test' This command will check the integrity of the data on disk and in some cases, (index problems mainly) the engine can dynamically repair problems. Depending on the size of the table, the oncheck can take a long time to complete. If this is a production instance you might want to wait until processing is at its slowest to run the oncheck. If the table is small enough you may wish to consider unloading the table, dropping it, and then recreating it. It is difficult to determine the root cause of this error with the information at hand - however it would not hurt to do a sanity check on the disk devices and other supporting hardware that supports the mydbs:test table. If you have ran the oncheck -cDI on the offending table, please post the results. Thanks! Mike

Hi Mike, I ran oncheck -cDI on the table and the engine did not repair the problem. Its on developement machine and the table contains 761 rows. There are 4 instance running on it and one being IDS version 10. All are sharing the same $INFORMIXDIR (except ver 10).It is sun4u Sun Fire V240 machine, has 2 cpus, 8gb memory, 167 MHZ ( clock freq) and cooked files as chunks. i do not know anything abt sanity check of hardware/disk devices. I tried to unload but could unload only 745 rows. (not much loss) output from oncheck -cDI says - DV11% oncheck -cDI mydbs:"informix".test WARNING: index check requires a s-lock on tables whose lock level is page. Validating indexes for mydbs:informix.test... WARNING: index check requires a s-lock on tables whose lock level is page. TBLspace data check for mydbs:informix.test ERROR: Tablespace 0x500002. Page 0x7 appears to be of type "btree", which is out of sync with its bitmap representation (0xc). I do not have any index on this table and i am getting a page error which is of type "btree". Output from oncheck -pp 0x500002 0x7 - addr stamp chksum nslots flag type frptr frcnt next prev 6:60 95868 7647 30 90 BTREE 1474 450 6 0 slot ptr len flg 1 1236 49 0 2 1285 47 0 ... slot 1: 0: b 20 47 4c 5f 43 4f 4c 4c 41 54 45 20 20 20 20 . GL_COLLATE 16: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 32: 20 20 20 20 20 20 20 20 20 20 20 20 0 0 5 10 .... 48: 0 ................ slot 2: 0: 9 20 47 4c 5f 43 54 59 50 45 20 20 20 20 20 20 . GL_CTYPE 16: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 32: 20 20 20 20 20 20 20 20 20 20 0 0 8 1 0 ...... It seems something got mixed. vartika.

VARTIKA AGRAWAL wrote: Is it possible that two instances are using the same file for some chunk(s)? Because it looked to me that chunk #5 page 7 thinks that it is page 60 of chunk #6 from the oncheck output. Unless I'm misreading the output. Art S. Kagel Oninit > Hi Mike, > > I ran oncheck -cDI on the table and the engine did not repair the problem. > Its on developement machine and the table contains 761 rows. There are 4 > instance running on it and one being IDS version 10. All are sharing the same > $INFORMIXDIR (except ver 10).It is sun4u Sun Fire V240 machine, has 2 cpus, > 8gb memory, 167 MHZ ( clock freq) and cooked files as chunks. > i do not know anything abt sanity check of hardware/disk devices. > I tried to unload but could unload only 745 rows. (not much loss) > > output from oncheck -cDI says - > > DV11% oncheck -cDI mydbs:"informix".test> > WARNING: index check requires a s-lock on tables whose lock level is page. > > Validating indexes for mydbs:informix.test... > > WARNING: index check requires a s-lock on tables whose lock level is page. > > TBLspace data check for mydbs:informix.test > > ERROR: Tablespace 0x500002. Page 0x7 appears to be of type "btree", > > which is out of sync with its bitmap representation (0xc). > > I do not have any index on this table and i am getting a page error which is > of type "btree". > > Output from oncheck -pp 0x500002 0x7 - > > addr stamp chksum nslots flag type frptr frcnt next prev > 6:60 95868 7647 30 90 BTREE 1474 450 6 0 > > slot ptr len flg > > 1 1236 49 0 > > 2 1285 47 0 ... > > slot 1: > > 0: b 20 47 4c 5f 43 4f 4c 4c 41 54 45 20 20 20 20 . GL_COLLATE > > 16: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 > > 32: 20 20 20 20 20 20 20 20 20 20 20 20 0 0 5 10 .... > > 48: 0 ................ > slot 2: > > 0: 9 20 47 4c 5f 43 54 59 50 45 20 20 20 20 20 20 . GL_CTYPE > > 16: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 > > 32: 20 20 20 20 20 20 20 20 20 20 0 0 8 1 0 ...... > > It seems something got mixed. > vartika. > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > See you at the IIUG Informix 2008 Conference > The Power Conference for Informix Professionals > April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas > http://www.iiug.org/conf > Registration Now Open!! > > >

It is not possible for two instances to use the same file for some chunk(s). onspace & onmonitor both report error. It seems the page/file got overwritten by some OS job, hence when engine searches for the page 7 on chunk 5 it finds page 60 of chunk 6 in its space and reports page check error. Anyways, this is my opinion. If you have any other obseration please share. vartika.

Are these chunk files on SAN or local disks? Recently I saw same chunks were shared by two instances on same box which I do not understand how??? but it was. The only thing I can think of that chunks were had sym links and even with same chunk name under lying volume groups/logical volumes would have be different. Apologies, I do not have much understanding about SAN, just knows a piece of disk device. VARTIKA AGRAWAL <vartika_agrawal@ril.com> wrote: It is not possible for two instances to use the same file for some chunk(s). onspace & onmonitor both report error. It seems the page/file got overwritten by some OS job, hence when engine searches for the page 7 on chunk 5 it finds page 60 of chunk 6 in its space and reports page check error. Anyways, this is my opinion. If you have any other obseration please share. vartika. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. See you at the IIUG Informix 2008 Conference The Power Conference for Informix Professionals April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas http://www.iiug.org/conf Registration Now Open!! --------------------------------- Get the name you always wanted with the new y7mail email address.

Yes, chunks can be shared by multiple instance on same machine. I tested that IDS supports it. My machine is on local disk and there is no chunk sharing. Thanks to you and apologies to Art. vartika.

I suspect that the problem is related to system catalog table entries for the table in question, mydbs:"informix".test. The oncheck you ran against this table returned an error referencing this tablespace 0x500002, which is probably the systables table for the mydbs database. To confirm this run oncheck -pt 0x500002 and you will the table's name at the top of the output. I would try unloading the mydbs:test table and then drop the table. Once the table is dropped run: oncheck -cc mydbs followed by oncheck -cDI 0x500002 At this point however - you REALLY should be engaging Informix Advanced Technical Support. Mike

Sorry for double posting - I posted earlier in the wrong thread... I suspect that the problem is related to system catalog table entries for the table in question, mydbs:"informix".test. The oncheck you ran against this table returned an error referencing this tablespace 0x500002, which is probably the systables table for the mydbs database. To confirm this run oncheck -pt 0x500002 and you will the table's name at the top of the output. I would try unloading the mydbs:test table and then drop the table. Once the table is dropped run: oncheck -cc mydbs followed by oncheck -cDI 0x500002 At this point however - you REALLY should be engaging Informix Advanced Technical Support. Mike

Regarding the possibility of mixed up chunks - I doubt this is the case. oncheck -pp 0x500002 0x7 addr stamp chksum nslots flag type frptr frcnt next prev 6:60 95868 7647 30 90 BTREE 1474 450 6 0 The oncheck command indicates that we want to dump out logical page 7 from partnum (or tablespace id) 0x500002. The partnum is made up of two values - the first value, 0x5, is the dbspace number where the tablespace resides. The second value, 0x00002, represents the third logical page from the tablespace tablespace we wish to look at, we start counting logical pages at 0. The concept of tablespace tablespace can be confusing, but just think of it as a collection of pages that contain biographies of all the tables created in the dbspace. The addr portion of the actual output indicates that the page we asked oncheck to dump out has a physical location of chunk #6 and is offset from the beginning of the chunk 60 pages. It is entirely possible that chunk #6 resides in dbspace #5. Mike

On Fri, Apr 18, 2008 at 2:23 AM, VARTIKA AGRAWAL <vartika_agrawal@ril.com> wrote: > Yes, chunks can be shared by multiple instance on same machine. I tested that > IDS supports it. Sarcasm mode on - but note that OP is not actually using shared chunks. Remind me to tell the Informix architect team that it is supported. We just spent a considerable effort on adding Shared Disk Secondaries (SDS) to IDS to allow disks to be shared between machines, and it was all unnecessary. Or maybe it was only necessary because they are shared between machines... Sarcasm mode off. Where did you find documentation to the effect that IDS allows you to share chunks between instances? I believe the answer is nowhere. Your experimentation simply shows that IDS has to trust you when you tell it "You have exclusive control of this chunk of disk". It can't say "Oh, look, some other instance is using it". It can't really do much even if it detects a valid IDS chunk layout on the disk -- you might quite reasonably be reusing a chunk that was previously allocated to a different instance. If you are trying to share a chunk between two different instances, both instances are hosed. > My machine is on local disk and there is no chunk sharing. OK - so there is no chunk sharing. Please do not trust your experiments. Please do not ever try to set up chunk sharing. -- Jonathan Leffler #include <disclaimer.h> Email: jleffler@earthlink.net, jleffler@us.ibm.com Guardian of DBD::Informix v2008.0229 -- http://dbi.perl.org/ "Blessed are we who can laugh at ourselves, for we shall never cease to be amused." NB: Please do not use this email for correspondence. I don't necessarily read it every week, even.

Yes Jonathan , it is not documented anywhere that a chunk can be shared, but it is supported on IDS 9.40.FC5XF and i have tested it NOT TO PROVE ANYONE OR ANYTHING WRONG but to check my belief. and i have lost a big table as the result of it. I could do it as it is development (or playground) machine and i have proper backups in place. I also agree with you that no one should set up chunk sharing at any cost. I sincerely apologise to Informix architect team if it hurt. vartika.

The system catalog table entries seems to be in right place. the result of oncheck -pt 0x500002 shows - TBLspace Report for mydbs:informix.test Physical Address 6:5 Creation date 04/19/2008 17:52:25 TBLspace Flags 801 Page Locking TBLspace use 4 bit bit-maps ... Index ix101_1 fragment in DBspace dbs1 Physical Address 2:926 Creation date 04/19/2008 17:52:25 TBLspace Flags 801 Page Locking TBLspace use 4 bit bit-maps .. I have created a table with same schema and loaded the rows from the original table for further testing. Tech support is not feasible (economically) as it is developement machine. vartika.

> Remind me to tell the Informix architect team that it is supported. > OI JON. REMIND THE INFORMIX ARCHITECT TEAM THAT IT'S SUPPORTED. Consider yourself reminded. Notice of Confidentiality: **This E-mail and any of its attachments may contain Lincoln National Corporation proprietary information, which is privileged, confidential, or subject to copyright belonging to the Lincoln National Corporation family of companies. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout. Thank You.**

Jonathan Leffler wrote: Jonathan, you missed his point. IB the point is that if you have multiple instances on a single machine, if there are separate links pointing to the same disk, at least in his environment, the engines allowed him to use the same physical disk space for chunks in two separate instances resulting in data loss. This used to be a major problem in 5.xx, but I thought that since somewhere around 7.20 the engine follow links keeps track of disk used by different instances in a place accessible to all of the instances so that these kind of cross-linked chunks can't happen. He does not mean to say that shared disks are 'permitted' as a good thing but rather not 'prevented' when created accidentally. Art S. Kagel Oninit > On Fri, Apr 18, 2008 at 2:23 AM, VARTIKA AGRAWAL > <vartika_agrawal@ril.com> wrote: > >> Yes, chunks can be shared by multiple instance on same machine. I tested >> > that > >> IDS supports it. >> > > Sarcasm mode on - but note that OP is not actually using shared chunks. > > Remind me to tell the Informix architect team that it is supported. > We just spent a considerable effort on adding Shared Disk Secondaries > (SDS) to IDS to allow disks to be shared between machines, and it was > all unnecessary. Or maybe it was only necessary because they are > shared between machines... > > Sarcasm mode off. > > Where did you find documentation to the effect that IDS allows you to > share chunks between instances? > > I believe the answer is nowhere. > > Your experimentation simply shows that IDS has to trust you when you > tell it "You have exclusive control of this chunk of disk". It can't > say "Oh, look, some other instance is using it". It can't really do > much even if it detects a valid IDS chunk layout on the disk -- you > might quite reasonably be reusing a chunk that was previously > allocated to a different instance. > > If you are trying to share a chunk between two different instances, > both instances are hosed. > > >> My machine is on local disk and there is no chunk sharing. >> > > OK - so there is no chunk sharing. Please do not trust your > experiments. Please do not ever try to set up chunk sharing. > >

On Mon, Apr 21, 2008 at 7:48 PM, Art S. Kagel (Oninit) <art@oninit.com> wrote: > Jonathan Leffler wrote: > > Jonathan, you missed his point. IB the point is that if you have > multiple instances on a single machine, if there are separate links > pointing to the same disk, at least in his environment, the engines > allowed him to use the same physical disk space for chunks in two > separate instances resulting in data loss. This used to be a major > problem in 5.xx, but I thought that since somewhere around 7.20 the > engine follow links keeps track of disk used by different instances in a > place accessible to all of the instances so that these kind of > cross-linked chunks can't happen. > > He does not mean to say that shared disks are 'permitted' as a good > thing but rather not 'prevented' when created accidentally. Your interpretation of what he is saying makes some sense - more so than the standard interpretation of the words used. I'm not aware of code in IDS that tracks cross-linked chunks - witness the fact that the OP was able to demonstrate that IDS 9.40 does not. For the original poster's benefit, if the manual doesn't say that something is 'supported' by documenting that it is supported, there is a very good chance that it is not 'supported' in the sense that reporting the problem to IBM/Informix Tech Support will likely get you the 'not supported' answer. > > On Fri, Apr 18, 2008 at 2:23 AM, VARTIKA AGRAWAL > > <vartika_agrawal@ril.com> wrote: > > > >> Yes, chunks can be shared by multiple instance on same machine. I tested > >> that IDS supports it. > > > > Sarcasm mode on - but note that OP is not actually using shared chunks. > > > > Remind me to tell the Informix architect team that it is supported. > > We just spent a considerable effort on adding Shared Disk Secondaries > > (SDS) to IDS to allow disks to be shared between machines, and it was > > all unnecessary. Or maybe it was only necessary because they are > > shared between machines... > > > > Sarcasm mode off. > > > > Where did you find documentation to the effect that IDS allows you to > > share chunks between instances? > > > > I believe the answer is nowhere. > > > > Your experimentation simply shows that IDS has to trust you when you > > tell it "You have exclusive control of this chunk of disk". It can't > > say "Oh, look, some other instance is using it". It can't really do > > much even if it detects a valid IDS chunk layout on the disk -- you > > might quite reasonably be reusing a chunk that was previously > > allocated to a different instance. > > > > If you are trying to share a chunk between two different instances, > > both instances are hosed. > > > >> My machine is on local disk and there is no chunk sharing. > > > > OK - so there is no chunk sharing. Please do not trust your > > experiments. Please do not ever try to set up chunk sharing. -- Jonathan Leffler #include <disclaimer.h> Email: jleffler@earthlink.net, jleffler@us.ibm.com Guardian of DBD::Informix v2008.0229 -- http://dbi.perl.org/ "Blessed are we who can laugh at ourselves, for we shall never cease to be amused." NB: Please do not use this email for correspondence. I don't necessarily read it every week, even.