I dropped from 3.5GB of RAM to 2.5GB and 4GL quit working

Posted in 2006

Summary

After moving a server and removing 1GB of RAM, all Informix 4GL reports on IFX 7.31.UD8 / RedHat 7.3 compiled fine but died with SIGSEGV at startup (no core). strace/gdb showed the crash inside lib4gsh730.so while reading GLS locale files during runtime initialisation, suggesting a bad library or corrupt locale data rather than a memory/engine issue. The poster compared md5sums against a development box, found lib4gsh730.so differed, replaced it, and 4GL worked again — the corruption was blamed on an earlier hung shutdown/fsck of the ext3 filesystem, not the RAM change.

Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.

Topics: Connectivity: ESQL/C, 4GL & Embedded SQL, Internationalization & Character Sets

Anthony — 19 Jun 2006 08:23:05 -0700 — source: Usenet: comp.databases.informix

Help? We made a physical move, and after some looking / poking around, decided to move 1GB of RAM from our db server and put it into our mail server. Simple change. Brought both systems up, one now reads 2GB (it read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No configurations were changed (I'm the only admin, and did the physical move and swap myself). However, all of the 4GL reports quit working. We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP 1900) system. I can compile the 4GL progams perfectly, but when run, they show "Segmentation Fault", when run as users root, informix, or myself (anthony). No core file is produced. I tried re-compiling all of the programs, but that hasn't helped at all. strace says this (I'm only including the last few lines): open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 close(3) = 0 munmap(0x40151000, 4096) = 0 open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 close(3) = 0 munmap(0x40151000, 4096) = 0 open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 brk(0x8059000) = 0x8059000 --- SIGSEGV (Segmentation fault) --- +++ killed by SIGSEGV +++ Thoughts? Anything would be appreciated ...... I'm at a loss at the moment. -- Anthony

↪ replying to Anthony

Marco Greco — Mon, 19 Jun 2006 16:32:04 +0100 — source: Usenet: comp.databases.informix

Anthony wrote: > Help? > > We made a physical move, and after some looking / poking around, > decided to move 1GB of RAM from our db server and put it into our mail > server. Simple change. Brought both systems up, one now reads 2GB (it > read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No > configurations were changed (I'm the only admin, and did the physical > move and swap myself). Please clarify physical move? did you change hardware? and if yes, are you running the same version of the OS? how did you reinstall the informix software? > However, all of the 4GL reports quit working. > > We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP > 1900) system. > > I can compile the 4GL progams perfectly, but when run, they show > "Segmentation Fault", when run as users root, informix, or myself > (anthony). No core file is produced. > > I tried re-compiling all of the programs, but that hasn't helped at > all. > > strace says this (I'm only including the last few lines): > > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > close(3) = 0 > munmap(0x40151000, 4096) = 0 > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > close(3) = 0 > munmap(0x40151000, 4096) = 0 > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > brk(0x8059000) = 0x8059000 > --- SIGSEGV (Segmentation fault) --- > +++ killed by SIGSEGV +++ Nothing to do with the engine. your 4gl app is segv'ing. if you don't get a core, try running the app through gdb, and get a stack after it fails. I'd say you are using a different libc, or picking up a wrong library along the way, or.... what does an ldd <your app> say? > > Thoughts? > > Anything would be appreciated ...... I'm at a loss at the moment. > > -- > Anthony -- Ciao, Marco ______________________________________________________________________________ Marco Greco /UK /IBM Standard disclaimers apply! Structured Query Scripting Language http://www.4glworks.com/sqsl.htm 4glworks http://www.4glworks.com Informix on Linux http://www.4glworks.com/ifmxlinux.htm

↪ replying to Marco Greco

Anthony — 19 Jun 2006 10:37:07 -0700 — source: Usenet: comp.databases.informix

Reply below... Marco Greco wrote: > Anthony wrote: > > Help? > > > > We made a physical move, and after some looking / poking around, > > decided to move 1GB of RAM from our db server and put it into our mail > > server. Simple change. Brought both systems up, one now reads 2GB (it > > read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No > > configurations were changed (I'm the only admin, and did the physical > > move and swap myself). > > Please clarify physical move? did you change hardware? and if yes, are you > running the same version of the OS? how did you reinstall the informix software? Physical move = Unplugged box. Drove box to new location. Opened box, pulled out (2) 512MB sticks of RAM. Replaced box cover. Power box up. No changes to any configuration files, OS, or Informix. > > However, all of the 4GL reports quit working. > > > > We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP > > 1900) system. > > > > I can compile the 4GL progams perfectly, but when run, they show > > "Segmentation Fault", when run as users root, informix, or myself > > (anthony). No core file is produced. > > > > I tried re-compiling all of the programs, but that hasn't helped at > > all. > > > > strace says this (I'm only including the last few lines): > > > > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > > 0) = 0x40151000 > > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > > close(3) = 0 > > munmap(0x40151000, 4096) = 0 > > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > > 0) = 0x40151000 > > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > > close(3) = 0 > > munmap(0x40151000, 4096) = 0 > > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > > 0) = 0x40151000 > > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > > brk(0x8059000) = 0x8059000 > > --- SIGSEGV (Segmentation fault) --- > > +++ killed by SIGSEGV +++ > > Nothing to do with the engine. your 4gl app is segv'ing. if you don't get a > core, try running the app through gdb, and get a stack after it fails. > I'd say you are using a different libc, or picking up a wrong library along > the way, or.... > what does an ldd <your app> say? [Update: I've shut down the box, replaced the missing RAM, and started back up. Still having the problem. More details below.] Ok ... no, no core is generated. I'll try gdb. ldd shows: [anthony@corp src]$ ldd ./commissionSummary lib4gsh730.so => /opt/informix/lib/tools/lib4gsh730.so (0x40014000) libcrypt.so.1 => /lib/libcrypt.so.1 (0x40159000) libm.so.6 => /lib/i686/libm.so.6 (0x40186000) libdl.so.2 => /lib/libdl.so.2 (0x401a8000) libc.so.6 => /lib/i686/libc.so.6 (0x42000000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) gdb shows: (gdb) run commissionSummary Starting program: /home/anthony/reports/src/commissionSummary commissionSummary Program received signal SIGSEGV, Segmentation fault. 0x400b6b7a in _ifx_gl_full_mbtowc () from /opt/informix/lib/tools/lib4gsh730.so (gdb) up #1 0x400b6c4f in gl_full_mbtowc () from /opt/informix/lib/tools/lib4gsh730.so (gdb) down #0 0x400b6b7a in _ifx_gl_full_mbtowc () from /opt/informix/lib/tools/lib4gsh730.so (gdb) bt #0 0x400b6b7a in _ifx_gl_full_mbtowc () from /opt/informix/lib/tools/lib4gsh730.so #1 0x400b6c4f in gl_full_mbtowc () from /opt/informix/lib/tools/lib4gsh730.so #2 0x400d7735 in _gl_cache_wc_string () from /opt/informix/lib/tools/lib4gsh730.so #3 0x400c0b84 in cache_strings_for_lookup () from /opt/informix/lib/tools/lib4gsh730.so #4 0x400c0d9f in init_time_info () from /opt/informix/lib/tools/lib4gsh730.so #5 0x400c1829 in read_one_category () from /opt/informix/lib/tools/lib4gsh730.so #6 0x400c2cff in load_category () from /opt/informix/lib/tools/lib4gsh730.so #7 0x400c38ff in set_categories () from /opt/informix/lib/tools/lib4gsh730.so #8 0x400c3acc in gl_setlocale_4 () from /opt/informix/lib/tools/lib4gsh730.so #9 0x400da797 in get_locale () from /opt/informix/lib/tools/lib4gsh730.so #10 0x400daac5 in init_fe () from /opt/informix/lib/tools/lib4gsh730.so #11 0x400daedd in initgls () from /opt/informix/lib/tools/lib4gsh730.so #12 0x400daf5d in chkglsinit () from /opt/informix/lib/tools/lib4gsh730.so #13 0x40110609 in CheckOsInit () from /opt/informix/lib/tools/lib4gsh730.so #14 0x4010eda1 in greadenv () from /opt/informix/lib/tools/lib4gsh730.so #15 0x4010f163 in ggetenv () from /opt/informix/lib/tools/lib4gsh730.so #16 0x40090845 in ix_infmxdir () from /opt/informix/lib/tools/lib4gsh730.so #17 0x4008f1f7 in fechkenv () from /opt/informix/lib/tools/lib4gsh730.so #18 0x40098a49 in fgl_init () from /opt/informix/lib/tools/lib4gsh730.so #19 0x08049946 in main (fgl_argc=2, fgl_argv=0xbfffd644) at commissionSummary.ec:57 #20 0x42017589 in __libc_start_main () from /lib/i686/libc.so.6 Helpful? Possibility that this library is corrupt (lib4gsh730.so)? The other libc's on my systerm are: [anthony@corp src]$ slocate *libc.so* /usr/lib/libc.so /usr/i386-glibc21-linux/lib/libc.so.6 /usr/i386-glibc21-linux/lib/libc.so /lib/i686/libc.so.6 /lib/libc.so.6 -- Anthony > > > > Thoughts? > > > > Anything would be appreciated ...... I'm at a loss at the moment. > > > > -- > > Anthony > > -- > Ciao, > Marco > ______________________________________________________________________________ > Marco Greco /UK /IBM Standard disclaimers apply! > > Structured Query Scripting Language http://www.4glworks.com/sqsl.htm > 4glworks http://www.4glworks.com > Informix on Linux http://www.4glworks.com/ifmxlinux.htm

↪ replying to Anthony

DL Redden — Mon, 19 Jun 2006 13:42:41 -0700 (PDT) — source: Usenet: comp.databases.informix

If your using 32 bit software which it looks like you are "7.31.UD8", I can only assume that your using 32 bit 4GL as well. I'm not sure about RH but I know that on HP-UX ALL of the 32 bit applications have to share the same 1.75 GB of memory. Anything above that is wasted RAM. If your engine is chewing up the majority of 1.75 GB then your 4GL apps will not have any room to run. ----- Original Message ---- From: Anthony <ressystems@gmail.com> To: informix-list@iiug.org Sent: Monday, June 19, 2006 10:23:05 AM Subject: I dropped from 3.5GB of RAM to 2.5GB and 4GL quit working Help? We made a physical move, and after some looking / poking around, decided to move 1GB of RAM from our db server and put it into our mail server. Simple change. Brought both systems up, one now reads 2GB (it read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No configurations were changed (I'm the only admin, and did the physical move and swap myself). However, all of the 4GL reports quit working. We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP 1900) system. I can compile the 4GL progams perfectly, but when run, they show "Segmentation Fault", when run as users root, informix, or myself (anthony). No core file is produced. I tried re-compiling all of the programs, but that hasn't helped at all. strace says this (I'm only including the last few lines): open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 close(3) = 0 munmap(0x40151000, 4096) = 0 open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 close(3) = 0 munmap(0x40151000, 4096) = 0 open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 brk(0x8059000) = 0x8059000 --- SIGSEGV (Segmentation fault) --- +++ killed by SIGSEGV +++ Thoughts? Anything would be appreciated ...... I'm at a loss at the moment. -- Anthony _______________________________________________ Informix-list mailing list Informix-list@iiug.org http://www.iiug.org/mailman/listinfo/informix-list

↪ replying to DL Redden

Anthony — 19 Jun 2006 15:32:30 -0700 — source: Usenet: comp.databases.informix

DL, Thanks for the response. I'm going to be bringing down everything except oninit tonight, and seeing if I have the same problems with the 4GL apps. I suppose it's possible that something is going on with the RAM being taken up (it looks like it's seg-faulting while trying to request memory?). I'm even going to try to roll back to an older kernel, however, we've been running on this one for more than 2.5 years. Not sure that rolling back will do more harm than good. Otherwise, it looks like we'll be running to Crystal Reports even faster than I thought. Any more good ideas? [Sidenote: Not to bash the db, but all of my major unsolvable problems that I get while admining this particular system, for the past 7 years, have all been from Informix or Informix/system issues. Strange.] -- Anthony DL Redden wrote: > If your using 32 bit software which it looks like you are "7.31.UD8", I can only assume that your using 32 bit 4GL as well. I'm not sure about RH but I know that on HP-UX ALL of the 32 bit applications have to share the same 1.75 GB of memory. Anything above that is wasted RAM. If your engine is chewing up the majority of 1.75 GB then your 4GL apps will not have any room to run. > > > ----- Original Message ---- > From: Anthony <ressystems@gmail.com> > To: informix-list@iiug.org > Sent: Monday, June 19, 2006 10:23:05 AM > Subject: I dropped from 3.5GB of RAM to 2.5GB and 4GL quit working > > Help? > > We made a physical move, and after some looking / poking around, > decided to move 1GB of RAM from our db server and put it into our mail > server. Simple change. Brought both systems up, one now reads 2GB (it > read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No > configurations were changed (I'm the only admin, and did the physical > move and swap myself). > > However, all of the 4GL reports quit working. > > We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP > 1900) system. > > I can compile the 4GL progams perfectly, but when run, they show > "Segmentation Fault", when run as users root, informix, or myself > (anthony). No core file is produced. > > I tried re-compiling all of the programs, but that hasn't helped at > all. > > strace says this (I'm only including the last few lines): > > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > close(3) = 0 > munmap(0x40151000, 4096) = 0 > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > close(3) = 0 > munmap(0x40151000, 4096) = 0 > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > brk(0x8059000) = 0x8059000 > --- SIGSEGV (Segmentation fault) --- > +++ killed by SIGSEGV +++ > > Thoughts? > > Anything would be appreciated ...... I'm at a loss at the moment. > > -- > Anthony > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list > > > > > > --0-520770435-1150749761=:91863 > Content-Type: text/html > X-Google-AttachSize: 4039 > > <html><head><style type="text/css"></style></head><body><div style="font-family:times new roman, new york, times, serif;font-size:12pt">If your using 32 bit software which it looks like you are "7.31.UD8", I can only assume that your using 32 bit 4GL as well. I'm not sure about RH but I know that on HP-UX ALL of the 32 bit applications have to share the same 1.75 GB of memory. Anything above that is wasted RAM. If your engine is chewing up the majority of 1.75 GB then your 4GL apps will not have any room to run. <div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"> <div style="font-family: times new roman,new york,times,serif; font-size: 12pt;">----- Original Message ---- From: Anthony <ressystems@gmail.com> To: informix-list@iiug.org Sent: Monday, June 19, 2006 10:23:05 AM Subject: I dropped from 3.5GB of RAM to 2.5GB and 4GL quit working <div>Help? We made a physical move, and > after some looking / poking around, decided to move 1GB of RAM from our db server and put it into our mail server.  Simple change.  Brought both systems up, one now reads 2GB (it read 1GB before), and the other reads 2.5GB (it read 3.5GB before).  No configurations were changed (I'm the only admin, and did the physical move and swap myself). However, all of the 4GL reports quit working. We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP 1900) system. I can compile the 4GL progams perfectly, but when run, they show "Segmentation Fault", when run as users root, informix, or myself (anthony).  No core file is produced. I tried re-compiling all of the programs, but that hasn't helped at all. strace says this (I'm only including the last few lines): open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, > st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 close(3)                                = 0 munmap(0x40151000, 4096)                = 0 open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = > 2628 close(3)                                = 0 munmap(0x40151000, 4096)                = 0 open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40151000 read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 brk(0x8059000)&n

↪ replying to Anthony

Anthony — 19 Jun 2006 22:43:54 -0700 — source: Usenet: comp.databases.informix

All, I'm replying to the top level, to ensure that everyone has a record of this particular problem. I seem to run across errors that only Informix finds (yes, Obnoxio, perhaps Informix is the only one running hard enuf to find these problems). When I shut down the system at the last location, it hung while trying to unmount an NFS filesystem (that system had errantly already been shut down). As such, when I powered it back up, I forced a fsck of the ext3 file system, which completed without any major errors (ie, I didn't have to get involved). The system booted up, and all was well. Except for the 4GL (we actually have less than 2MB of 4GL binary files, tied up in about 36 reports), that is. I looked back through the strace and gdb stack, and decided to investigate the lib4gsh730.so file. The md5sum on our development system and the production system didn't match, so I replaced the file, and it appears to work. All of the 4GL appears to be working now. Needless to say, I'm at a bit of a loss. Either I've got massive amounts of corruption, and I don't know about it, or the hung reboot corrupted the individual file. Thoughts? -- Anthony Anthony wrote: > Help? > > We made a physical move, and after some looking / poking around, > decided to move 1GB of RAM from our db server and put it into our mail > server. Simple change. Brought both systems up, one now reads 2GB (it > read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No > configurations were changed (I'm the only admin, and did the physical > move and swap myself). > > However, all of the 4GL reports quit working. > > We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP > 1900) system. > > I can compile the 4GL progams perfectly, but when run, they show > "Segmentation Fault", when run as users root, informix, or myself > (anthony). No core file is produced. > > I tried re-compiling all of the programs, but that hasn't helped at > all. > > strace says this (I'm only including the last few lines): > > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > close(3) = 0 > munmap(0x40151000, 4096) = 0 > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > close(3) = 0 > munmap(0x40151000, 4096) = 0 > open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 > fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, > 0) = 0x40151000 > read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 > brk(0x8059000) = 0x8059000 > --- SIGSEGV (Segmentation fault) --- > +++ killed by SIGSEGV +++ > > Thoughts? > > Anything would be appreciated ...... I'm at a loss at the moment. > > -- > Anthony

↪ replying to Anthony

Marco Greco — Tue, 20 Jun 2006 09:41:56 +0100 — source: Usenet: comp.databases.informix

Anthony wrote: [snip] > (gdb) run commissionSummary > Starting program: /home/anthony/reports/src/commissionSummary > commissionSummary > > Program received signal SIGSEGV, Segmentation fault. > 0x400b6b7a in _ifx_gl_full_mbtowc () from > /opt/informix/lib/tools/lib4gsh730.so > > (gdb) up > #1 0x400b6c4f in gl_full_mbtowc () from > /opt/informix/lib/tools/lib4gsh730.so > > (gdb) down > #0 0x400b6b7a in _ifx_gl_full_mbtowc () > from /opt/informix/lib/tools/lib4gsh730.so > > (gdb) bt > #0 0x400b6b7a in _ifx_gl_full_mbtowc () > from /opt/informix/lib/tools/lib4gsh730.so > #1 0x400b6c4f in gl_full_mbtowc () from > /opt/informix/lib/tools/lib4gsh730.so > #2 0x400d7735 in _gl_cache_wc_string () > from /opt/informix/lib/tools/lib4gsh730.so > #3 0x400c0b84 in cache_strings_for_lookup () > from /opt/informix/lib/tools/lib4gsh730.so > #4 0x400c0d9f in init_time_info () from > /opt/informix/lib/tools/lib4gsh730.so > #5 0x400c1829 in read_one_category () > from /opt/informix/lib/tools/lib4gsh730.so > #6 0x400c2cff in load_category () from > /opt/informix/lib/tools/lib4gsh730.so > #7 0x400c38ff in set_categories () from > /opt/informix/lib/tools/lib4gsh730.so > #8 0x400c3acc in gl_setlocale_4 () from > /opt/informix/lib/tools/lib4gsh730.so > #9 0x400da797 in get_locale () from > /opt/informix/lib/tools/lib4gsh730.so > #10 0x400daac5 in init_fe () from /opt/informix/lib/tools/lib4gsh730.so > #11 0x400daedd in initgls () from /opt/informix/lib/tools/lib4gsh730.so > #12 0x400daf5d in chkglsinit () from > /opt/informix/lib/tools/lib4gsh730.so > #13 0x40110609 in CheckOsInit () from > /opt/informix/lib/tools/lib4gsh730.so > #14 0x4010eda1 in greadenv () from > /opt/informix/lib/tools/lib4gsh730.so > #15 0x4010f163 in ggetenv () from /opt/informix/lib/tools/lib4gsh730.so > #16 0x40090845 in ix_infmxdir () from > /opt/informix/lib/tools/lib4gsh730.so > #17 0x4008f1f7 in fechkenv () from > /opt/informix/lib/tools/lib4gsh730.so > #18 0x40098a49 in fgl_init () from > /opt/informix/lib/tools/lib4gsh730.so > #19 0x08049946 in main (fgl_argc=2, fgl_argv=0xbfffd644) > at commissionSummary.ec:57 > #20 0x42017589 in __libc_start_main () from /lib/i686/libc.so.6 > > Helpful? actually yes - I don't have 4gl source at hand but you seem to be segv'ing while parsing locale files during the 4gl runtime initialization. maybe some corruption in your gls directory/locale files? -- Ciao, Marco ______________________________________________________________________________ Marco Greco /UK /IBM Standard disclaimers apply! Structured Query Scripting Language http://www.4glworks.com/sqsl.htm 4glworks http://www.4glworks.com Informix on Linux http://www.4glworks.com/ifmxlinux.htm

↪ replying to Anthony

Marco Greco — Tue, 20 Jun 2006 09:51:39 +0100 — source: Usenet: comp.databases.informix

Anthony wrote: reports), that is. > > I looked back through the strace and gdb stack, and decided to > investigate the lib4gsh730.so file. The md5sum on our development > system and the production system didn't match, so I replaced the file, > and it appears to work. All of the 4GL appears to be working now. > Oh! wasn't too far off though -- Ciao, Marco ______________________________________________________________________________ Marco Greco /UK /IBM Standard disclaimers apply! Structured Query Scripting Language http://www.4glworks.com/sqsl.htm 4glworks http://www.4glworks.com Informix on Linux http://www.4glworks.com/ifmxlinux.htm

↪ replying to Anthony

ChrisSalch — Tue, 20 Jun 2006 08:18:48 -0500 — source: Usenet: comp.databases.informix

Some earlier versions of ext3 don't like being shutdown improperly, in some cases it's no better than ext2. Rieserfs seems to do a little bit better with improper shutdowns than some of the others. Chris S. Anthony wrote: > All, > > I'm replying to the top level, to ensure that everyone has a record of > this particular problem. I seem to run across errors that only > Informix finds (yes, Obnoxio, perhaps Informix is the only one running > hard enuf to find these problems). > > When I shut down the system at the last location, it hung while trying > to unmount an NFS filesystem (that system had errantly already been > shut down). As such, when I powered it back up, I forced a fsck of the > ext3 file system, which completed without any major errors (ie, I > didn't have to get involved). > > The system booted up, and all was well. Except for the 4GL (we > actually have less than 2MB of 4GL binary files, tied up in about 36 > reports), that is. > > I looked back through the strace and gdb stack, and decided to > investigate the lib4gsh730.so file. The md5sum on our development > system and the production system didn't match, so I replaced the file, > and it appears to work. All of the 4GL appears to be working now. > > Needless to say, I'm at a bit of a loss. Either I've got massive > amounts of corruption, and I don't know about it, or the hung reboot > corrupted the individual file. > > Thoughts? > > -- > Anthony > > > Anthony wrote: > >> Help? >> >> We made a physical move, and after some looking / poking around, >> decided to move 1GB of RAM from our db server and put it into our mail >> server. Simple change. Brought both systems up, one now reads 2GB (it >> read 1GB before), and the other reads 2.5GB (it read 3.5GB before). No >> configurations were changed (I'm the only admin, and did the physical >> move and swap myself). >> >> However, all of the 4GL reports quit working. >> >> We're running Informix 7.31.UD8, on a RedHat 7.3 SMP (Dual Athlon MP >> 1900) system. >> >> I can compile the 4GL progams perfectly, but when run, they show >> "Segmentation Fault", when run as users root, informix, or myself >> (anthony). No core file is produced. >> >> I tried re-compiling all of the programs, but that hasn't helped at >> all. >> >> strace says this (I'm only including the last few lines): >> >> open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 >> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, >> 0) = 0x40151000 >> read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 >> close(3) = 0 >> munmap(0x40151000, 4096) = 0 >> open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 >> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, >> 0) = 0x40151000 >> read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 >> close(3) = 0 >> munmap(0x40151000, 4096) = 0 >> open("/opt/informix/gls/lc11/en_us/0333.lco", O_RDONLY) = 3 >> fstat64(3, {st_mode=S_IFREG|0644, st_size=2628, ...}) = 0 >> mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, >> 0) = 0x40151000 >> read(3, "FILE_TYPE=locale_object\\nCOMPILER"..., 4096) = 2628 >> brk(0x8059000) = 0x8059000 >> --- SIGSEGV (Segmentation fault) --- >> +++ killed by SIGSEGV +++ >> >> Thoughts? >> >> Anything would be appreciated ...... I'm at a loss at the moment. >> >> -- >> Anthony >> > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list >

Glossary

4GL (Informix-4GL)