IDS11 startup problems
Posted in 2008
A user testing IDS 11 on Red Hat EL4 found that after a power loss or abnormal shutdown the server would not restart: oninit -yv hangs at 'Bringing up ADM VP', onstat reports the server is no longer running, onmode -ky refuses to act, and oninit -i gives 'Fatal error in shared memory creation'. His only workaround was deleting $INFORMIXDIR and reinstalling. Responders suggested checking chunk/ROOTPATH locations, DUMPDIR and assertion files, OS logs, and oninit's setuid permissions/shared libraries, plus the Linux 2.6 need for chunk symlinks to /dev entries recreated at boot. The thread ends with the user giving his ROOTPATH; no resolution is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Performance & Tuning, Installation, Setup & Upgrades, Stored Procedures & SPL, Server Administration, Licensing & Editions
Hi,
I have been testing IDS 11 for a few days and I am enjoying its speed and
other performance factors, especially since my organization still uses
Standard Engine 7.24 but with a firm intention to upgrade to IDS 11 within a
couple of months.
However, I have a big issue that I don't know how to get around:
Whenever there is power loss or the system is shut down abnormally, the IDS
server cannot recover from that and everytime I have to re-install everything
(actually delete the $INFORMIXDIR and then re-install). Obviously this would
be unacceptable if the server was in production and contained real data.
Here are a couple of things that can give the experts out there a starting
point.
1) My OS is Redhat EL4 (1 processor, 512 Mb )
2) My machine's name is gage, $INFORMIXSERVER = "ksems"
3) [root@gage ~]# oninit -yv
Your evaluation license will expire on 2008-08-11 00:00:00
Checking group membership to determine server run mode...succeeded
Reading configuration file '/usr/local/ids/etc/onconfig.ksems'...succeeded
Creating /INFORMIXTMP/.infxdirs...succeeded
Creating infos file "/usr/local/ids/etc/.infos.ksems"...succeeded
Linking conf file "/usr/local/ids/etc/.conf.ksems"...succeeded
Writing to infos file...succeeded
Checking config parameters...Invalid value of DUMPDIR '/usr/informix/tmp' in
onconfig file. Setting it to default value
'/usr/local/ids/tmp'...succeeded
Allocating and attaching to shared memory...succeeded
Creating resident pool 896 kbytes...succeeded
Allocating 2016 kbytes for buffer pool of 2K page size...succeeded
Initializing rhead structure...succeeded
Initializing ASF...succeeded
Initializing Dictionary Cache and SPL Routine Cache...succeeded
Bringing up ADM VP...[root@gage ~]#
4) [root@gage ~]# onstat -
Your evaluation license will expire on 2008-08-11 00:00:00
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
5) [root@gage ~]# onmode -ky
Your evaluation license will expire on 2008-08-11 00:00:00
onmode: Shutdown of the database server shall be done by killing
the database server processes, as the database server is
in Initialization mode and cannot accept onmode commands.
5) [root@gage ~]# oninit -i
Your evaluation license will expire on 2008-08-11 00:00:00
This action will initialize IBM Informix Dynamic Server;
any existing IBM Informix Dynamic Server databases will NOT be accessible -
Do you wish to continue (y/n)? y
oninit: Fatal error in shared memory creation
So guys, I don't know what to do to bring this server backup without
re-installing.
Thanks,
JC.
Where are your chunks located (onstat -d)? Where is $INFORMIXDIR point=
ing
to?
-------------------------------------
Madison Pruet, STSM
IDS Replication Architect
=
"DENIS DIDEROT" =
<nyakizu2000@yaho =
o.com> =
To
Sent by: ids@iiug.org =
ids-bounces@iiug. =
cc
org =
Subj=
ect
IDS11 startup problems [11402] =
02/22/2008 04:01 =
PM =
=
=
Please respond to =
ids@iiug.org =
=
=
Hi,
I have been testing IDS 11 for a few days and I am enjoying its speed a=
nd
other performance factors, especially since my organization still uses
Standard Engine 7.24 but with a firm intention to upgrade to IDS 11 wit=
hin
a
couple of months.
However, I have a big issue that I don't know how to get around:
Whenever there is power loss or the system is shut down abnormally, the=
IDS
server cannot recover from that and everytime I have to re-install
everything
(actually delete the $INFORMIXDIR and then re-install). Obviously this
would
be unacceptable if the server was in production and contained real data=
.
Here are a couple of things that can give the experts out there a start=
ing
point.
1) My OS is Redhat EL4 (1 processor, 512 Mb )
2) My machine's name is gage, $INFORMIXSERVER =3D "ksems"
3) [root@gage ~]# oninit -yv
Your evaluation license will expire on 2008-08-11 00:00:00
Checking group membership to determine server run mode...succeeded
Reading configuration file '/usr/local/ids/etc/onconfig.ksems'...succee=
ded
Creating /INFORMIXTMP/.infxdirs...succeeded
Creating infos file "/usr/local/ids/etc/.infos.ksems"...succeeded
Linking conf file "/usr/local/ids/etc/.conf.ksems"...succeeded
Writing to infos file...succeeded
Checking config parameters...Invalid value of DUMPDIR '/usr/informix/tm=
p'
in
onconfig file. Setting it to default value
'/usr/local/ids/tmp'...succeeded
Allocating and attaching to shared memory...succeeded
Creating resident pool 896 kbytes...succeeded
Allocating 2016 kbytes for buffer pool of 2K page size...succeeded
Initializing rhead structure...succeeded
Initializing ASF...succeeded
Initializing Dictionary Cache and SPL Routine Cache...succeeded
Bringing up ADM VP...[root@gage ~]#
4) [root@gage ~]# onstat -
Your evaluation license will expire on 2008-08-11 00:00:00
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
5) [root@gage ~]# onmode -ky
Your evaluation license will expire on 2008-08-11 00:00:00
onmode: Shutdown of the database server shall be done by killing
the database server processes, as the database server is
in Initialization mode and cannot accept onmode commands.
5) [root@gage ~]# oninit -i
Your evaluation license will expire on 2008-08-11 00:00:00
This action will initialize IBM Informix Dynamic Server;
any existing IBM Informix Dynamic Server databases will NOT be accessib=
le -
Do you wish to continue (y/n)? y
oninit: Fatal error in shared memory creation
So guys, I don't know what to do to bring this server backup without
re-installing.
Thanks,
JC.
***********************************************************************=
********
Forum Note: Use "Reply" to post a response in the discussion forum.
See you at the IIUG Informix 2008 Conference
The Power Conference for Informix Professionals
April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas
http://www.iiug.org/conf
Registration Now Open!!
=
Are you getting anything in your o/s system messages? How about in
'/usr/local/ids/tmp'? The 'oninit -yv' output shows your informix
DUMPDIR in onconfig is set to '/usr/informix/tmp' which the output says
does not exist so it is using '/usr/local/ids/tmp' as DUMPDIR. Maybe
there is something in that dir that would help?
=20
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
DENIS DIDEROT
Sent: Friday, February 22, 2008 4:02 PM
To: ids@iiug.org
Subject: IDS11 startup problems [11402]
Hi,=20
I have been testing IDS 11 for a few days and I am enjoying its speed
and=20
other performance factors, especially since my organization still uses=20
Standard Engine 7.24 but with a firm intention to upgrade to IDS 11
within a=20
couple of months.=20
However, I have a big issue that I don't know how to get around:=20
Whenever there is power loss or the system is shut down abnormally, the
IDS=20
server cannot recover from that and everytime I have to re-install
everything=20
(actually delete the $INFORMIXDIR and then re-install). Obviously this
would=20
be unacceptable if the server was in production and contained real data.
Here are a couple of things that can give the experts out there a
starting=20
point.=20
1) My OS is Redhat EL4 (1 processor, 512 Mb )=20
2) My machine's name is gage, $INFORMIXSERVER =3D "ksems"=20
3) [root@gage ~]# oninit -yv=20
Your evaluation license will expire on 2008-08-11 00:00:00=20
Checking group membership to determine server run mode...succeeded=20
Reading configuration file
'/usr/local/ids/etc/onconfig.ksems'...succeeded=20
Creating /INFORMIXTMP/.infxdirs...succeeded=20
Creating infos file "/usr/local/ids/etc/.infos.ksems"...succeeded=20
Linking conf file "/usr/local/ids/etc/.conf.ksems"...succeeded=20
Writing to infos file...succeeded=20
Checking config parameters...Invalid value of DUMPDIR
'/usr/informix/tmp' in=20
onconfig file. Setting it to default value=20
'/usr/local/ids/tmp'...succeeded=20
Allocating and attaching to shared memory...succeeded=20
Creating resident pool 896 kbytes...succeeded=20
Allocating 2016 kbytes for buffer pool of 2K page size...succeeded=20
Initializing rhead structure...succeeded=20
Initializing ASF...succeeded=20
Initializing Dictionary Cache and SPL Routine Cache...succeeded=20
Bringing up ADM VP...[root@gage ~]#=20
4) [root@gage ~]# onstat -=20
Your evaluation license will expire on 2008-08-11 00:00:00=20
*** WARNING: IBM Informix Dynamic Server is no longer running. ***=20
*** WARNING: IBM Informix Dynamic Server is no longer running. ***=20
5) [root@gage ~]# onmode -ky=20
Your evaluation license will expire on 2008-08-11 00:00:00=20
onmode: Shutdown of the database server shall be done by killing=20
the database server processes, as the database server is=20
in Initialization mode and cannot accept onmode commands.=20
5) [root@gage ~]# oninit -i=20
Your evaluation license will expire on 2008-08-11 00:00:00=20
This action will initialize IBM Informix Dynamic Server;=20
any existing IBM Informix Dynamic Server databases will NOT be
accessible -=20
Do you wish to continue (y/n)? y=20
oninit: Fatal error in shared memory creation=20
So guys, I don't know what to do to bring this server backup without=20
re-installing.=20
Thanks,=20
JC.=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
The information contained in this message may be privileged
and confidential and protected from disclosure. If the reader
of this message is not the intended recipient, or an employee
or agent responsible for delivering this message to the
intended recipient, you are hereby notified that any reproduction,
dissemination or distribution of this communication is strictly
prohibited. If you have received this communication in error,
please notify us immediately by replying to the message and
deleting it from your computer. Thank you. Tellabs
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Hi, Denis,
For some reason, at the second restart, your Informix
is unable to 'fork' to create additional virtual processors.
In this case, it crashes without any reasonable diagnostic output.
Looks like, some permissions (ownership, setuid bit) might get
corrupt after machine's restart... Please, check, that
$INFORMIXDIR/bin/oninit has proper permissions after machine's restart:
ifmx@db3$ ls -l $INFORMIXDIR/bin/oninit-rwsr-sr-- 1 root informix 16736772 Feb 20 13:42 oninit
Also, make sure, that 'oninit' is able to see the required shared
libraries (ldd -v $INFORMIXDIR/bin/oninit)
May be, some RH security-related script is modifying attributes
of Informix binaries to help you protect a very valuable machine
against suspicious software, trying to run with root privileges :-)
-Alexey
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
DENIS DIDEROT
Sent: Friday, February 22, 2008 5:02 PM
To: ids@iiug.org
Subject: IDS11 startup problems [11402]
Hi,
I have been testing IDS 11 for a few days and I am enjoying its speed
and
other performance factors, especially since my organization still uses
Standard Engine 7.24 but with a firm intention to upgrade to IDS 11
within a
couple of months.
However, I have a big issue that I don't know how to get around:
Whenever there is power loss or the system is shut down abnormally, the
IDS
server cannot recover from that and everytime I have to re-install
everything
(actually delete the $INFORMIXDIR and then re-install). Obviously this
would
be unacceptable if the server was in production and contained real data.
Here are a couple of things that can give the experts out there a
starting
point.
1) My OS is Redhat EL4 (1 processor, 512 Mb )
2) My machine's name is gage, $INFORMIXSERVER = "ksems"
3) [root@gage ~]# oninit -yv
Your evaluation license will expire on 2008-08-11 00:00:00
Checking group membership to determine server run mode...succeeded
Reading configuration file
'/usr/local/ids/etc/onconfig.ksems'...succeeded
Creating /INFORMIXTMP/.infxdirs...succeeded
Creating infos file "/usr/local/ids/etc/.infos.ksems"...succeeded
Linking conf file "/usr/local/ids/etc/.conf.ksems"...succeeded
Writing to infos file...succeeded
Checking config parameters...Invalid value of DUMPDIR
'/usr/informix/tmp' in
onconfig file. Setting it to default value
'/usr/local/ids/tmp'...succeeded
Allocating and attaching to shared memory...succeeded
Creating resident pool 896 kbytes...succeeded
Allocating 2016 kbytes for buffer pool of 2K page size...succeeded
Initializing rhead structure...succeeded
Initializing ASF...succeeded
Initializing Dictionary Cache and SPL Routine Cache...succeeded
Bringing up ADM VP...[root@gage ~]#
4) [root@gage ~]# onstat -
Your evaluation license will expire on 2008-08-11 00:00:00
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
5) [root@gage ~]# onmode -ky
Your evaluation license will expire on 2008-08-11 00:00:00
onmode: Shutdown of the database server shall be done by killing
the database server processes, as the database server is
in Initialization mode and cannot accept onmode commands.
5) [root@gage ~]# oninit -i
Your evaluation license will expire on 2008-08-11 00:00:00
This action will initialize IBM Informix Dynamic Server;
any existing IBM Informix Dynamic Server databases will NOT be
accessible -
Do you wish to continue (y/n)? y
oninit: Fatal error in shared memory creation
So guys, I don't know what to do to bring this server backup without
re-installing.
Thanks,
JC.
************************************************************************
*******
Forum Note: Use "Reply" to post a response in the discussion forum.
See you at the IIUG Informix 2008 Conference
The Power Conference for Informix Professionals
April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas
http://www.iiug.org/conf
Registration Now Open!!
I have had family matters that prevented me from providing clarifications
right away. But here we go:
1) Where are your chunks located (onstat -d)? Where is $INFORMIXDIR point=
ing
to?
Response: [root@gage ~]# onstat -d
Your evaluation license will expire on 2008-08-11 00:00:00
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
2) Are you getting anything in your o/s system messages? How about in
'/usr/local/ids/tmp'? The 'oninit -yv' output shows your informix
DUMPDIR in onconfig is set to '/usr/informix/tmp' which the output says
does not exist so it is using '/usr/local/ids/tmp' as DUMPDIR. Maybe
there is something in that dir that would help?
Response: No OS messages (weird, huh?).
I had forgotten to set the DUMPDIR to /usr/local/ids/tmp, but I didn't think
that would be a problem because after the many reinstalls I have done, I
have noticed that if you forget to set it to the correct path, oninit will do
it for you.
3)The server I'm using is a test box and not security-hardened.
I don't know of any script that might be changing any attributes.
[root@gage ~]# ls -l $INFORMIXDIR/bin/oninit
-rwsr-sr-- 1 root informix 14315870 Feb 13 13:56 /usr/local/ids/bin/oninit
I have had family matters that prevented me from providing clarifications
right away. But here we go:
Where are your chunks located (onstat -d)? Where is $INFORMIXDIR point=
ing
to?
Response: [root@gage ~]# onstat -d
Your evaluation license will expire on 2008-08-11 00:00:00
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
Are you getting anything in your o/s system messages? How about in
'/usr/local/ids/tmp'? The 'oninit -yv' output shows your informix
DUMPDIR in onconfig is set to '/usr/informix/tmp' which the output says
does not exist so it is using '/usr/local/ids/tmp' as DUMPDIR. Maybe
there is something in that dir that would help?
Response: No OS messages (weird, huh?).
I had forgotten to set the DUMPDIR to /usr/local/ids/tmp, but I didn't think
that would be a problem because after the many reinstalls I have done, I
have noticed that if you forget to set it to the correct path, oninit will do
it for you.
Where did you create your rootdbs (ROOTPATH in $ONCONFIG)? In Linux
2.6 kernel (your RH EL 4) it should be a symbolic link from some
dir/mountpoint you created to a /dev entry (that gets rebuilt every
reboot).
Bob Roussey
Unix / Informix Administration
Spirit Airlines
Robert.Roussey@SpiritAir.com
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
DENIS DIDEROT
Sent: Monday, February 25, 2008 3:06 PM
To: ids@iiug.org
Subject: Re: IDS11 startup problems [11427]
I have had family matters that prevented me from providing
clarifications
right away. But here we go:
Where are your chunks located (onstat -d)? Where is $INFORMIXDIR point=
ing
to?
Response: [root@gage ~]# onstat -d
Your evaluation license will expire on 2008-08-11 00:00:00
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
*** WARNING: IBM Informix Dynamic Server is no longer running. ***
************************************************************************
*******
Forum Note: Use "Reply" to post a response in the discussion forum.
See you at the IIUG Informix 2008 Conference
The Power Conference for Informix Professionals
April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas
http://www.iiug.org/conf
Registration Now Open!!
ROOTPATH /usr/local/ids/demo/server/online_root
Related threads
- IDS 10 table-level restore
- Informix Development Webinar December 11, 2007
- ontape -p/r with changed ROOTPATH
- Migrate from HP PA-RISC to HP ITANIUM by ontape