Instance Ignoring Onmode Commands
A 7.31 HDR primary had fallen out of sync (logs rolled past the secondary) and needed to be switched to standard mode for a fresh archive/restore, but it ignored all onmode commands (-d standard, -ky) with no message-log output, while onstat and ontape still worked and users were unaffected. Replies said this indicates the engine is wedged and cannot respond: check INFORMIXSERVER/ONCONFIG environment, kill any hung onmode processes, otherwise kill the main oninit (Art suggested kill -15 on CPU VP #1 and the ADM VP just after a checkpoint, with everyone off), clear shared memory, /INFORMIXTMP and the TCP port, then restart. No tidier fix was found, and the poster didn't report the outcome.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Hi
I have a HDR 7:31 system which is out of sync and needs a restore from archive
as the logs have rolled on the primary.
Trouble is the primary will not respond to onmode commends so I can't set it
to standard or shut it down.
I have checked the site for hints on this with no luck. It's over 40Gb and
hake a long time to backup and restore.
Hi,
please check the settings in the environment in which you
execute onmode for the primary server (INFORMIXSERVER,
ONCONFIG, etc.). Is there any message when you execute
onmode and it does not have any effect? That might give
you a clue as to what is missing ...
Also try onstat to see if it is trying to connect to the correct
server instance (i.e. the primary).
Regards, Martin
--
Martin Fuerderer
IBM Informix Development Munich, Germany
Information Management
IBM Deutschland Research & Development GmbH
Chairman of the Supervisory Board: Martin Jetter
Board of Management: Erich Baier
Corporate Seat: Boeblingen, Germany
Reg.-Gericht: Amtsgericht Stuttgart, HRB 243294
From:
"PETER LANE" <pglxpglx@ntlworld.com>
To:
ids@iiug.org
Date:
08/13/2009 10:12 AM
Subject:
Instance Ignoring Onmode Commands [16671]
Sent by:
ids-bounces@iiug.org
Hi
I have a HDR 7:31 system which is out of sync and needs a restore from
archive
as the logs have rolled on the primary.
Trouble is the primary will not respond to onmode commends so I can't set
it
to standard or shut it down.
I have checked the site for hints on this with no luck. It's over 40Gb and
hake a long time to backup and restore.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
The onstats work and ontape. Just onmode. It does not respond no matter how
long the command is left there. All the functionality works, system connect
etc. Trouble is even if you want to change the config, it will only take
effect after a restart of the instance. Ideally what I want to do is change
the mode to standard, get a backup and restore to the secondary and
re-establish HDR. Done this many times due to kit failing but not experienced
onmode (-d standard, -yuk any onmode command) being ignored. There is nothing
in the message log when the command is issued, work carries on as normal from
a user point of view.
2009/8/13 PETER LANE <pglxpglx@ntlworld.com>:
> The onstats work and ontape. Just onmode. It does not respond no matter how
> long the command is left there. All the functionality works, system connect
> etc. Trouble is even if you want to change the config, it will only take
> effect after a restart of the instance. Ideally what I want to do is change
> the mode to standard, get a backup and restore to the secondary and
> re-establish HDR. Done this many times due to kit failing but not experienced
> onmode (-d standard, -yuk any onmode command) being ignored. There is nothing
> in the message log when the command is issued, work carries on as normal from
> a user point of view.
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
Peter
This usually indicates the engine is in such a state that it cannot
respond, particularly if onmode -k will not stop things. Raise a call
if you have support, else you will need to kill the main oninit
process. (You will probably end up killing this anyway !!). The engine
should then restart successfully in what ever mode you need.
Keith
If you're going to restore it can't you just kill the engine and clear up shared memory manually? It doesn't explain the problem, but does get you going again. Regards Simon
Thanks Keith. That was my last option. I was hoping to find a tidier way to influence the instance to respond to onmodes. Yes the instance should come back but this instance is fundamental and should is a risk I want to avoid. Simon, I want to restore the secondary from a back of the primary in standard mode which the primary is not at the moment. Restoring a secondary is the same procedure as starting HDR for the first time. I.e. from a backup in standard mode.
2009/8/13 PETER LANE <pglxpglx@ntlworld.com>:
> Thanks Keith. That was my last option. I was hoping to find a tidier way to
> influence the instance to respond to onmodes. Yes the instance should come
> back but this instance is fundamental and should is a risk I want to avoid.
> Simon, I want to restore the secondary from a back of the primary in standard
> mode which the primary is not at the moment. Restoring a secondary is the
same
> procedure as starting HDR for the first time. I.e. from a backup in standard
> mode.
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
Peter
Unfortunately I think this is now your only option ofr getting this
server to respond correctly :-( Is the instance still checkpointing
and logging correctly, are there any outstanding onmode commands still
running (ps -ef |grep onmode) if so it might be worth killing them and
trying again.
If you do end up crashing the engine to get it stopped make sure all
shared memory segments are removed, that /INFORMIXTMP is cleared and
(if you are using tcp connections) the port is released else you may
need a server reboot (bitter experiance !!)
keith
Might want to try a "kill -6" on the main oninit. I believe that will create a
core dump for any further investigations you might want to pursue.
Bob
----- Original Message -----
From: "Keith Simmons" <smiley73@googlemail.com>
To: ids@iiug.org
Sent: Thursday, August 13, 2009 6:23:07 AM GMT -05:00 US/Canada Eastern
Subject: Re: Instance Ignoring Onmode Commands [16677]
2009/8/13 PETER LANE <pglxpglx@ntlworld.com>:
> Thanks Keith. That was my last option. I was hoping to find a tidier way to
> influence the instance to respond to onmodes. Yes the instance should come
> back but this instance is fundamental and should is a risk I want to avoid.
> Simon, I want to restore the secondary from a back of the primary in
standard
> mode which the primary is not at the moment. Restoring a secondary is the
same
> procedure as starting HDR for the first time. I.e. from a backup in standard
> mode.
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
Peter
Unfortunately I think this is now your only option ofr getting this
server to respond correctly :-( Is the instance still checkpointing
and logging correctly, are there any outstanding onmode commands still
running (ps -ef |grep onmode) if so it might be worth killing them and
trying again.
If you do end up crashing the engine to get it stopped make sure all
shared memory segments are removed, that /INFORMIXTMP is cleared and
(if you are using tcp connections) the port is released else you may
need a server reboot (bitter experiance !!)
keith
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
You will have to force it to crash. Get everyone off the instance, then wait
until just after a checkpoint (to minimize recovery time) and kill -15 the
master oninit (CPU VP #1) and the ADM vp - see onstat -g glo for the pids of
these processes. Then you should be able to use onmode -ky to remove shared
memory and then try to restart the instance and take the archive.
Art
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
On Thu, Aug 13, 2009 at 4:12 AM, PETER LANE <pglxpglx@ntlworld.com> wrote:
> Hi
> I have a HDR 7:31 system which is out of sync and needs a restore from
> archive
> as the logs have rolled on the primary.
> Trouble is the primary will not respond to onmode commends so I can't set
> it
> to standard or shut it down.
> I have checked the site for hints on this with no luck. It's over 40Gb and
> hake a long time to backup and restore.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--000e0ce044b4aa408d04710b41f5
Related threads
- Posting from the Informix-list
- Migrating from IDS 9.40.UC6 to 11.50.UC3
- Ip for a network session
- questions onstat -g