identifying a lock seemingly without an owner
Andy G (IDS 10.00.FC10 on Solaris 10) saw locks in onstat -k whose owner address was shown in parentheses and didn't appear in onstat -u, and asked how to clear them short of restarting; the locks reappeared immediately after a bounce with no application connections. onstat -G showed no global transactions. Suggestions: it looks like an orphaned XA/distributed (2-phase commit) transaction — check the owner against onstat -x, force log switches with onmode -l until a long transaction triggers then roll back with the undocumented onmode -H (or a small Java program attaching to the XA transaction), or possibly onmode -Z on the transaction address, and otherwise open a PMR. No confirmation of what actually worked is recorded in the thread.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
All,
Sun Solaris 10
Informix V10.00.FC10
All,
I have an instance with a few locks which do not appear to have an owner. In
the list below you have locks on a table with the number 80005a. As you can
see the owner is in brackets. When I look for that owner in the onstat -u
output nothing is found.
How can I identify the owner and therefore kill the locks other than by
bouncing the instance?
Thanks
Andy G
233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
233cdfc8 0 1b481278 0 S 100002 205 0
2378ad28 0 1b480b10 0 S 100002 205 0
2378b1d8 0 1b494988 0 S 100002 205 0
2378b7f0 0 1f9a3fd0 0 S 100002 205 0
2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
2378c3a8 0 1f99c950 0 S 100002 205 0
2378cfd8 0 1f9adb58 0 S 100002 205 0
2378e388 0 1b483ee8 0 S 100002 205 0
2378e4f0 0 1b482148 0 S 100002 205 0
23792e10 0 1f9a6c40 0 S 100002 205 0
237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
237948c8 0 1f999ce0 0 S 100002 205 0
Hello Andrew,
Nice to "see" you.
Here's some info:
http://publib.boulder.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm
.adref.doc%2Fids_adr_0597.htm
Does it apply to your situation?
Regards
On Mon, Jun 3, 2013 at 4:37 PM, Andrew Grantham <agrantha@hotmail.com>wrote:
> All,
>
> Sun Solaris 10
> Informix V10.00.FC10
>
> All,
>
> I have an instance with a few locks which do not appear to have an owner.
> In
> the list below you have locks on a table with the number 80005a. As you can
> see the owner is in brackets. When I look for that owner in the onstat -u
> output nothing is found.
> How can I identify the owner and therefore kill the locks other than by
> bouncing the instance?
>
> Thanks
>
> Andy G
>
> 233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
> 233cdfc8 0 1b481278 0 S 100002 205 0
> 2378ad28 0 1b480b10 0 S 100002 205 0
> 2378b1d8 0 1b494988 0 S 100002 205 0
> 2378b7f0 0 1f9a3fd0 0 S 100002 205 0
> 2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
> 2378c3a8 0 1f99c950 0 S 100002 205 0
> 2378cfd8 0 1f9adb58 0 S 100002 205 0
> 2378e388 0 1b483ee8 0 S 100002 205 0
> 2378e4f0 0 1b482148 0 S 100002 205 0
> 23792e10 0 1f9a6c40 0 S 100002 205 0
> 237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
> 237948c8 0 1f999ce0 0 S 100002 205 0
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Fernando Nunes
Portugal
http://informix-technology.blogspot.com
My email works... but I don't check it frequently...
--20cf3071cbb4ceb3dd04de42291d
It was a new feature in 10.00.xC5. Added to the documentation notes:
http://publib.boulder.ibm.com/infocenter/idshelp/v10/topic/com.ibm.docnotes.doc/
uc5/ids_adref_docnotes_10.0.html#ToC_16
Regards.
On Mon, Jun 3, 2013 at 5:07 PM, Fernando Nunes <domusonline@gmail.com>wrote:
> Hello Andrew,
> Nice to "see" you.
>
> Here's some info:
>
>
>
>
http://publib.boulder.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm
.adref.doc%2Fids_adr_0597.htm
>
> Does it apply to your situation?
> Regards
>
> On Mon, Jun 3, 2013 at 4:37 PM, Andrew Grantham <agrantha@hotmail.com
> >wrote:
>
> > All,
> >
> > Sun Solaris 10
> > Informix V10.00.FC10
> >
> > All,
> >
> > I have an instance with a few locks which do not appear to have an owner.
> > In
> > the list below you have locks on a table with the number 80005a. As you
> can
> > see the owner is in brackets. When I look for that owner in the onstat -u
> > output nothing is found.
> > How can I identify the owner and therefore kill the locks other than by
> > bouncing the instance?
> >
> > Thanks
> >
> > Andy G
> >
> > 233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
> > 233cdfc8 0 1b481278 0 S 100002 205 0
> > 2378ad28 0 1b480b10 0 S 100002 205 0
> > 2378b1d8 0 1b494988 0 S 100002 205 0
> > 2378b7f0 0 1f9a3fd0 0 S 100002 205 0
> > 2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
> > 2378c3a8 0 1f99c950 0 S 100002 205 0
> > 2378cfd8 0 1f9adb58 0 S 100002 205 0
> > 2378e388 0 1b483ee8 0 S 100002 205 0
> > 2378e4f0 0 1b482148 0 S 100002 205 0
> > 23792e10 0 1f9a6c40 0 S 100002 205 0
> > 237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
> > 237948c8 0 1f999ce0 0 S 100002 205 0
> >
> >
> >
> >
>
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --
> Fernando Nunes
> Portugal
>
> http://informix-technology.blogspot.com
> My email works... but I don't check it frequently...
>
> --20cf3071cbb4ceb3dd04de42291d
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Fernando Nunes
Portugal
http://informix-technology.blogspot.com
My email works... but I don't check it frequently...
--047d7b6d9f0c6a14cf04de426600
All,
After a bit more reading I checked for Global locks ( onstat -G ) and no
output was returned so please don´t suggest that idea!!!
Thanks
Andy G
> To: ids@iiug.org
> From: agrantha@hotmail.com
> Subject: identifying a lock seemingly without an owner [30405]
> Date: Mon, 3 Jun 2013 11:37:54 -0400
>
> All,
>
> Sun Solaris 10
> Informix V10.00.FC10
>
> All,
>
> I have an instance with a few locks which do not appear to have an owner. In
> the list below you have locks on a table with the number 80005a. As you can
> see the owner is in brackets. When I look for that owner in the onstat -u
> output nothing is found.
> How can I identify the owner and therefore kill the locks other than by
> bouncing the instance?
>
> Thanks
>
> Andy G
>
> 233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
> 233cdfc8 0 1b481278 0 S 100002 205 0
> 2378ad28 0 1b480b10 0 S 100002 205 0
> 2378b1d8 0 1b494988 0 S 100002 205 0
> 2378b7f0 0 1f9a3fd0 0 S 100002 205 0
> 2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
> 2378c3a8 0 1f99c950 0 S 100002 205 0
> 2378cfd8 0 1f9adb58 0 S 100002 205 0
> 2378e388 0 1b483ee8 0 S 100002 205 0
> 2378e4f0 0 1b482148 0 S 100002 205 0
> 23792e10 0 1f9a6c40 0 S 100002 205 0
> 237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
> 237948c8 0 1f999ce0 0 S 100002 205 0
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
Fernando,
Yes it has been a long time. Hope all is well with you.
I tried onstat -G and the list was empty.
I just stopped and restarted the instance and 2 locks immediately appeared (
same table ). The application was down at the time and they were no other
connections except for Informix ones. As far as I can tell it is not informix
launched application connections either.
Any ideas?
Regards
Andy G
> To: ids@iiug.org
> From: domusonline@gmail.com
> Subject: Re: identifying a lock seemingly without an owner [30407]
> Date: Mon, 3 Jun 2013 12:24:59 -0400
>
> It was a new feature in 10.00.xC5. Added to the documentation notes:
>
>
>
http://publib.boulder.ibm.com/infocenter/idshelp/v10/topic/com.ibm.docnotes.doc/
uc5/ids_adref_docnotes_10.0.html#ToC_16
>
> Regards.
>
> On Mon, Jun 3, 2013 at 5:07 PM, Fernando Nunes <domusonline@gmail.com>wrote:
>
> > Hello Andrew,
> > Nice to "see" you.
> >
> > Here's some info:
> >
> >
> >
> >
>
http://publib.boulder.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm
.adref.doc%2Fids_adr_0597.htm
> >
> > Does it apply to your situation?
> > Regards
> >
> > On Mon, Jun 3, 2013 at 4:37 PM, Andrew Grantham <agrantha@hotmail.com
> > >wrote:
> >
> > > All,
> > >
> > > Sun Solaris 10
> > > Informix V10.00.FC10
> > >
> > > All,
> > >
> > > I have an instance with a few locks which do not appear to have an owner.
> > > In
> > > the list below you have locks on a table with the number 80005a. As you
> > can
> > > see the owner is in brackets. When I look for that owner in the onstat -u
> > > output nothing is found.
> > > How can I identify the owner and therefore kill the locks other than by
> > > bouncing the instance?
> > >
> > > Thanks
> > >
> > > Andy G
> > >
> > > 233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
> > > 233cdfc8 0 1b481278 0 S 100002 205 0
> > > 2378ad28 0 1b480b10 0 S 100002 205 0
> > > 2378b1d8 0 1b494988 0 S 100002 205 0
> > > 2378b7f0 0 1f9a3fd0 0 S 100002 205 0
> > > 2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
> > > 2378c3a8 0 1f99c950 0 S 100002 205 0
> > > 2378cfd8 0 1f9adb58 0 S 100002 205 0
> > > 2378e388 0 1b483ee8 0 S 100002 205 0
> > > 2378e4f0 0 1b482148 0 S 100002 205 0
> > > 23792e10 0 1f9a6c40 0 S 100002 205 0
> > > 237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
> > > 237948c8 0 1f999ce0 0 S 100002 205 0
> > >
> > >
> > >
> > >
> >
> >
>
*******************************************************************************
> > > Forum Note: Use "Reply" to post a response in the discussion forum.
> > >
> > >
> >
> > --
> > Fernando Nunes
> > Portugal
> >
> > http://informix-technology.blogspot.com
> > My email works... but I don't check it frequently...
> >
> > --20cf3071cbb4ceb3dd04de42291d
> >
> >
> >
> >
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --
> Fernando Nunes
> Portugal
>
> http://informix-technology.blogspot.com
> My email works... but I don't check it frequently...
>
> --047d7b6d9f0c6a14cf04de426600
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
It may be better to open a PMR. It seems like a XA transaction, but if
onstat -G is empty I think something is wrong...Just in case, does the value showed in onstat -k appear in onstat -x?
Regards
On Mon, Jun 3, 2013 at 6:22 PM, Andrew Grantham <agrantha@hotmail.com>wrote:
> Fernando,
>
> Yes it has been a long time. Hope all is well with you.
>
> I tried onstat -G and the list was empty.
>
> I just stopped and restarted the instance and 2 locks immediately appeared
> (
> same table ). The application was down at the time and they were no other
> connections except for Informix ones. As far as I can tell it is not
> informix
> launched application connections either.
>
> Any ideas?
>
> Regards
>
> Andy G
>
> > To: ids@iiug.org
> > From: domusonline@gmail.com
> > Subject: Re: identifying a lock seemingly without an owner [30407]
> > Date: Mon, 3 Jun 2013 12:24:59 -0400
> >
> > It was a new feature in 10.00.xC5. Added to the documentation notes:
> >
> >
> >
>
>
http://publib.boulder.ibm.com/infocenter/idshelp/v10/topic/com.ibm.docnotes.doc/
uc5/ids_adref_docnotes_10.0.html#ToC_16
> >
> > Regards.
> >
> > On Mon, Jun 3, 2013 at 5:07 PM, Fernando Nunes <domusonline@gmail.com
> >wrote:
> >
> > > Hello Andrew,
> > > Nice to "see" you.
> > >
> > > Here's some info:
> > >
> > >
> > >
> > >
> >
>
>
http://publib.boulder.ibm.com/infocenter/idshelp/v117/index.jsp?topic=%2Fcom.ibm
.adref.doc%2Fids_adr_0597.htm
> > >
> > > Does it apply to your situation?
> > > Regards
> > >
> > > On Mon, Jun 3, 2013 at 4:37 PM, Andrew Grantham <agrantha@hotmail.com
> > > >wrote:
> > >
> > > > All,
> > > >
> > > > Sun Solaris 10
> > > > Informix V10.00.FC10
> > > >
> > > > All,
> > > >
> > > > I have an instance with a few locks which do not appear to have an
> owner.
> > > > In
> > > > the list below you have locks on a table with the number 80005a. As
> you
> > > can
> > > > see the owner is in brackets. When I look for that owner in the
> onstat
> -u> > > > output nothing is found.
> > > > How can I identify the owner and therefore kill the locks other than
> by
> > > > bouncing the instance?
> > > >
> > > > Thanks
> > > >
> > > > Andy G
> > > >
> > > > 233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
> > > > 233cdfc8 0 1b481278 0 S 100002 205 0
> > > > 2378ad28 0 1b480b10 0 S 100002 205 0
> > > > 2378b1d8 0 1b494988 0 S 100002 205 0
> > > > 2378b7f0 0 1f9a3fd0 0 S 100002 205 0
> > > > 2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
> > > > 2378c3a8 0 1f99c950 0 S 100002 205 0
> > > > 2378cfd8 0 1f9adb58 0 S 100002 205 0
> > > > 2378e388 0 1b483ee8 0 S 100002 205 0
> > > > 2378e4f0 0 1b482148 0 S 100002 205 0
> > > > 23792e10 0 1f9a6c40 0 S 100002 205 0
> > > > 237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
> > > > 237948c8 0 1f999ce0 0 S 100002 205 0
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
>
>
*******************************************************************************
> > > > Forum Note: Use "Reply" to post a response in the discussion forum.
> > > >
> > > >
> > >
> > > --
> > > Fernando Nunes
> > > Portugal
> > >
> > > http://informix-technology.blogspot.com
> > > My email works... but I don't check it frequently...
> > >
> > > --20cf3071cbb4ceb3dd04de42291d
> > >
> > >
> > >
> > >
> >
>
>
*******************************************************************************
> > > Forum Note: Use "Reply" to post a response in the discussion forum.
> > >
> > >
> >
> > --
> > Fernando Nunes
> > Portugal
> >
> > http://informix-technology.blogspot.com
> > My email works... but I don't check it frequently...
> >
> > --047d7b6d9f0c6a14cf04de426600
> >
> >
> >
>
>
*******************************************************************************
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Fernando Nunes
Portugal
http://informix-technology.blogspot.com
My email works... but I don't check it frequently...
--20cf307abe8728d4f604de4378a8
Hi Andrew,
you seem to suffer from a orphaned XA transaction, which cannot be closed,
because the initiating process has died.
Informix 10.0 was very vulnerable for this kind of problem, when you have
and unclean shutdown of your application server (such as kill -9).
We even had crashes from this situation.
There is only two ways to get rid of these:
You can cycle through the logfiles by executing onmode -l multiple times,
until the engine detects a long transaction (about to overwrite the log where
the transaction started). The engine will block at this moment.
Then, you can rollback the transaction with onmode -H (undocumented).
see this thread for details:
http://www.iiug.org/forums/ids/index.cgi/read/7425
Or you can use a simple Java program, (we use it for 11.70, but should work
with 10.0).
That one attaches to the XA transaction and executes a rollback.
In case you are interested, I can send you the sources.
So, when using the XA transactions, make sure you shut down the application
server
in a very clean way to prevent such orphaned transactions. We do this by
stopping
the HA-JNDI (JBoss, using JMX-console), check there are no more active
requests (via Logfile)
and then stop the Appserver using standard shutdown script.
Hope this helps.
Marcus Haarmann
----- Ursprüngliche Mail -----
Von: "Andrew Grantham" <agrantha@hotmail.com>
An: ids@iiug.org
Gesendet: Montag, 3. Juni 2013 18:27:50
Betreff: RE: identifying a lock seemingly without an owner [30408]
All,
After a bit more reading I checked for Global locks ( onstat -G ) and no
output was returned so please don´t suggest that idea!!!
Thanks
Andy G
> To: ids@iiug.org
> From: agrantha@hotmail.com
> Subject: identifying a lock seemingly without an owner [30405]
> Date: Mon, 3 Jun 2013 11:37:54 -0400
>
> All,
>
> Sun Solaris 10
> Informix V10.00.FC10
>
> All,
>
> I have an instance with a few locks which do not appear to have an owner. In
> the list below you have locks on a table with the number 80005a. As you can
> see the owner is in brackets. When I look for that owner in the onstat -u
> output nothing is found.
> How can I identify the owner and therefore kill the locks other than by
> bouncing the instance?
>
> Thanks
>
> Andy G
>
> 233c8208 0 (1b4b29b8) a2b3368 HDR+IX 80005a 0 0
> 233cdfc8 0 1b481278 0 S 100002 205 0
> 2378ad28 0 1b480b10 0 S 100002 205 0
> 2378b1d8 0 1b494988 0 S 100002 205 0
> 2378b7f0 0 1f9a3fd0 0 S 100002 205 0
> 2378bef8 0 (1b4b29b8) 237939c8 HDR+X 800063 3ae2f K- 1
> 2378c3a8 0 1f99c950 0 S 100002 205 0
> 2378cfd8 0 1f9adb58 0 S 100002 205 0
> 2378e388 0 1b483ee8 0 S 100002 205 0
> 2378e4f0 0 1b482148 0 S 100002 205 0
> 23792e10 0 1f9a6c40 0 S 100002 205 0
> 237939c8 0 (1b4b29b8) 233c8208 HDR+X 80005a 3ae2f 0
> 237948c8 0 1f999ce0 0 S 100002 205 0
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
If you want to see "SOME" of the undocumented onmode and onstat commands do
the following:
export RASHELP=YES
onstat --
onmode --
They will show some of the "hidden" commands.
Most of which will not be supported if used.
Original post:
Fernando,
Yes it has been a long time. Hope all is well with you.
I tried onstat -G and the list was empty.
I just stopped and restarted the instance and 2 locks immediately appeared (
same table ). The application was down at the time and they were no other
connections except for Informix ones. As far as I can tell it is not informix
launched application connections either.
Any ideas?
Regards
Andy G
Response:
Beside an open XA transaction, which should show in onstat -G. I believe a
left open distributed transaction (just a regular 2 phase commit transaction
involving 2 IDS servers) on both either the coordinator or subordinate site
can persist through server bounces. I think in this case you would see the
address of the lock owner be a transaction (from onstat -x). If you use onlog
-l to look at the checkpoint record of the checkpoint performed right after
the server was brought up, I'd guess you'd see 1 open transaction, which
matches to this transaction that's got the lock on the table you are seeing.
I'd check the doc on 2 phase commit transactions, and I think onmode -Z
<address> can be used to try and terminate it, but you'd want to check the doc
first as if you do that, you could end up with a transaction that was only
partially committed between 2 servers.
Regular 2 phase commit transactions occur when you do something like the
following:
connect to server A
issue begin work
do some work on server A (like insert into tab a)
do work on a remote tab (like insert into db_b@server_b:tab_b)
The above transaction is now a 2 phase commit transaction with server A being
the coordinator and server B being a subordinate.
Jacques Renaut
IBM Informix Advanced Support
APD Team