Post-Upgrade HPL issue
Mike Magie upgraded from 10.00.UC6 to 11.50.UC6 on Solaris 10, and some express-mode HPL onpladm loads hung with plsig_segv SIGSEGV in onploaderr files. He doubled SHMVIRTSIZE and suspected swap. Solved by his own finding: RESIDENT had been 0; setting it to 1 stopped the segvs, while 0 or -1 gave sporadic ones. He thought it a bug.
Auto-generated by Claude from the posts below — may be imperfect; read the full thread.
We just upgraded our three production instances from 10.00.UC6 to 11.50.UC6 on
Solaris 10.
The onpload database was converted without error or issue - when I query the
version table in the onpload db it correctly shows version 3.
I kick off a handful of loads, 6 or 7 at a time, and a couple of them will
appear to hang. These are express mode, conversion jobs started via an onpladm
command. There will be no message written to the log file in /tmp or anything.
While one of the jobs was hung I noticed that the onploaderr.7 file was
greater than 0 bytes. The file contained the following:
plsig_segv(): received SIGSEGV
That is the only sign of a problem anywhere. I killed the onpladm job
associated with the hung job. The script that I use to call onpladm will retry
4 times if there is a failure, so it tried a second time and succeeded. I then
went back and found an onploaderr.X file (where X = sessionid from the
onpload:session table) for every "hung" onpload job I had seen. Each
onploaderr.X file had the same plsig_segv message.
Segv's almost always indicate a bug - but before I go through the support
chain I wanted to see if any IBM'ers monitoring this or other folks have ran
into this problem. The issue is odd in that we never saw this problem occur on
our test/dev instances and they are configured identical to prod.
Thanks -
Mike
I want to add that there was a single config change made to the onconfig on
the affected instance - we doubled shmvirtsize, going from 1 gig to 2 gig. The
issue seems resource related for sure. I just started another set of jobs,
only running two at a time and they ran fine. When the hung jobs exist I can't
find anything weird in any of the onstat mem related cmds or anything else.
MM
Could be an issue with poorly configured swap on the server - stay tuned.
We figured out the problem here - weird also. We had the RESIDENT setting in our onconfig to 0 for some reason - it was 1 forever on v10 - anyways we set RESIDENT to 1 and no more pload segv's. I tested it with RESIDENT set to -1 and 0 and got segv's sporadically - just an FYI. Almost certainly a bug... Probably should report it.