Re: Problem with ER : cdr sync replicate
Philippe reported that 'cdr sync replicate' (Enterprise Replication, IDS 10.00.UC4) always dies with a segmentation fault. Madison Pruet (IBM) asked him to run it under gdb and post the stack, which showed the crash in sqli_connect_set called from part_tdDtor/syncVar_tdDtor/checkRepl — i.e. during cleanup, where cdr reconnects to the server tied to each 'part'. Pruet suspected a related error 47 (failed lookup of group name in the hostdef table) plus faulty cleanup when the initial connection never happened, possibly fixed in a later release. Philippe showed the hostdef queries returned rows correctly for both g_root and g_leaf, and the error 47 wasn't reproducible while the segv and stack were. The thread ends there with no resolution; a case had been opened via an IBM reseller.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
> First of all, whenever the cdr utility detects an error, it kills itself with > a segv so we can get a core file. Please run the command under a debugger so > that we can get a stack trace. That will help to see where the error is > occurring. We will probably need to have the schema definition of the table > being synced. Here is stdout of the debugger session: (gdb) r sync replicate --master=g_root --repl=My_repl --extratargetrows=delete g_leaf Starting program: /usr/ids100uc4/bin/cdr sync replicate --master=g_root --repl=My_Repl --extratargetrows=delete g_leaf (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) Program received signal SIGSEGV, Segmentation fault. 0x080df255 in sqli_connect_set () (gdb) And here is the schema of the table that participate to the replicate: Column name Type Nulls b5num serial no b5nom char(30) yes b5flaact char(1) yes b5flafac char(1) yes b5o1numadd integer yes b5o1nummod integer yes b5datadd date yes b5datmod date yes b5typ char(1) yes b5datdeb date yes b5datfin date yes b5o3num integer yes b5c0num integer yes The primary key is the serial column b5num. I have opened a case through our IBM Informix products reseller. I don't know the IBM case number. Philippe
philippe wrote: >> First of all, whenever the cdr utility detects an error, it kills >> itself with a segv so we can get a core file. Please run the command >> under a debugger so that we can get a stack trace. That will help to >> see where the error is occurring. We will probably need to have the >> schema definition of the table being synced. > > Here is stdout of the debugger session: > > (gdb) r sync replicate --master=g_root --repl=My_repl > --extratargetrows=delete g_leaf > Starting program: /usr/ids100uc4/bin/cdr sync replicate --master=g_root > --repl=My_Repl --extratargetrows=delete g_leaf > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > (no debugging symbols found) > > Program received signal SIGSEGV, Segmentation fault. > 0x080df255 in sqli_connect_set () > (gdb) At the time that you get the SEGV, we need you to get the stack. Please retry and enter "where" when you get the segv. Thanks. > > > And here is the schema of the table that participate to the replicate: > > Column name Type Nulls > b5num > serial no > b5nom char(30) yes > b5flaact char(1) yes > b5flafac char(1) yes > b5o1numadd integer yes > b5o1nummod integer yes > b5datadd date yes > b5datmod date yes > b5typ char(1) yes > b5datdeb date yes > b5datfin date yes > b5o3num integer yes > b5c0num integer yes > > The primary key is the serial column b5num. > > > I have opened a case through our IBM Informix products reseller. I don't > know the IBM case number. > > Philippe > >
> At the time that you get the SEGV, we need you to get the stack. Please > retry and enter "where" when you get the segv. ... (no debugging symbols found) Error returned 47 at 5417 Error returned 47 at 5417 Program received signal SIGSEGV, Segmentation fault. 0x080df255 in sqli_connect_set () (gdb) where #0 0x080df255 in sqli_connect_set () #1 0x08090c0d in part_tdDtor () #2 0x08090b5e in syncVar_tdDtor () #3 0x08092382 in checkRepl () #4 0x0809112f in cdr_check () #5 0x0804ee22 in sync_repl () #6 0x0804bf06 in main () (gdb) Thank you for your help. Philippe
philippe wrote: >> At the time that you get the SEGV, we need you to get the stack. >> Please retry and enter "where" when you get the segv. > > ... > (no debugging symbols found) > Error returned 47 at 5417 > Error returned 47 at 5417 > > Program received signal SIGSEGV, Segmentation fault. > 0x080df255 in sqli_connect_set () > (gdb) where > #0 0x080df255 in sqli_connect_set () > #1 0x08090c0d in part_tdDtor () > #2 0x08090b5e in syncVar_tdDtor () > #3 0x08092382 in checkRepl () > #4 0x0809112f in cdr_check () > #5 0x0804ee22 in sync_repl () > #6 0x0804bf06 in main () > (gdb) Well getting a bit internal..... ;-) This is where the 47 error is being returned. $select groupname into :groupName from hostdef where name = :nameIn; if (sqlca.sqlcode == 100) { rc = CDR_ESERV; break; } I suspect that the 47 error is leading eventually to the other failures because logic is that when we are trying to destroy the part structure (part_tdDtor), that we attempt to connect to the server that the part is associated with and then perform a clean shutdown. I suspect that in UC4 that there is some issue with improper cleanup when the initial connection never occured. Not sure. This may have been fixed in subsequent releases - again not sure. Probably the best thing to do is to examine the sqlhost file and/or examine the hostdef table that you are connecting to. What it is trying to do is to convert the group name into a server name - and that is failing. You might want to try to issue the select statement for each node that is involved in the sync operation. (nameIn should be the group name...) M.P. > > Thank you for your help. > Philippe > >
> Well getting a bit internal..... ;-)
>
> This is where the 47 error is being returned.
> $select groupname into :groupName
> from hostdef
> where name = :nameIn;
> if (sqlca.sqlcode == 100)
> {
> rc = CDR_ESERV;
> break;
> }
>
> I suspect that the 47 error is leading eventually to the other failures
> because logic is that when we are trying to destroy the part structure
> (part_tdDtor), that we attempt to connect to the server that the part is
> associated with and then perform a clean shutdown. I suspect that in UC4
> that there is some issue with improper cleanup when the initial connection
> never occured. Not sure. This may have been fixed in subsequent releases -
> again not sure.
In fact, the error 'Error returned 47 at 5417' is NOT reproducible,
while segmentation fault occurs each times.
> Probably the best thing to do is to examine the sqlhost file and/or examine
> the hostdef table that you are connecting to. What it is trying to do is to
> convert the group name into a server name - and that is failing. You might
> want to try to issue the select statement for each node that is involved in
> the sync operation. (nameIn should be the group name...)
The select statements return always 1 row:
on g_root server:
select groupname from hostdef where name = 'g_root' -> return'g_root'
select groupname from hostdef where name = 'g_leaf' -> return'g_leaf'
same result on g_leaf server.
Philippe
philippe wrote:
>> Well getting a bit internal..... ;-)
>>
>> This is where the 47 error is being returned.
>> $select groupname into :groupName
>> from hostdef
>> where name = :nameIn;
>> if (sqlca.sqlcode == 100)
>> {
>> rc = CDR_ESERV;
>> break;
>> }
>>
>> I suspect that the 47 error is leading eventually to the other
>> failures because logic is that when we are trying to destroy the part
>> structure (part_tdDtor), that we attempt to connect to the server that
>> the part is associated with and then perform a clean shutdown. I
>> suspect that in UC4 that there is some issue with improper cleanup
>> when the initial connection never occured. Not sure. This may have
>> been fixed in subsequent releases - again not sure.
>
> In fact, the error 'Error returned 47 at 5417' is NOT reproducible,
> while segmentation fault occurs each times.
With the same stack???
>
>
>> Probably the best thing to do is to examine the sqlhost file and/or
>> examine the hostdef table that you are connecting to. What it is
>> trying to do is to convert the group name into a server name - and
>> that is failing. You might want to try to issue the select statement
>> for each node that is involved in the sync operation. (nameIn should
>> be the group name...)
>
> The select statements return always 1 row:
>
> on g_root server:
> select groupname from hostdef where name = 'g_root' -> return 'g_root'
> select groupname from hostdef where name = 'g_leaf' -> return 'g_leaf'>
> same result on g_leaf server.
>
> Philippe
>
>
>> In fact, the error 'Error returned 47 at 5417' is NOT reproducible, while >> segmentation fault occurs each times. > > With the same stack??? > yes : ... (no debugging symbols found) Program received signal SIGSEGV, Segmentation fault. 0x080df255 in sqli_connect_set () (gdb) where #0 0x080df255 in sqli_connect_set () #1 0x08090c0d in part_tdDtor () #2 0x08090b5e in syncVar_tdDtor () #3 0x08092382 in checkRepl () #4 0x0809112f in cdr_check () #5 0x0804ee22 in sync_repl () #6 0x0804bf06 in main () (gdb)