Segfault since CSDK 4.50.FC10 with LVARCHAR and CLIENT_LOCALE different from DB_LOCALE

Hello! Starting with CSDK 4.50.FC10 (we cannot reproduce with 4.50.FC9W1), With DB_LOCALE=en_us.8859-15 and CLIENT_LOCALE=en_us.utf8, LC_ALL=en_US.utf8, Fetching an LVARCHAR with auto-allocation API using ifx_var_flag(ptr,1), we get a segfault when fetching. THE PROBLEM DOES NOT OCCUR, when the client and db locales match! I will try to provide a pure ESQL/C program to reproduce. In the meantime, here is the gcc address sanitizer error we see: ================================================================= ==165896==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62100003327c at pc 0x7f1ee8847c21 bp 0x7fffce40d760 sp 0x7fffce40cf10 WRITE of size 13429 at 0x62100003327c thread T0 #0 0x7f1ee8847c20 in __interceptor_memmove ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:810 #1 0x7f1ed881ef54 in _iqupdtargs (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x1ef54) #2 0x7f1ed8820510 in _iqftch (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x20510) #3 0x7f1ed882084f in sqli_curs_fetch (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x2084f) #4 0x7f1ee78e1c43 in ODI_fetchCursor /home/sf/genero/devel/fgl/fgl/src/sqldriver/informix/ifx91.ec:384 #5 0x7f1ee8145c6f in fglsql_cuFetch ../base/fglsql.c:892 #6 0x7f1ee8138296 in rts_cursorFetch ../base/Sql.c:408 #7 0x7f1ee80f9818 in callFu0 /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:331 #8 0x7f1ee80f99dd in invoke /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:362 #9 0x7f1ee81003a1 in vm_interprete_impl /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:1102 #10 0x7f1ee80d0993 in fdb_debug /home/sf/genero/devel/fgl/fgl/src/vm/fdb.c:3572 #11 0x7f1ee80f56b6 in vm_main /home/sf/genero/devel/fgl/fgl/src/vm/vm.c:945 #12 0x55dadc9cc1bf in main /home/sf/genero/devel/fgl/fgl/src/vm/fglrun.c:5 #13 0x7f1ee7e46249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #14 0x7f1ee7e46304 in __libc_start_main_impl ../csu/libc-start.c:360 #15 0x55dadc9cc0c0 in _start (/home/sf/genero/devel/fgl/fgl/opt/fgl/bin/fglrun+0x10c0) 0x62100003327c is located 0 bytes to the right of 4476-byte region [0x621000032100,0x62100003327c) allocated by thread T0 here: #0 0x7f1ee88b89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x7f1ed88328e3 in _sqrealloc (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x328e3) #2 0x7f1ed881ecbb in _iqupdtargs (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x1ecbb) #3 0x7f1ed8820510 in _iqftch (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x20510) #4 0x7f1ed882084f in sqli_curs_fetch (/opt3/dbs/ifx/CSDK-4.50.FC10/lib/esql/libifsql.so+0x2084f) #5 0x7f1ee78e1c43 in ODI_fetchCursor /home/sf/genero/devel/fgl/fgl/src/sqldriver/informix/ifx91.ec:384 #6 0x7f1ee8145c6f in fglsql_cuFetch ../base/fglsql.c:892 #7 0x7f1ee8138296 in rts_cursorFetch ../base/Sql.c:408 #8 0x7f1ee80f9818 in callFu0 /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:331 #9 0x7f1ee80f99dd in invoke /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:362 #10 0x7f1ee81003a1 in vm_interprete_impl /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:1102 #11 0x7f1ee80d0993 in fdb_debug /home/sf/genero/devel/fgl/fgl/src/vm/fdb.c:3572 #12 0x7f1ee80f56b6 in vm_main /home/sf/genero/devel/fgl/fgl/src/vm/vm.c:945 #13 0x55dadc9cc1bf in main /home/sf/genero/devel/fgl/fgl/src/vm/fglrun.c:5 #14 0x7f1ee7e46249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:810 in __interceptor_memmove Shadow bytes around the buggy address: 0x0c427fffe5f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c427fffe600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c427fffe610: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c427fffe620: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c427fffe630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x0c427fffe640: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00[04] 0x0c427fffe650: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c427fffe660: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c427fffe670: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c427fffe680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c427fffe690: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==165896==ABORTING Seb ------------------------------ Sebastien FLAESCH ------------------------------

With CSDK 4.50.FC11W1 we get this: ================================================================= ==166749==ERROR: AddressSanitizer: unknown-crash on address 0x002b41414141 at pc 0x7fc0d2c47681 bp 0x7ffc1e3119a0 sp 0x7ffc1e311150 WRITE of size 13426 at 0x002b41414141 thread T0 #0 0x7fc0d2c47680 in __interceptor_memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799 #1 0x7fc0c2c3ecc1 in ifx_var_alloc (/opt3/dbs/ifx/CSDK-4.50.FC11W1/lib/esql/libifsql.so+0x3ecc1) #2 0x7fc0c2c1e671 in _iqupdtargs (/opt3/dbs/ifx/CSDK-4.50.FC11W1/lib/esql/libifsql.so+0x1e671) #3 0x7fc0c2c1fd3f in _iqftch (/opt3/dbs/ifx/CSDK-4.50.FC11W1/lib/esql/libifsql.so+0x1fd3f) #4 0x7fc0c2c2007e in sqli_curs_fetch (/opt3/dbs/ifx/CSDK-4.50.FC11W1/lib/esql/libifsql.so+0x2007e) #5 0x7fc0d279ec43 in ODI_fetchCursor /home/sf/genero/devel/fgl/fgl/src/sqldriver/informix/ifx91.ec:384 #6 0x7fc0d2545c6f in fglsql_cuFetch ../base/fglsql.c:892 #7 0x7fc0d2538296 in rts_cursorFetch ../base/Sql.c:408 #8 0x7fc0d24f9818 in callFu0 /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:331 #9 0x7fc0d24f99dd in invoke /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:362 #10 0x7fc0d25003a1 in vm_interprete_impl /home/sf/genero/devel/fgl/fgl/src/vm/vm_interp.c:1102 #11 0x7fc0d24d0993 in fdb_debug /home/sf/genero/devel/fgl/fgl/src/vm/fdb.c:3572 #12 0x7fc0d24f56b6 in vm_main /home/sf/genero/devel/fgl/fgl/src/vm/vm.c:945 #13 0x5598a367b1bf in main /home/sf/genero/devel/fgl/fgl/src/vm/fglrun.c:5 #14 0x7fc0d2246249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #15 0x7fc0d2246304 in __libc_start_main_impl ../csu/libc-start.c:360 #16 0x5598a367b0c0 in _start (/home/sf/genero/devel/fgl/fgl/opt/fgl/bin/fglrun+0x10c0) Address 0x002b41414141 is located in the shadow gap area. SUMMARY: AddressSanitizer: unknown-crash ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799 in __interceptor_memset ==166749==ABORTING ------------------------------ Sebastien FLAESCH ------------------------------

Hi Sebastien, I can reproduce the same segv with a simple esql/c test case 8( Just logged a new defect ( idsdb00116388 ) against 4.50.FC11W1. I guess there was something "not quite right" with the fix for # idsdb00112743 that may be causing this memory violation. Javier ------------------------------ Javier Sagrera ------------------------------

Hi Javier, Here is my ESQL/C sample to reproduce Please can you try it as well and compare with yours? I believe the use of ifx_var_*() function is correct, please double check as well... We also wonder about ifx_var_init(&lvarchar) function usage, it's not documented and should be clarified if mandatory or if ifx_var_flag() is sufficiant. Seb /* * Fetching LVARCHAR with ifx_var_*() API fails when database and client encoding differ * * The problem appeared with CSDK 4.50.FC10 and is still present in 4.50.FC11W1 * * The test database must use ISO-8859-15 locale and client must use UTF-8 * export DB_LOCALE=en_us.8859-15 * export CLIENT_LOCALE=en_us.utf8 * * There is no issue when the database and client use the same locale. * * We suspect some buffer allocation issue to convert ISO-8859-15 data to UTF-8. * * Best way is to use the gcc address sanitizer to see where is happens: * * export INFORMIXC="gcc -fsanitize=address" * esql -o test010.bin test010.ec * */ #include <stdlib.h> #include <stdio.h> #include <signal.h> #include <string.h> #include <locale.h> EXEC SQL include sqlca; EXEC SQL include sqlda; EXEC SQL include sqltypes; void check_sqlcode(const char *msg) { bigint bs; printf("[%s]: sqlcode=%d\\n", msg, sqlca.sqlcode); if (sqlca.sqlcode != 0 && sqlca.sqlcode != 100) { exit(1); } } int main(int argc, char ** argv) { struct sqlda da; ifx_sqlvar_t var; void *lvarchar; EXEC SQL BEGIN DECLARE SECTION; struct sqlda *intoSqlda; EXEC SQL END DECLARE SECTION; EXEC SQL DATABASE test2; check_sqlcode("DATABASE test2"); EXEC SQL DROP TABLE tab1; EXEC SQL CREATE TABLE tab1 ( pkey INTEGER, data LVARCHAR(5000) ); check_sqlcode("CREATE TABLE tab1"); EXEC SQL INSERT INTO tab1 VALUES (101,'aaaa'); check_sqlcode("INSERT INTO tab1 #1"); memset(&da, 0, sizeof(struct sqlda)); da.sqld = 1; da.sqlvar = &var; lvarchar = NULL; // ifx_var_init(&lvarchar); // ??? NOT DOCUMENTED ??? ifx_var_flag(&lvarchar, 1); printf("lvarchar pointer: %p\\n", lvarchar); memset(&var, 0, sizeof(ifx_sqlvar_t)); var.sqldata = lvarchar; var.sqltype = CLVCHARPTRTYPE; var.sqltypename = "lvarchar"; var.sqltypelen = 8; EXEC SQL DECLARE c1 CURSOR FOR SELECT data FROM tab1 ORDER BY pkey; check_sqlcode("DECLARE c1 CURSOR"); EXEC SQL OPEN c1; check_sqlcode("OPEN c1"); intoSqlda = &da; EXEC SQL FETCH c1 USING DESCRIPTOR intoSqlda; check_sqlcode("FETCH c1"); printf("FETCH data: [%s]\\n", (char*) ifx_var_getdata(&lvarchar)); ifx_var_freevar(&lvarchar); ifx_var_dealloc(&lvarchar); return 0; } ------------------------------ Sebastien FLAESCH ------------------------------

It fails in the same way: Starting program: /home/informix/4.50.FC11W1/tmp/a.out [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [DATABASE test2]: sqlcode=0 [CREATE TABLE tab1]: sqlcode=0 [INSERT INTO tab1 #1]: sqlcode=0 lvarchar pointer: 0x5555556acf40 [DECLARE c1 CURSOR]: sqlcode=0 [OPEN c1]: sqlcode=0 Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7d946e9 in _iqupdtargs () from /home/informix/4.50.FC11/lib/esql/libifsql.so 3: x/i $pc => 0x7ffff7d946e9 <_iqupdtargs+5523>: mov rax,QWORD PTR [rax+0x10] (gdb) where #0 0x00007ffff7d946e9 in _iqupdtargs () from /home/informix/4.50.FC11/lib/esql/libifsql.so #1 0x00007ffff7d95d40 in _iqftch () from /home/informix/4.50.FC11/lib/esql/libifsql.so #2 0x00007ffff7d9607f in sqli_curs_fetch () from /home/informix/4.50.FC11/lib/esql/libifsql.so #3 0x00005555555554f8 in main () (gdb) regarding 'ifx_var_init()', No harm on adding it, but the esql/c preprocesor should add that to the .C code if it knows its dealing with a LVARCHAR varaible (e.g. 'lvarchar *lv1;' ) ------------------------------ Javier Sagrera ------------------------------

I will open another thread for ifx_var_init() usage. Seb ------------------------------ Sebastien FLAESCH ------------------------------

The only thing that ifx_var_init() does is to allocate and zero space for the ifx_varlena_t structure (if its not been allocated before). It does that calling sqli_private_var_init(), If you don't call it, it will be called from ifx_var_flag() (gdb) where #0 0x00007ffff7db4a92 in sqli_private_var_init () from /home/informix/4.50.FC11W1/lib/esql/libifsql.so #1 0x00007ffff7db4b3c in ifx_var_flag () from /home/informix/4.50.FC11W1/lib/esql/libifsql.so #2 0x00005555555553aa in main () (gdb) ------------------------------ Javier Sagrera ------------------------------

Just tested Informix 15.0.0.0 + CSDK 15.0.0.0: Same issue $ ./test010.bin [DATABASE test2]: sqlcode=0 [CREATE TABLE tab1]: sqlcode=0 [INSERT INTO tab1 #1]: sqlcode=0lvarchar pointer: 0x6030000091f0 [DECLARE c1 CURSOR]: sqlcode=0 [OPEN c1]: sqlcode=0 AddressSanitizer:DEADLYSIGNAL ================================================================= ==222113==ERROR: AddressSanitizer: SEGV on unknown address 0x0005dfff8000 (pc 0x7fcb48a3bd5a bp 0x7ffc2d68dde0 sp 0x7ffc2d68d578 T0) ==222113==The signal is caused by a READ memory access. #0 0x7fcb48a3bd5a in AddressIsPoisoned ../../../../src/libsanitizer/asan/asan_mapping.h:379 #1 0x7fcb48a3bd5a in QuickCheckForUnpoisonedRegion ../../../../src/libsanitizer/asan/asan_interceptors_memintrinsics.h:31 #2 0x7fcb48a47621 in __interceptor_memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:799 #3 0x7fcb491a1fcf in ifx_var_alloc (/opt3/dbs/ifx/CSDK-15.0.0.0/lib/esql/libifsql15a.so+0x40fcf) #4 0x7fcb49180b48 in _iqupdtargs (/opt3/dbs/ifx/CSDK-15.0.0.0/lib/esql/libifsql15a.so+0x1fb48) #5 0x7fcb4918224c in _iqftch (/opt3/dbs/ifx/CSDK-15.0.0.0/lib/esql/libifsql15a.so+0x2124c) #6 0x7fcb49182524 in sqli_curs_fetch (/opt3/dbs/ifx/CSDK-15.0.0.0/lib/esql/libifsql15a.so+0x21524) #7 0x55b38108c83f in main (/home/sf/dbvendors/informix/problems/test010.bin+0x283f) #8 0x7fcb486b0249 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 #9 0x7fcb486b0304 in __libc_start_main_impl ../csu/libc-start.c:360 #10 0x55b38108c1d0 in _start (/home/sf/dbvendors/informix/problems/test010.bin+0x21d0) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: SEGV ../../../../src/libsanitizer/asan/asan_mapping.h:379 in AddressIsPoisoned ==222113==ABORTING ------------------------------ Sebastien FLAESCH ------------------------------