Endian issue, BYTE columns and INFORMIX

Hi, We are building INFORMIX client applications on Linux using SDK 2.40 to communicate with INFORMIX 7.31 server on HPUX 10.20. The applications fetch and store "ordinary" data types just fine, e.g. integers, character strings, floats. However when it comes to fetching BYTE or TEXT columns, we run into a bit of trouble with malloc complaining about the number of bytes to allocate for the loc_t buffer is invalid (I am recalling this from memory and the details are a bit hazy, but I think this was the gist of the error). Our API sets the loc_t structure so that the server allocates the buffer and returns a pointer to the object and the malloc error is occurring inside a INFORMIX library routine, not ours. Is this related to the fact that HP-UX byte ordering is big-endian while Linux's is little-endian? Given that we're able to fetch and store ordinary data types just fine, it appears that server-client software recognizes the difference in byte ordering between the two or uses some kind of XDR format but doesn't extend to BYTE or TEXT columns? Or am I on the wrong path here? Do we have to supply user defined open(), read(), write(), close() functions for loc_t in our Linux APIs to do the bit swapping? I can supply more details on the exact error message we get tomorrow (Monday) morning, if needed. Mark -- "Honesty is the best policy, but insanity is a better defense." -- Anonymous

Mark Oberfield wrote: > We are building INFORMIX client applications on Linux using SDK 2.40 > to communicate with INFORMIX 7.31 server on HPUX 10.20. The > applications fetch and store "ordinary" data types just fine, > e.g. integers, character strings, floats. However when it comes to > fetching BYTE or TEXT columns, we run into a bit of trouble with > malloc complaining about the number of bytes to allocate for the loc_t > buffer is invalid (I am recalling this from memory and the details are > a bit hazy, but I think this was the gist of the error). > > Our API sets the loc_t structure so that the server allocates the > buffer and returns a pointer to the object and the malloc error is > occurring inside a INFORMIX library routine, not ours. How many of the loc_t fields do you set (and which ones)? When I do it, I use: blob->loc_loctype = LOCMEMORY; blob->loc_size = 0; blob->loc_bufsize = -1; blob->loc_buffer = (char *)0; blob->loc_indicator = 0; blob->loc_oflags = 0; (see SQLCMD from the IIUG software archives, and file ixblob.ec). > Is this related to the fact that HP-UX byte ordering is big-endian > while Linux's is little-endian? No; I don't think that can possibly be the problem. Certainly, I've never heard of an endian-ness problem in anything resembling this context. Within the data inside the blob, then yes, you could run into problems, but not in simply fetching it. > Given that we're able to fetch and > store ordinary data types just fine, it appears that server-client > software recognizes the difference in byte ordering between the two or > uses some kind of XDR format but doesn't extend to BYTE or TEXT > columns? Neither the server nor ESQL/C imposes any interpretation on the actual data in the BYTE blob (nor, in practice, in the TEXT blob). If you store data in a binary blob that should be treated with endian issues taken into account, that is (officially) your problem. However, as I said, the retrieval phase should not ever run into this as a problem. The data is collected over the wire in the same way, and the Linux box interprets how to stuff that into the loc_t using just local information. All sizes relayed over the wire are in a defined order, so the specification is neutral between big-endian and little-endian (though one set of machines have to formally remap the bytes into the requisite order for their hardware, but this applies to everything, and is incredibly unlikely to be the problem - speaking from experience). > Or am I on the wrong path here? Do we have to supply user defined > open(), read(), write(), close() functions for loc_t in our Linux APIs > to do the bit swapping? Not at the loc_t level, unless you want to do so. > I can supply more details on the exact error message we get tomorrow > (Monday) morning, if needed. Compare my loc_t settings with yours; if there's still trouble, then send the detailed error message, and probably the code too. -- Yours, Jonathan Leffler (Jonathan.Leffler@Informix.com) #include <disclaimer.h> Guardian of DBD::Informix v1.00.PC1 -- http://www.perl.com/CPAN "I don't suffer from insanity; I enjoy every minute of it!"

>>>>> "JL" == Jonathan Leffler <jleffler@informix.com> writes: Please see below: JL> How many of the loc_t fields do you set (and which JL> ones)? When I do it, I use: blob-> loc_loctype = LOCMEMORY; loc_size = 0; loc_bufsize = -1; blob-> loc_buffer = (char *)0; loc_indicator = 0; loc_oflags = 0; JL> (see SQLCMD from the IIUG software archives, and file JL> ixblob.ec). >> Is this related to the fact that HP-UX byte ordering is >> big-endian while Linux's is little-endian? JL> No; I don't think that can possibly be the problem. JL> Certainly, I've never heard of an endian-ness problem in JL> anything resembling this context. Within the data JL> inside the blob, then yes, you could run into problems, JL> but not in simply fetching it. >> Given that we're able to fetch and store ordinary data types >> just fine, it appears that server-client software recognizes >> the difference in byte ordering between the two or uses some >> kind of XDR format but doesn't extend to BYTE or TEXT columns? JL> Neither the server nor ESQL/C imposes any interpretation JL> on the actual data in the BYTE blob (nor, in practice, JL> in the TEXT blob). If you store data in a binary blob JL> that should be treated with endian issues taken into JL> account, that is (officially) your problem. However, as JL> I said, the retrieval phase should not ever run into JL> this as a problem. The data is collected over the wire JL> in the same way, and the Linux box interprets how to JL> stuff that into the loc_t using just local information. JL> All sizes relayed over the wire are in a defined order, JL> so the specification is neutral between big-endian and JL> little-endian (though one set of machines have to JL> formally remap the bytes into the requisite order for JL> their hardware, but this applies to everything, and is JL> incredibly unlikely to be the problem - speaking from JL> experience). >> Or am I on the wrong path here? Do we have to supply user >> defined open(), read(), write(), close() functions for loc_t in >> our Linux APIs to do the bit swapping? JL> Not at the loc_t level, unless you want to do so. >> I can supply more details on the exact error message we get >> tomorrow (Monday) morning, if needed. JL> Compare my loc_t settings with yours; if there's still JL> trouble, then send the detailed error message, and JL> probably the code too. First, a correction in my original posting on this topic. I have IDS Version 7.30.UC2 on HP-UX 10.20, not 7.31 as stated earlier. Table schema for stdtextproducts: Column name Type Nulls site char(4) no cccid char(3) no nnnid char(3) no xxxid char(3) no product text no Test program "gettext" on Linux box: #include <string.h> EXEC SQL INCLUDE locator; int main() { EXEC SQL BEGIN DECLARE SECTION; loc_t blob; char dbname[32]; EXEC SQL END DECLARE SECTION; blob.loc_loctype = LOCMEMORY; blob.loc_bufsize = -1; blob.loc_size = 0; blob.loc_indicator = 0; blob.loc_buffer = 0; blob.loc_oflags = 0; strcpy(dbname, "fxatext"); EXEC SQL CONNECT TO :dbname; EXEC SQL DECLARE cur CURSOR FOR SELECT product FROM stdtextproducts WHERE cccid = "WBC" AND nnnid = "MTR" AND xxxid = "DCA"; EXEC SQL OPEN cur; EXEC SQL FETCH cur into :blob; return 0; } Compiled with Client SDK 2.40 for Linux % gettext Memory fault (core dumped) % gdb gettext core GNU gdb 19991004 Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are [...] This GDB was configured as "i386-redhat-linux"... Core was generated by `gettext'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libc.so.6...done. Reading symbols from /lib/libm.so.6...done. Reading symbols from /lib/libdl.so.2...done. Reading symbols from /lib/libcrypt.so.1...done. Reading symbols from /lib/ld-linux.so.2...done. Reading symbols from /lib/libnss_files.so.2...done. #0 chunk_alloc (ar_ptr=0x4010bd40, nb=120) at malloc.c:2814 2814 malloc.c: No such file or directory. (gdb) bt #0 chunk_alloc (ar_ptr=0x4010bd40, nb=120) at malloc.c:2814 #1 0x400765ae in __libc_malloc (bytes=113) at malloc.c:2696 #2 0x806ed26 in _sqlocopen () #3 0x806d2ff in _sqg_blob () #4 0x805448a in _iqupdtargs () #5 0x8053007 in _iqftch () #6 0x80528dc in sqli_curs_fetch () #7 0x804a496 in main () at gettext.ec:26 Then we tried SQLCMD which was built on the Linux box: % sqlcmd -d fxatext SQL[1]: select product from stdtextproducts; SQL -403: The size of a received row disagrees with the expected size. Memory fault (core dumped) We downloaded Client SDK 2.40 for Linux. Is this appropriate for the server we have? Mark -- "Experience is a hard teacher. She gives the test first and the lessons afterwards." -- Anonymous