SMALLINT equivalent in SQL Server.

I'm importing several SE table schemas to SQL Server, where SMALLINT does not exist. So I changed SMALLINT to INTEGER, then ETL'd the SE data to the SQL tables and I'm getting a data load error with the ifx SMALLINT value, which is between -32,767 and +32,767.. (Strange!).. Is there a byte-alignment problem with the way the ifx SMALL INT value is represented?

Did you export the SE data to a delimited file? If so, it should just import into an integer with no trouble. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Oct 17, 2012 at 12:01 AM, FRANK J. COMPUTER <frank_in_pr@hotmail.com > wrote: > I'm importing several SE table schemas to SQL Server, where SMALLINT does > not > exist. So I changed SMALLINT to INTEGER, then ETL'd the SE data to the SQL > tables and I'm getting a data load error with the ifx SMALLINT value, > which is > between -32,767 and +32,767.. (Strange!).. Is there a byte-alignment > problem > with the way the ifx SMALL INT value is represented? > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --e89a8ff1c94c71eccf04cc3a153c

>Did you export the SE data to a delimited file? If so, it should just >import into an integer with no trouble. Hello Art, I used SE 4.10's UNLOAD statement, with pipe delimiters. I examined the .unl flat file and did not detect any anomaly with SMALLINT or any other values.

Yeah, so the file should have been imported with that field mapped to a 32bit integer with no problems. This sounds like a problem with the SQL Server import process. Text is text and a 'text' number looks the same to everyone and there is no difference between a text "99" whether it was exported from a SMALLINT, INTEGER, or BIGINT from a big-endian, little-endian, or big-little-endian machine, from Informix SE or Oracle, or SQL Server. So... Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Oct 17, 2012 at 12:35 PM, FRANK J. COMPUTER <frank_in_pr@hotmail.com > wrote: > >Did you export the SE data to a delimited file? If so, it should just > >import into an integer with no trouble. > > Hello Art, I used SE 4.10's UNLOAD statement, with pipe delimiters. I > examined > the .unl flat file and did not detect any anomaly with SMALLINT or any > other > values. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --14dae93406d797938704cc43f39c

Had this problem yesterday. I changed the SQLServer column to an int = and took out the trailing field separator and it loaded fine. j. On Oct 17, 2012, at 12:35 PM, FRANK J. COMPUTER wrote: >> Did you export the SE data to a delimited file? If so, it should just=20= >> import into an integer with no trouble.=20 >=20 > Hello Art, I used SE 4.10's UNLOAD statement, with pipe delimiters. I = examined=20 > the .unl flat file and did not detect any anomaly with SMALLINT or any = other=20 > values.=20 >=20 >=20 > = **************************************************************************= *****=20 > Forum Note: Use "Reply" to post a response in the discussion forum.=20= >=20

Ahh yes, I should have remembered that one, it's bitten me often enough. Only Informix supports that trailing field separator and it's required for import too! Never understood why, historical I know, it's always been that way, but dumb. I have a very old spec on a standard for delimited data files, and NO ONE actually does it correctly, but nowhere in the spec is a trailing separator allowed. Of course, I may be one of only 10 people who still remember there ever was a standard. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Oct 17, 2012 at 12:45 PM, Jack Parker <jack.parker4@verizon.net>wrote: > Had this problem yesterday. I changed the SQLServer column to an int = > and took out the trailing field separator and it loaded fine. > > j. > > On Oct 17, 2012, at 12:35 PM, FRANK J. COMPUTER wrote: > > >> Did you export the SE data to a delimited file? If so, it should > just=20= > > >> import into an integer with no trouble.=20 > >=20 > > Hello Art, I used SE 4.10's UNLOAD statement, with pipe delimiters. I = > examined=20 > > the .unl flat file and did not detect any anomaly with SMALLINT or any = > other=20 > > values.=20 > >=20 > >=20 > > = > **************************************************************************= > *****=20 > > Forum Note: Use "Reply" to post a response in the discussion forum.=20= > > >=20 > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --14dae93406d7acdc4c04cc4415ed

Art, You can't be expected to remember everything. It is amazing to me how = much you do remember and how much help you provide through this and = other forums. cheers j. On Oct 17, 2012, at 12:51 PM, Art Kagel wrote: > Ahh yes, I should have remembered that one, it's bitten me often = enough.=20 > Only Informix supports that trailing field separator and it's required = for=20 > import too! Never understood why, historical I know, it's always been = that=20 > way, but dumb. I have a very old spec on a standard for delimited data=20= > files, and NO ONE actually does it correctly, but nowhere in the spec = is a=20 > trailing separator allowed. Of course, I may be one of only 10 people = who=20 > still remember there ever was a standard.=20 >=20 > Art=20 >=20 > Art S. Kagel=20 > Advanced DataTools (www.advancedatatools.com)=20 > Blog: http://informix-myview.blogspot.com/=20 >=20 > Disclaimer: Please keep in mind that my own opinions are my own = opinions=20 > and do not reflect on my employer, Advanced DataTools, the IIUG, nor = any=20 > other organization with which I am associated either explicitly,=20 > implicitly, or by inference. Neither do those opinions reflect those = of=20 > other individuals affiliated with any entity with which I am = affiliated nor=20 > those of the entities themselves.=20 >=20 > On Wed, Oct 17, 2012 at 12:45 PM, Jack Parker = <jack.parker4@verizon.net>wrote:=20 >=20 >> Had this problem yesterday. I changed the SQLServer column to an int = =3D=20 >> and took out the trailing field separator and it loaded fine.=20 >>=20 >> j.=20 >>=20 >> On Oct 17, 2012, at 12:35 PM, FRANK J. COMPUTER wrote:=20 >>=20 >>>> Did you export the SE data to a delimited file? If so, it should=20 >> just=3D20=3D=20 >>=20 >>>> import into an integer with no trouble.=3D20=20 >>> =3D20=20 >>> Hello Art, I used SE 4.10's UNLOAD statement, with pipe delimiters. = I =3D=20 >> examined=3D20=20 >>> the .unl flat file and did not detect any anomaly with SMALLINT or = any =3D=20 >> other=3D20=20 >>> values.=3D20=20 >>> =3D20=20 >>> =3D20=20 >>> =3D=20 >> = **************************************************************************= =3D=20 >> *****=3D20=20 >>> Forum Note: Use "Reply" to post a response in the discussion = forum.=3D20=3D=20 >>=20 >>> =3D20=20 >>=20 >>=20 >>=20 >>=20 > = **************************************************************************= *****=20 >> Forum Note: Use "Reply" to post a response in the discussion forum.=20= >>=20 >>=20 >=20 > --14dae93406d7acdc4c04cc4415ed=20 >=20 >=20 > = **************************************************************************= *****=20 > Forum Note: Use "Reply" to post a response in the discussion forum.=20= >=20

Art wrote: Ahh yes, I should have remembered that one, it's bitten me often enough. Only Informix supports that trailing field separator and it's required for import too! Never understood why, historical I know, it's always been that way, but dumb. I have a very old spec on a standard for delimited data files, and NO ONE actually does it correctly, but nowhere in the spec is a trailing separator allowed. Of course, I may be one of only 10 people who still remember there ever was a standard. ------------------------- 1. Didn't know a standard existed for import/export of data, other than CSV. 2. So how can I tell Informix to suppress the trailing delimiter? 3. The SMALLINT value is in between other values, example: ABC|32766|10172012| 4. The SQLS import did not fail on the first line read.

See my comments inline below: Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Oct 17, 2012 at 2:10 PM, FRANK J. COMPUTER <frank_in_pr@hotmail.com>wrote: > Art wrote: > > Ahh yes, I should have remembered that one, it's bitten me often enough. > Only Informix supports that trailing field separator and it's required for > import too! Never understood why, historical I know, it's always been that > way, but dumb. I have a very old spec on a standard for delimited data > files, and NO ONE actually does it correctly, but nowhere in the spec is a > trailing separator allowed. Of course, I may be one of only 10 people who > still remember there ever was a standard. > ------------------------- > > 1. Didn't know a standard existed for import/export of data, other than > CSV. > Yup, several actually, but even the CSV standard is not followed by any software out there. The closest to come to compliance was a card-oriented database in the early '90s that's long gone. Among other things everyone misses is a requirement for all "text" type fields to be quoted with double quotes. Most systems do not write the quotes when they create delimited or CSV files, many also do not process them properly - including the quotes in the parsed data instead of treating them as delimiters. Xcel and Open Office Calc handle the import OK (except that they do not automatically assign numeric attributes to fields that are not quoted), but don't quote properly on output. <sigh> > 2. So how can I tell Informix to suppress the trailing delimiter? > You cannot, though it's not the engine, but the tools like dbaccess and dbexport that are producing the export file. You have to strip out the trailing delimiter as a post-process. Here's a sed script for that if it helps: sed 's/|$//' <infile >outfile Another option would be to use Jonathan Leffler's sqlcmd package which can produce a fairly compliant CSV file without the trailing delimiter. However, I don't know if it will still work with and SE database. 3. The SMALLINT value is in between other values, example: > ABC|32766|10172012| > So you ultimately want the record to look like: ABS|32766|10172012 Note that while I understand that this was a hand carved sample, 32766 isn't a valid SMALLINT, though it would be a valid INTEGER. ;-) > 4. The SQLS import did not fail on the first line read. > Right, because it got confused by the trailing delimiter into thinking that a new record had started with an record delimiter character embedded in the second field. IIRC SQLS has you define the number of columns in the data file and so, contrary to the standard which would require it to obey the record delimiter and either ignore the extra column implied by the trailing delimiter or declare an error condition, it assumed that once it had parsed the Nth data filed a new record was starting even though it had not found a record delimiter yet. That's the other major "bug" in most delimited file processors. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --14dae9341075200e1a04cc4597a2

SEE MY COMMENTS WITHIN *** > Art wrote: > > Ahh yes, I should have remembered that one, it's bitten me often enough. > Only Informix supports that trailing field separator and it's required for > import too! Never understood why, historical I know, it's always been that > way, but dumb. I have a very old spec on a standard for delimited data > files, and NO ONE actually does it correctly, but nowhere in the spec is a > trailing separator allowed. Of course, I may be one of only 10 people who > still remember there ever was a standard. > ------------------------- > > 1. Didn't know a standard existed for import/export of data, other than > CSV. > Yup, several actually, but even the CSV standard is not followed by any software out there. The closest to come to compliance was a card-oriented database in the early '90s that's long gone. Among other things everyone misses is a requirement for all "text" type fields to be quoted with double quotes. Most systems do not write the quotes when they create delimited or CSV files, many also do not process them properly - including the quotes in the parsed data instead of treating them as delimiters. Xcel and Open Office Calc handle the import OK (except that they do not automatically assign numeric attributes to fields that are not quoted), but don't quote properly on output. <sigh> *** card-oriented db like "Tracker"?.. So, has any industry attempt been made to create an import/export standard? *** > 2. So how can I tell Informix to suppress the trailing delimiter? > You cannot, though it's not the engine, but the tools like dbaccess and dbexport that are producing the export file. You have to strip out the trailing delimiter as a post-process. Here's a sed script for that if it helps: sed 's/|$//' <infile >outfile Another option would be to use Jonathan Leffler's sqlcmd package which can produce a fairly compliant CSV file without the trailing delimiter. However, I don't know if it will still work with and SE database. *** Thanks for the sed script!.. It would be a good idea to add a "WITH NO TRAILING DELIMITER" directive to the dbaccess UNLOAD statement. *** 3. The SMALLINT value is in between other values, example: > ABC|32766|10172012| > So you ultimately want the record to look like: ABS|32766|10172012 *** YES *** Note that while I understand that this was a hand carved sample, 32766 isn't a valid SMALLINT, though it would be a valid INTEGER. ;-) *** Even though I'm trying to load 32766 into an INTEGER column, why would 32766 be an invalid SMALLINT value if overflow does not occur until a value is >= 32768? *** > 4. The SQLS import did not fail on the first line read. > Right, because it got confused by the trailing delimiter into thinking that a new record had started with an record delimiter character embedded in the second field. IIRC SQLS has you define the number of columns in the data file and so, contrary to the standard which would require it to obey the record delimiter and either ignore the extra column implied by the trailing delimiter or declare an error condition, it assumed that once it had parsed the Nth data filed a new record was starting even though it had not found a record delimiter yet. That's the other major "bug" in most delimited file processors. *** Your explanation absolutely makes sense! ***

Back at you, below: Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Oct 17, 2012 at 10:49 PM, FRANK J. COMPUTER <frank_in_pr@hotmail.com > wrote: > SEE MY COMMENTS WITHIN *** > > > Art wrote: > > > > Ahh yes, I should have remembered that one, it's bitten me often enough. > > Only Informix supports that trailing field separator and it's required > for > > import too! Never understood why, historical I know, it's always been > that > > way, but dumb. I have a very old spec on a standard for delimited data > > files, and NO ONE actually does it correctly, but nowhere in the spec is > a > > trailing separator allowed. Of course, I may be one of only 10 people who > > still remember there ever was a standard. > > ------------------------- > > > > 1. Didn't know a standard existed for import/export of data, other than > > CSV. > > > > Yup, several actually, but even the CSV standard is not followed by any > software out there. The closest to come to compliance was a card-oriented > database in the early '90s that's long gone. Among other things everyone > misses is a requirement for all "text" type fields to be quoted with double > quotes. Most systems do not write the quotes when they create delimited or > CSV files, many also do not process them properly - including the quotes in > the parsed data instead of treating them as delimiters. Xcel and Open > Office Calc handle the import OK (except that they do not automatically > assign numeric attributes to fields that are not quoted), but don't quote > properly on output. <sigh> > > *** card-oriented db like "Tracker"?.. So, has any industry attempt been > made > to create an import/export standard? *** > The product I was thinking of was called DataEase, which apparently is still around. IIRC they even supported both optional header records (Excell supports the column name record but not the datatype header record). The delimited file, or CSV file, standard was supposed to be that standard. Then case tagged entry format which was adopted by the financial industry and is still the basis of the EDI (Electronic Data Interchange) format standard that the banking industry still uses AFAIK. That was followed by the XML standard. So, yes, many attempts. > > > 2. So how can I tell Informix to suppress the trailing delimiter? > > > > You cannot, though it's not the engine, but the tools like dbaccess and > dbexport that are producing the export file. You have to strip out the > trailing delimiter as a post-process. Here's a sed script for that if it > helps: > > sed 's/|$//' <infile >outfile > > Another option would be to use Jonathan Leffler's sqlcmd package which can > produce a fairly compliant CSV file without the trailing delimiter. > However, I don't know if it will still work with and SE database. > > *** Thanks for the sed script!.. It would be a good idea to add a "WITH NO > TRAILING DELIMITER" directive to the dbaccess UNLOAD statement. *** > Would be nice. Call tech support or go to online support and enter a feature request. > > 3. The SMALLINT value is in between other values, example: > > ABC|32766|10172012| > > > > So you ultimately want the record to look like: ABS|32766|10172012 > > *** YES *** > > Note that while I understand that this was a hand carved sample, 32766 > isn't a valid SMALLINT, though it would be a valid INTEGER. ;-) > > *** Even though I'm trying to load 32766 into an INTEGER column, why would > 32766 be an invalid SMALLINT value if overflow does not occur until a > value is > >= 32768? *** > Senior moment triggered by the fact that many Informix 16bit data strucgtures internally have a maximum value of 32765 - DUH! 8^( > > > 4. The SQLS import did not fail on the first line read. > > > > Right, because it got confused by the trailing delimiter into thinking that > a new record had started with an record delimiter character embedded in the > second field. IIRC SQLS has you define the number of columns in the data > file and so, contrary to the standard which would require it to obey the > record delimiter and either ignore the extra column implied by the trailing > delimiter or declare an error condition, it assumed that once it had parsed > the Nth data filed a new record was starting even though it had not found a > record delimiter yet. That's the other major "bug" in most delimited file > processors. > > *** Your explanation absolutely makes sense! *** > We aim to please. B^) > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --14dae93406d7c661fe04cc4d161c