float precision ...
Peter Logan asked whether Informix FLOAT (8 bytes) is limited to about 12 decimal digits of precision, since he needed to load SQL Server data with 15-digit precision. Art Kagel replied that FLOAT matches the C double type (roughly 15.5 digits), and recommended the DECIMAL type instead, which supports up to 32 digits, either floating (DECIMAL(32)) or fixed precision (e.g. DECIMAL(17,15)). Martin Fuerderer agreed DECIMAL is generally preferable, as floating-point types carry inherent rounding inaccuracies, especially after arithmetic.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
From what I've been able to find ... a float is 8 bytes ... and it appears to have a precision of 12 decimal places ... Is that the max precision ... or is there something that you can do to get some more ... looking to load some sql server data ... and the precision there is 15 ... might just have to define as a decimal ... any suggestions would be appreciated .... v11.50.fc3 Aix 6.1 Peter Logan Senior Database Administrator Phone: 616/878-8309
You can use the Informix DECIMAL data type which has floating point precision up to 32 decimal digits: DECIMAL(32) or you can make it fixed precision up to 32 digits total, so: DECIMAL(16,16) or DECIMAL(17,15), ETC. OH, BTW, the precision of the FLOAT type is the same as the 'C' double type, about 15.5 digits. Art Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Tue, Oct 6, 2009 at 11:08 AM, Peter_Logan@spartanstores.com < Peter_Logan@spartanstores.com> wrote: > >From what I've been able to find ... a float is 8 bytes ... and it appears > to have a precision of 12 decimal places ... Is that the max precision > .... or is there something that you can do to get some more ... looking to > load some sql server data ... and the precision there is 15 ... might > just have to define as a decimal ... any suggestions would be appreciated > ..... > > v11.50.fc3 > Aix 6.1 > > Peter Logan > Senior Database Administrator > Phone: 616/878-8309 > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --000e0cd6e31c3d1330047545f284
Hi, using a DECIMAL should mostly be preferred over using a FLOAT data type, because the FLOAT data types have the inherent rounding problems (independent of IDS), which means that they are not really exact, especially when later performing math operations on them. Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management IBM Deutschland Research & Development GmbH Chairman of the Supervisory Board: Martin Jetter Board of Management: Erich Baier Corporate Seat: Boeblingen, Germany Reg.-Gericht: Amtsgericht Stuttgart, HRB 243294 ids-bounces@iiug.org wrote on 10/06/2009 05:08:23 PM: > > >From what I've been able to find ... a float is 8 bytes ... and it appears > to have a precision of 12 decimal places ... Is that the max precision > .... or is there something that you can do to get some more ... looking to > load some sql server data ... and the precision there is 15 ... might > just have to define as a decimal ... any suggestions would be appreciated > ..... > > v11.50.fc3 > Aix 6.1 > > Peter Logan > Senior Database Administrator > Phone: 616/878-8309 > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >