Re: Floats
John Shoemaker (fms2000!johns@uunet.UU.NET) wrote: : > > I encounter a precision problem when I want to select a column with : > > float data type. The value i inserted was "15.20", and when i use : > > isql to check, it's fine. But if I used a double type host variable : > > to get its value in my ESQL/C program, it got "15.199999". Could I : > > get the correct data value, or should I change that column's data type : > > if the correct value is very important? : This is a common problem with FLOAT types - in any language. When you : inserted the value into the database, the value of 15.199999 was inserted. : When you query it via isql, isql will *round* it! If you want to look at FYI... isql will "round" to the number of decimal places specified in environment variable DBFLTMASK. It will also "truncate" any input to this number of decimal points as well. We had an application which piped input/output to "isql - - " and noticed that floats were being "rounded/truncated" to 2 decimal places by default (if DBFLTMASK is not defined). : the actual value, you can use an ACE program to print it, specifying : a using: using "#####.########". : If you cannot live with this, you need to use DECIMAL types. : ------------------------------------------------------------------------------ : John M. Shoemaker internet: shoemaker@tweety.digsys.com : Daugherty Systems UUCP: uunet!fms2000!johns : One City Place, Suite 240 FAX: (314) 432-8217 : St Louis, MO 63141 Voice: (314) 432-8200 : ------------------------------------------------------------------------------ : EXEC SQL include "disclaimer.h"; : ------------------------------------------------------------------------------ -- *--------------------------------------------------------------------------* * Ronald R. Larson * * Staff Engineer * * * * Harris Corporation * * Controls Division Phone: (407) 242-5578 * * P.O. Box 430, MS 205 Fax: (407) 242-4268 * * Melbourne, FL 32902-0430 internet: rrl@ccd.harris.com * * EPRInet: UA4N311 * *--------------------------------------------------------------------------*