Re: Wrong result when divide by float
Posted in 2005
As the Scottish Poet said, don't use FLOAT. However, that is equivalent to a C double, and the error should not be in the second digit -- maybe in the 14th, more likely 15th or 16th. So, the next question is: What version of what software are you using. It looks like I4GL - it could be NewEra, D4GL, EGL or one of I4GL's other heirs and successors. Which platform are you running on? Which version of I4GL? Which version of the database server? Without such information, no-one can help you very much, except by accident. And it would be nice if you wrote a self-contained test program that demonstrated the problem. Those 'double-dots' can sometimes have a big effect on the behaviour of things. It's not all that likely given that the 'same' code works with DECIMAL(32,10), but ... On 25 Jul 2005 04:42:16 -0700, tabusoft@gmail.com <tabusoft@gmail.com> wrote: > I have this source > define tempf float > define a decimal(16,2) > define b integer > > .. > > let a=85945.56 > let b=488 > .. > > let tempf=a/b > > tempf is 156.25 actualu 176.11795081967213114754098360656 > > with this modification > define tempf decimal(32,10) > > evrithing is ok. > > Sugestion > > -- Jonathan Leffler #include <disclaimer.h> Email: jleffler@earthlink.net, jleffler@us.ibm.com Guardian of DBD::Informix v2005.01 -- http://dbi.perl.org/ sending to informix-list