Float Multiplication
Posted in 1994
Topics: Third-Party Tools & Monitoring
I have a problem with calculations in ace reports giving invalid results
(Informix-ISQL version 5.0)
This seems to be an "occasional" problem - i.e. it happened once before and
then all of a sudden the problem "went away". But now it is back.
The problem occurs in an ace report where I am doing calculations using
variables of type "float". For some reason, when I multiply two numbers
which are both type "float", the result (also stored as a type "float"
variable) is rounded to the nearest integer value. Why????
A sample ace report:
database cars end
define
variable calc1 float
variable calc2 float
variable calc3 float
end
select id, loan_amt
from dsstuln_rec
where id = 124677
order by id
end
format
on every row
let calc1 = 1.0
let calc2 = 1.09
let calc3 = calc1 * calc2
print calc1 using "&.&&"," times ",
calc2 using "&.&&"," equals ",
calc3 using "&.&&&"
end
Here is the output I got from that report:
1.00 times 1.09 equals 1.000
Does anyone have any idea why this might happen?
Mark Sadley Dallas Theological Seminary
mark@dts.edu 3909 Swiss Avenue
voice: (214) 841-3785 Dallas, TX 75204
fax: (214) 841-3642
(Geographic location in NO WAY implies endorsement of any NFL team)