Page 1 of 1
Why is 2-2 = 8.8817841970013e-016
Posted: Thu Apr 25, 2019 1:44 am
by ITISTHEBKID
Here is my code:
Code: Select all
io.stdout:setvbuf("no")
function love.load()
bope(6,2)
end
function bope(xs,w)
for i=1,100,1 do
numb = (i/xs%1)*xs
if numb == 0 then
numb = xs
end
x = (w * numb)
print(x,w,x-w)
end
end
as you can see at the bottom, it prints x,w and then x-2. After the first iteration, every time x = 2, x-w is equal to a seemingly random number. w is always equal to 2, so why is my program telling me 2-2 = 8.8817841970013e-016?
Re: Why is 2-2 = 8.8817841970013e-016
Posted: Thu Apr 25, 2019 2:44 am
by zorg
The short answer is that those values are close enough to zero that they are effectively zero; note the e-016, meaning the number's about 0.00000000000000008 whatever- pretty close to zero.
Longer answer is that lua uses 64bit double-precision floating point numbers for its number values, and there are operations on floats that when done in succession will accumulate and propagate errors... the longer that goes on, the larger the error will be.
Re: Why is 2-2 = 8.8817841970013e-016
Posted: Thu Apr 25, 2019 2:50 am
by ITISTHEBKID
Thanks, I understand now. Is it worth it to put something like
or will it not impact preformance
Re: Why is 2-2 = 8.8817841970013e-016
Posted: Thu Apr 25, 2019 4:33 am
by ivan
Also note that the precision will be incorrect for very large numbers too.
You should be fine as long as you don't compare numbers directly, like a == b
That's just how floating point math works.
I don't recommend it, but I've seen things like:
Code: Select all
math.epsilon = 0.001
function isEqual(a, b)
return math.abs(a - b) < math.epsilon
end
Like I said, if you are careful with your floating point numbers, you should be fine.
Re: Why is 2-2 = 8.8817841970013e-016
Posted: Thu Apr 25, 2019 9:32 am
by pgimeno
To display the actual numbers you're operating with, you can use string.format("%.17g", value). Lua's default format does not always distinguish different values, but that format string does.
Re: Why is 2-2 = 8.8817841970013e-016
Posted: Thu Apr 25, 2019 11:25 am
by raidho36
As long as you use exact integers for math operations, exact integers will come out - this is a special property of double precision floating point format. Doubles are 64 bit, but 11 bits is reserved for the exponent value, so your integers are limited to 53 bits. It can still hold very large values (in 9 quadrillion range) so it shouldn't be a limiting factor if you use integer math for practical item counting and such.
Re: Why is 2-2 = 8.8817841970013e-016
Posted: Fri Apr 26, 2019 3:17 am
by zorg
raidho36 wrote: ↑Thu Apr 25, 2019 11:25 am
As long as you use exact integers for math operations, exact integers will come out - this is a special property of double precision floating point format. (...) if you use integer math for practical item counting and such.
Addition, subtraction, multiplication, "integer division" and powers are the math ops specifically that will give back exact integer output for integer inputs, within the previously mentioned range, of course.