Rounding issues & Memory

[retrotails]

Sometimes, for whatever reason, Lua says 1 + 1 = 1.99999999. No dt involved, just basic addition. It puzzles me that integers aren't available in Lua. 90% of the time I don't need decimals, and doesn't math with floating point numbers seem like it should be slower and eat more RAM? The only way to get integers is to use the math lib, but that's even slower.
So, a few questions.
How can I avoid 1 + 1 = 1.99999999?
- What causes it? (using love.graphics.print() and a global modified in love.keypressed() if it matters)
- I also get stuff like .9999 - .9999 = -2.8865798640254e-15
If I were to be able to use bytes and integers, could I see a performance increase?
Would it be noticeably more efficient in any manner to have all globals in ONE string? (assuming they're numbers from 0-255)

[micha]

Programming in Lua say this about floating points numbers and integers:

The number type represents real (double-precision floating-point) numbers. Lua has no integer type, as it does not need it. There is a widespread misconception about floating-point arithmetic errors and some people fear that even a simple increment can go weird with floating-point numbers. The fact is that, when you use a double to represent an integer, there is no rounding error at all (unless the number is greater than 100,000,000,000,000). Specifically, a Lua number can represent any long integer without rounding problems. Moreover, most modern CPUs do floating-point arithmetic as fast as (or even faster than) integer arithmetic.

In other words, all integers up to 100,000,000,000,000 are, without rounding errors, also floating point numbers. So the result of 1+1 is 2, if you really write it this way. I suspect that the two 1s you have are not exactly 1 and together the deviation from 1 becomes visible.

Can you post a minimal example (.love file)?

[Boolsheet]

I suspect that the two 1s you have are not exactly 1 and together the deviation from 1 becomes visible.

That's probably what it is.

If you print a number, Lua has to convert it to a string and tries to give you an approximation of that value. This may include rounding. Use string.format to get more control over this conversion.

Code: Select all

a, b = 1, 0.999999999999999
x = a * b
print(x)
print(string.format("%.20f", x))
print(string.format("%.20e", x)) -- or in scientific notation.

Try it in the Lua demo.

The '.20' in the format string means 20 digits after the decimal point which gives us a better approximation of the 64-bit floating-point number.

Also beware of constant folding if you test things like this. Lua can decide to optimize two constants if it sees them together in an operation and that may give unexpected results. Always assign the values to variables first.

[retrotails]

Can you post a minimal example (.love file)?

Code: Select all

x = .9999
j = 0
for i = 1,5 do
  j = j + x
end
for i = 1,5 do
  j = j - x
end
print(j)

Use this in the online Lua demo - I get -2.2204460492503e-16 but it should be 0. I have had integer + integer = something.9999999999 but I can't reproduce that.

[T-Bone]

Can you post a minimal example (.love file)?

Code: Select all

x = .9999
j = 0
for i = 1,5 do
  j = j + x
end
for i = 1,5 do
  j = j - x
end
print(j)

Use this in the online Lua demo - I get -2.2204460492503e-16 but it should be 0. I have had integer + integer = something.9999999999 but I can't reproduce that.

In this example, you set x to 0.9999, not 1. As long as you set everything to integers to begin with, you'll end up with integers. If you involve non-integers anywhere, you can never count on them being exact.

[Boolsheet]

Use this in the online Lua demo - I get -2.2204460492503e-16 but it should be 0.

No, it can't be 0. At least not with 64-bit IEEE floating-point numbers and the default rounding settings. They can't represent 0.9999 exactly and it goes for the closest approximation instead.

Code: Select all

-- In decimal.
0.9999000000000000110134124042815528810024261474609375

-- How it is calculated from the binary data.
(4503149267407759 / 4503599627370496 + 1) -- significand (here the numerator) with the implied 1
* 2^-1 -- exponent

-- The significand part in base 2.
1111111111110010111001001000111010001010011100011110

As you can see, it ranges over almost every bit except the last one of the 52 it has available. When you now start adding numbers, this bit pattern will get shifted to the right to match the exponent of the other number and the lower bits will get lost.

What happens in your example (I think) is that the number close to 3 has a higher exponent causing the 0.9999 value to get rounded up. The subtraction will take away more than the adding put in, resulting in the negative value.

(I hope I got that right.)

[retrotails]

In this example, you set x to 0.9999, not 1. As long as you set everything to integers to begin with, you'll end up with integers. If you involve non-integers anywhere, you can never count on them being exact.

That was a separate question from using integers. Besides, it still shouldn't lose accuracy over adding and subtracting such simple numbers. This causes an issue as well.

Code: Select all

x = .9
j = 0
for i = 1,3 do
  j = j + x
end
for i = 1,3 do
  j = j - x
end
print(j)

[micha]

This causes an issue as well.

Code: Select all

x = .9
j = 0
for i = 1,3 do
  j = j + x
end
for i = 1,3 do
  j = j - x
end
print(j)

These are interesting examples and it's good to keep these problems in mind.

However, I don't see how this really is a problem. Either I work with integer number. Then I am pretty safe to not leave the set of integers. Or I work with floating point number (e.g. 0.9) but then I usually don't expect any result to be an integer anyway. And any function later on should be able to handle floating points numbers. Or, if I already know that a result is an integer, I can math.floor() it.
Do you have an example, where it is important to obtain an integer in the end, while in between you have non-integers?

[Boolsheet]

To address your other questions.

If I were to be able to use bytes and integers, could I see a performance increase?

Depends on a few things, but yes there could be an improvement. It's mostly resolved by using LuaJIT.

Would it be noticeably more efficient in any manner to have all globals in ONE string? (assuming they're numbers from 0-255)

I don't understand this question. Can you rephrase it?

[kikito]

I remember reading something in the Lua mailing list about using integers "under the hood" when numbers were integer-only and switching to float when an integer or non-integer division was found. I don't remember if it was for Lua 5.2.x or for a future Lua. That should make number manipulation a bit faster.

It would not solve the "mixing numbers and floats sometimes gives floats instead of integers" issue, though.