Rounding mode

[pgimeno]

Is the rounding mode guaranteed to be round-to-nearest in all platforms?

If not, could the engine please set it explicitly to FE_TONEAREST?

[raidho36]

Are you talking about Lua or GL? In Lua there's no rounding, only flooring and ceiling. In OpenGL it's implementation-defined, but "roundEven" always rounds towards nearest even number on the boundary condition (it's slower though, normal rounding is allowed to go either way for performance reasons specifically).

Floating point math in general is not expected to produce any uniform results across different platforms so unless you pass a constant 0.5 to round function it can go either way just because of least significant bit differences.

[pgimeno]

Are you talking about Lua or GL? In Lua there's no rounding, only flooring and ceiling.

I'm talking about Lua. Lua floating-point operations are subject to rounding. See e.g. https://stackoverflow.com/questions/183 ... e#58411671

What I'm seeking is for Löve to guarantee that the FP rounding mode is round-to-nearest-or-even, instead of leaving it up to the OS. I don't think any supported OS defaults to a different rounding mode, but it'd be nice to have explicit support from the engine.

Floating point math in general is not expected to produce any uniform results across different platforms so unless you pass a constant 0.5 to round function it can go either way just because of least significant bit differences.

There should be no least significant bit differences at least in +, -, *, / and square root operations when working with normal (i.e. not denormal) numbers, if the rounding mode is the same in all platforms.

[raidho36]

Oh, that. Well AFAIK there's no way to do this from plain Lua because you need to make an OS call, and there's no way to make it fully portable because it's OS-specific, and LuaJIT basically has --ffast-math flag force-enabled so it will use floating point hardware that's not compliant with IEEE because it's faster.

You should be able to call fesetround on Windows and Linux (constants might be different values though), Android has some JAVA interface but the C call might work, no idea about Apple OSes.

There should be no least significant bit differences at least in +, -, *, / and square root operations when working with normal (i.e. not denormal) numbers, if the rounding mode is the same in all platforms.

"Should" is the keyword. There will be subtle differences and you will spend absolute forever hunting them down and you might never arrive at a fully operational solution. You deal with floating point math quirks by writing code to allow these differences, not by brute forcing it into producing exact results. Or by using fixed point math, that never fails.

[pgimeno]

I just want to be able to rely on the use of a certain rounding mode, nothing else. See the above SO post for an example of its usefulness. I don't want to change the rounding mode from Lua, I'm asking for the engine to enable it.

Oh, that. Well AFAIK there's no way to do this from plain Lua because you need to make an OS call, and there's no way to make it fully portable because it's OS-specific, and LuaJIT basically has --ffast-math flag force-enabled so it will use floating point hardware that's not compliant with IEEE because it's faster.

-ffast-math does many kinds of destructive transformations that LuaJIT does not do. Among other things, it assumes associativity of sums and products, and divides by a constant by pre-calculating the inverse. LuaJIT does neither of these. If addition was considered associative, in the rounding function from the SO post the compiler would probably cancel the constants, and the rounding would not work. Multiplication by the inverse does not always work well; see this thread: https://love2d.org/forums/viewtopic.php?f=5&t=85236 but LuaJIT does not optimize division by 255 to multiplication by the inverse of 255, so if you divide, there's nothing to worry about.

[raidho36]

That's why I said "basically", it doesn't have such setting it doesn't do exactly the same thing. But it does uses fast non compliant hardware whenever available; even if you make an explicit os setting it might not honor it anyway.

You say "just rounding mode" but it's the boundary condition, and you might arrive at a different point entirely if the floating point engine works slightly differently, or even if the jit compiler produces different assembly for some random reason. It's not just rounding setting problem, it's a nasty can of worms about the whole floating point math concept.

If you need to mess with OS settings like this you're probably doing something wrong, in all likelihood it is pretty much used just to patch logical errors elsewhere in the code. Having such setting in the engine is probably a bad idea in many many ways; there's a reason plain Lua doesn't have such facilities.

Also if you just want rounding to nearest integer, you can do floor(value + 0.5), no need to screw around with exponents.

[pgimeno]

If you need to mess with OS settings like this you're probably doing something wrong,

Calling a C++ standard function is not "messing with OS settings", it's just "ensuring we're using the correct rounding even if the OS does not have it as a default".

[wolfboyft]

Round to nearest, ties at even seems most common according to some tests I did with the community. I didn't get back any others. Even so, I would very much appreciate this feature being added. That is to say: bump.

[raidho36]

Calling a C++ standard function is not "messing with OS settings"

People who wrote printer drivers to change floating point OS settings had the same idea.

[pgimeno]

We're not talking an OS- or driver-level setting, but an application-level [Edit: actually thread-level, see https://en.cppreference.com/w/cpp/numeric/fenv] setting mandated by the C++11 standard. The OS just provides a default; it's up to the application to change it. All I'm asking is that LÖVE does this to guarantee a sane rounding mode. It *might* not be necessary, because probably all modern OSes do this as well, but it's better to be able to rely on it regardless of the OS' default.

Edit2: Apparently, gcc builds already guarantee it: https://www.gnu.org/software/libc/manua ... nding.html