[SOLVED] Lua math.cos issue

[XHH]

I simply want to get cos(90 deg) using the following:

Code: Select all

math.cos(math.rad(90))

This is giving me a nasty 6.1232339957368e-17.
I thought cos(90 deg) was 0. My calculator IRL and google also gives me 0.

[zorg]

Code: Select all

return ('%0.19f'):format(math.cos(math.pi/2)) -- same as if you pass in math.rad(90)
> 0.0000000000000000612

That's just how it's formatted; for all intents and purposes, that is zero, with some floating point error.

[XHH]

Interesting! Thanks for explaining.

[monolifed]

Also don't forget that math.pi is just an approximation of the number pi

[raidho36]

Welcome to the world of floating point numbers. It patently sucks but it's the best we have.

[pgimeno]

Also don't forget that math.pi is just an approximation of the number pi

This is exactly the reason. The value returned is the cosine of that approximation. Something similar happens with math.sin(math.pi).

[raidho36]

This is exactly the reason. The value returned is the cosine of that approximation. Something similar happens with math.sin(math.pi).

Nevermind that math.cos function itself only returns approximate cosine values (and of not particularly great accuracy, FYI).

[pgimeno]

This is exactly the reason. The value returned is the cosine of that approximation. Something similar happens with math.sin(math.pi).

Nevermind that math.cos function itself only returns approximate cosine values (and of not particularly great accuracy, FYI).

Apparently that happens on LuaJIT only. The closest number to Pi/2 that can be exactly represented in double precision floating point is:

1.5707963267948965579989817342720925807952880859375

whose cosine is:

0.0000000000000000612323399573676588613032966137500138418265969...

In my system, Lua reports:

Code: Select all

$ lua
Lua 5.1.5  Copyright (C) 1994-2012 Lua.org, PUC-Rio
> =("%.17g"):format(math.cos(math.pi/2))
6.123233995736766e-17

which is accurate up to the maximum precision of the type, but LuaJIT's result is disastrous:

Code: Select all

$ luajit
LuaJIT 2.0.4 -- Copyright (C) 2005-2015 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse
> =("%.17g"):format(math.cos(math.pi/2))
6.1230317691118863e-17

which is accurate only up to the fourth significant digit.

That said, it's a function for which the IEEE standard gives no guarantees of precision, unlike +,-,*,/,%,sqrt.

[raidho36]

Lua uses C standard library (by design) and it usually provides fairly good scientific functions, horrendously slow though. LuaJIT uses hardware acceleration whenever available (by design) and hardware accelerated floating point math tends to not be very accurate, particularly the transcendental functions such as trigonometry, it's really fast though. I would also argue that 4 significant digits is good enough if you're not trying to do anything silly, like trying to rotate a 100 million mile stick to move 1/8" at the tip; 1 part in 10 000 is equivalent to instrument-grade precision in real life.