Page 1 of 2

Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 12:34 pm
by grump

Code: Select all

local inp = 1 / 42
love.graphics.setColor(inp, inp, inp)
local outp = love.graphics.getColor()
print(inp, outp)
Output:

Code: Select all

0.023809523809524	0.023809524253011
I'm not sure if this is considered a bug, but I would expect getColor() to return the same values that were set with setColor(). It's caused by the Colorf type on the C++ side, because of the precision loss that occurs (double vs. float).

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 1:14 pm
by zorg
Yes, the internal color precision of any texture will never be as precise as a double float, unless there'd be such a format supported by OpenGL. In short, it's not a bug. Similarly you can't set SoundData samplepoints to (normalized [-1,1] instead of [0,1]) values it couldn't represent with the chosen bit depth; it gets converted to the nearest possible value... although i'm assuming that since i usually only ever assigned correct values to them...

Still, for colors, the code will coerce the values to the nearest representable ones, so if you really want exact values (which are still probably os-dependent, but functionally generating them should work) then you could make tables:

Code: Select all

local int8 = {}
for i=0,255 do int8[i] = i/255 end
if you want to use 16 bit per color channel then the constant to divide by is 65535; basically it's (2^n)-1.

Also, you don't need to pre-generate those values, of course; you can use such a thing to check inputs whether they can be exactly represented or not, if you prefer errors/asserts in these areas. I think i already posted a snippet to the 11.0 thread, or a thread similar to that.

Edit: i realize that color values may be floats on the shader end but again, that's even less of a real concern, in my opinion; that said, i'm not too knowledgeable on that side so it's possible i'm wrong.

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 1:39 pm
by raidho36
Colors were always floats on the shader end, integers have never been used for the purpose. Mainly because the output would look extremely ugly.

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 1:41 pm
by grump
I get what you're saying, but it's not an OpenGL, Texture or Shader issue. Graphics::setColor on the C++ side of things tracks the color independently from OpenGL. It does it with 32 bit precision, while Lua uses 64 bit precision.

It's not a huge issue, I was just surprised by the precision loss. It came up when I wrote code to convert between the [0-255] and [0-1] color range and my unit tests failed with some values.

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 1:44 pm
by zorg
Okay i misunderstood; that's actually a legit thing... i wonder why it does it like that.
My best mostly uneducated guess would be that löve also has a 32bit version, and 64bit values are slower on that? Not sure how much sense this makes though, probably not much. :crazy: In which case it should be turned into a double.

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 1:59 pm
by egorcod
You can use simple patch:

Code: Select all

local lg = love.graphics

local oldSetColor = lg.setColor
local oldGetColor = lg.getColor
local oldReset = lg.reset

local def = {1, 1, 1, 1}
local color = def

lg.setColor = function(...)
  color = {...}
  oldSetColor(...)
end

lg.getColor = function()
  return unpack(color)
end

lg.reset = function()
  color = def
  oldReset()
end

Code: Select all

require 'colors'
local inp = 1 / 42
love.graphics.setColor(inp, inp, inp)
local outp = love.graphics.getColor()
print(inp, outp)
Result:

Code: Select all

0.023809523809524	0.023809523809524

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 2:09 pm
by ivan
I'm with Grumps on this one.
getColor returns junk after the 7th decimal REGARDLESS of the "actual" input precision.
So for example:

Code: Select all

local lg = love.graphics
local c1 = 0.1
lg.setColor(c1,c1,c1)
local c2,_,_,_ = lg.getColor()
assert(c1 == c2) -- assertion failed
One "fix" could be to format/truncate the output of getColor.

Code: Select all

local function d2f(n)
  return n - n%0.0000001 -- positive numbers only
end
function love.graphics.getColor2()
  local r,g,b,a = love.graphics.getColor()
  return d2f(r),d2f(g),d2f(b),d2f(a)
end
Not very pretty but ensures that the test above is passed.
getColor is used rarely compared to setColor so it's not a big deal if it's a little bit slower.

PS. Of course, input must be floating point so anything after the 7th decimal or so will be truncated!
PPS. egorcod, that won't work for things like imageData:getPixel()

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 2:38 pm
by pgimeno
You can also adjust your unit tests with FFI:

Code: Select all

local ffi = require 'ffi'

print(tonumber(ffi.cast('float', 0.1))) -- prints 0.10000000149012
Edit: Alternatively, you can use math.ldexp and math.frexp, but getting the rounding right is costly.

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 2:49 pm
by ivan
It's not really a big issue, but it should be noted somewhere.
Oh yea, did I mention that box2d/love.physics also works with floats? :)

Re: Precision issue with lg.getColor()

Posted: Sun Apr 22, 2018 5:42 pm
by slime
Indeed, almost every love API that accepts a number stores the number in something less precise than a 64 bit float. That's a lot of APIs.