a working setColor compatibilty multiplier

[memzen]

When making a library compatible with all versions concerning the new setColor parameters range such that

lg.setColor(r, g, b, a or 255)

becomes

lg.setColor(r * CDIV, g * CDIV, b * CDIV, (a or 255) * CDIV)

using the following formula is not exactly accurate.

local CDIV = love._version_major >= 11 and 1/255 or 1

This is because the result is floored and not rounded. Using this conversion formula does work.

local CDIV = love._version_major >= 11 and 0.003922 or 1

[zorg]

I'm not sure why it would be either floored or rounded; 1/255 is 0.00392156862, and that should work. Löve uses luaJIT by default, so you don't even need to do bull like 1.0/255.0 for explicit non-integers either, like what the newer lua versions would assume you'd *want* to do.

[Nixola]

If I recall correctly, the / operator means float division while the // means integer division, so you wouldn't need to use floats explicitly, but that discussion would be a bit off topic.

[pgimeno]

This thread would be better suited for General, but I think I get the point of the OP. It has to do with FP precision.

Code: Select all

$ luajit
LuaJIT 2.0.4 -- Copyright (C) 2005-2015 Mike Pall. http://luajit.org/
JIT: ON CMOV SSE2 SSE3 SSE4.1 fold cse dce fwd dse narrow loop abc sink fuse
> CDIV = 1/255
> print(math.floor(CDIV*33*255))
32

Or, in more practical and LÖVEly terms (for 11.0):

Code: Select all

local CDIV = 1/255
local img = love.image.newImageData(1, 1)
img:setPixel(0, 0, CDIV*33, CDIV*37, CDIV*41) -- chosen values
local a, b, c = img:getPixel(0,0)
print(a*255, b*255, c*255) -- prints 32, 36, 40

Making the division factor slightly bigger, gets around that problem and manages to assign the right colours to the textures with a byte encoding.

It's easy to see which values cause problems with Python, typing this in the REPL: [CDIV*i*255 for i in range(256)]

[grump]

BTW, when I made cindy, I used multiplication instead of division at first, because I naturally assumed it would be faster. When I started measuring, I was surprised that x / 255 seemed to be consistently faster than x * 0.0039..., by a small margin though.

Code: Select all

local t = love.timer.getTime()
for i = 1, 1e10 do
	local x = 37 / 255
end
print(love.timer.getTime() - t)

local t = love.timer.getTime()
for i = 1, 1e10 do
	local x = 37 * 0.003921568627451
end
print(love.timer.getTime() - t)

Code: Select all

12.711109103999
12.764651178004

I didn't investigate any further and just left / 255 in the code.

[memzen]

This thread could be moved to General but the point is for libraries which are made to work with ALL VERSIONS and to be ACCURATE. I see libraries using r/255, g/255, b/255 but that produces colors on the screen which can be off by 1 for each color component. Not a big problem but this can be avoided.

[grump]

I see libraries using r/255, g/255, b/255 but that produces colors on the screen which can be off by 1 for each color component.

Can you show an example where that happens? I had that only happen with r * (1 / 255), not with r / 255.

Code: Select all

for i = 0, 255 do
	local color = (i / 255) * 255
	assert(color == i, string.format("%d ~= %d", color, i))
end

local m = 1 / 255
for i = 0, 255 do
	local color = (i * m) * 255
	assert(color == i, string.format("%d ~= %d", color, i)) -- fails at 33
end

[memzen]

grump wrote: Can you show an example where that happens? I had that only happen with r * (1 / 255), not with r / 255

No.
I don't know how multiply is different than division for setColor() parameters but dividing does work.
And it looks nicer.
Thanks for the correction.

local CDIV = love._version_major >= 11 and 255 or 1
love.graphics.setColor(r/CDIV, g/CDIV, b/CDIV, (a or 255)/CDIV)

[pgimeno]

It just seems to get rounding right all the time, by pure chance. It doesn't happen with all divisors, but it happens with 255.

Code: Select all

> print(string.format("%.17g", 1/255*255))
1
> print(string.format("%.17g", 1/253*253))
0.99999999999999989

[memzen]

It just seems to get rounding right all the time, by pure chance.

I speculate it is because 255 is at the boundary of 2^8, meaning 255 in binary = 11111111. Maybe because all bits are on the floating point math has the desired rounding up.