Floating Point color

[grump]

A uniform distribution can be obtained by multiplying the random number by 1.003921568627451 (256/255), otherwise the maximum value wont ever be produced. To keep the result in [0, 1] the formula can be: math.min(1.0, love.math.random()*1.003921568627451)

I may be doing something wrong here, but:

Code: Select all

local random, histogram = love.math.random, {}
for i = 0, 255 do histogram[i] = 0 end

for i = 1, 1e8 do
	local r = math.min(1, random() * 1.003921568627451)
	local n = math.floor(255 * r + .5)
	histogram[n] = histogram[n] + 1
end

for i = 0, 255 do print(i, histogram[i]) end

Results:

Code: Select all

0	194956
1	389922
2	389875
3	391098
... (251 pretty uniformly distributed values)
255	585194

That is far from uniform. 0 is underrepresented, 255 is overrepresented.

If you use

Code: Select all

local n = random(0, 255)

it becomes uniformly distributed over the entire range.

No conclusion here, just an observation.

EDIT:

Edit: Sorry, that's wrong, I get you now. You're right, given the rounding in the internal conversion, you need to compensate with this formula:
math.floor(love.math.random()*256)/255

Yep, that works fine.

[MachineCode]

Let me just explain why I was looking at this and became confused. I have been working on a drawing engine that uses 16 bit instructions to do basic primitive things - like set pixel, draw block. Because of the 16 bit format, 4:4:4 color (12bit) is the native format, so to encode an image you first need to clip it back to 4:4:4 color space. The instructions also support a 10 bit color 3:4:3 space (green always gets the most bits).

Why restrict color spaces? Well, apart from data size, another reason is that restricting color space changes the "look" of the result and can give a retro quality. The biggest drawback is that images with gradual color changes suffer from banding, which can be annoying.

It turns out that you can actually apply a type of "dither" to 4:4:4 or 3:4:3 images to restore them to very close to the original 8:8:8 representation. Effectively you steal a bit of the color bandwidth and move it into the intensity bandwidth. For each color component of a pixel, you split the fractional integer up into an int.frac and use the fractional part as a statistical weight to decide whether that int part is bumped up to the next value. So, in a 4:4:4 encode from 8:8:8 space, each pixel is encoded as a weighted random value from the 8 possible values r = d,d+1 g = e,e+1 b = f,f+1

This encoding imposes a bit of grain on the image, however on most lcd displays, the eye just integrates this and it is invisible. I even tried it with 256 color 3:3:2 and you get visible noise, but the full color range is effectively restored. A nice feature of this scheme is that it can be applied pixel by pixel, so you could use 3:4:3 mark out regions of the image with faces or skintone for enhancement and leave the rest as is.

So, here is the encoding function to do this dithering. This is the old pre 11 code that uses integer color value. To fix these for 11.x, needs a conversion from fp to integer, so that led me to think about how that works.

Code: Select all

local function clip_10( x, y, r, g, b, a )
		local rr = band(r, 0x1f)/32
		local gg = band(g, 0x0f)/16
		local bb = band(b, 0x1f)/32
		r = band(r,  0xe0) 
		g = band(g,  0xf0)
		b = band(b,  0xe0)
		if rr > rand() then r = r+31 end
		if gg > rand() then g = g+15 end
		if bb > rand() then b = b+31 end
		return r,g,b,a
	end
	
local function clip_12( x, y, r, g, b, a )
		local rr = band(r, 0x0f)/16
		local gg = band(g, 0x0f)/16
		local bb = band(b, 0x0f)/16
		r = band(r,  0xf0) 
		g = band(g,  0xf0)
		b = band(b,  0xf0)
		if rr > rand() then r = r+15 end
		if gg > rand() then g = g+15 end
		if bb > rand() then b = b+15 end
		return r,g,b,a
	end

You can do this in the world of fp, but thinking about color spaces as integers and int.fracs just seems a lot more precise and natural to me.

[pgimeno]

Just wait for 11.3 and use colorToBytes (or make your own, or use cindy).

[MachineCode]

Currently I am using

Code: Select all

 col_int = floor(255.9 * col_fp /256) 

That effectively divides the color space up into 256 regions and excludes 1.0 from giving the out of range 9bit value of 256.

[MachineCode]

Brain snap when I typed that above. i was mixing up the forward and reverse conversions.

col_int_8 = floor(255.9 * col_fp)

col_fp_4bit = floor(15.9 * col_fp) / 15 -- that gives 0 --> 1 in the floating point range, clipped to a 4 bit space.

It just goes to show that the continuous representation of color is not at all as simple as it seems.

[MachineCode]

This is what the color space transforms look like. Compared to the 8:8:8, the 4:4:4 has banding. In the third image, it is still 4:4:4, but weighted dither applied to the lsb's of the color components. The banding is gone but there is some graininess visible.

sky-set.png (141.06 KiB) Viewed 5584 times