Bit-based file loading?

[RonanZero]

As opposed to:

Code: Select all

f = love.filesystem.newFile("data.bin");

f:open("r");
	local data = f:read();
f:close();

which, as I understand it, is byte-based file loading because it returns 640 which is the length, in bytes, of the file I loaded. The only problem is each piece of data in the file is 4 bits, or half a byte. Is it possible to load a file like that, but bit-based?

Excerpt from hex data of the file:

Code: Select all

66 66 00 00 60 33 33 06

Instead of reading "66, 66, 00, 00, 60, 33, 33, 06" I want to read it as "6, 6, 6, 6, 0, 0, 0" etc. (not sure what love does, it's a bit confusing, but it causes the length to be 640)

[ivan]

f:read() reads bytes and each byte is a value between 0-255 (that's 8 bits).
A hex is 4 bits so you can pack two hexes in a single byte.
For example:

Code: Select all

local n = string.byte(chunk, offset) -- n is between 0-255
local lo = n%16 -- low part, first hex
local hi = (n - lo)/16 -- high part, second hex

That's how you split a byte into 2 hexes.
In order to put them back together:

Code: Select all

local lo = 14 -- first hex, between 0-15
local hi = 3 -- second hex, between 0-15
local n = lo + hi*16

Note that it's also possible to pack 2 bytes into a single 16-bit "WORD" value and vice versa.
Having said that, I'll go out on a limb and ask - are you sure the file is really composed of hexes?

[RonanZero]

Yes, it's a grayscale raw image data file. (.gray) I was able to load it only half-correctly.

I'm guessing chunk is the entire file data?

[ivan]

What do you mean by "raw image data"? Even BMP files have headers and so on. If it's "indexed color" then the header may contain the palette too. A while ago I wrote a pure Lua BMP loader, but I'm not sure if this is really the route you want to be going in.

[RonanZero]

What do you mean by "raw image data"? Even BMP files have headers and so on. If it's "indexed color" then the header may contain the palette too. A while ago I wrote a pure Lua BMP loader, but I'm not sure if this is really the route you want to be going in.

It's indexed color without a header or footer; it's 4-bit and 80x16, leading to 640 bytes. The palette and w/h is stored elsewhere. It's not practical, but I'm doing this whole thing as a sort of experiment.

[ivan]

If it's an image, why can't you just save the file to a format that Love2D can load automatically?

[RonanZero]

Wanted to see how close I could get to how the SNES does things. Also wanted to see how small I could get it.

[Positive07]

I think you are going out of the way to acomplish a really simple task... I could do whatever you are doing with much less effort, storing a 4bit greyscale BMP and then using a shader to color it with whatever palette I need. It has been done multiple times, rm-code used a similar system for his bomberman clone.

If you want to split an 8bit string I would do it like this:

Code: Select all

local bit = require "bit"
local encoded = "your string or whatever data"
local decoded = {}

for i=1, #encoded do
   local byte = string.char(encoded, i)
   --Invert this line to match whatever endianness you need
   table.insert(decoded, bit.rshift(byte, 4))    --Higher nibble
   table.insert(decoded, bit.band(byte, 0x0F) --Lower nibble
end

Even then this gets more and more inefficient when you keep adding images, and if you have to turn that into ImageData then even more so!
LÖVE is far more efficient at dealing with images, and it knows quite a few formats.

Do you have a reason for loading a raw format that is not standardized at all? Can't you turn your images into more common formats? Is there any benefit at doing it this way?

[ivan]

Wanted to see how close I could get to how the SNES does things. Also wanted to see how small I could get it.

You really don't want to be emulating this sort of thing in Lua for several reasons.
For example, Windows bitmaps have a lot of idiosyncrasies like the width being a multiple of 4 and so forth.
So I would assume SNES data would be working with even more obscure limitations.
Sure it may be possible to parse those files in Lua, but it's not something you do in realtime -
that is, 99% of the time you want to convert these assets once and not worry about it.

If size is your concern then, PNG with indexed color (optimized using something like PNGauntlet)
is as small as you can get (since that uses actual compression).
Also, make sure to pack your all of your assets in a single tilesheet which can save a lot of padding space.

PS. Thumbs up to Positive07 for a saner solutions using the "bit" module.

[RonanZero]

So you're saying I can turn a 4-bit 80x16 image smaller than 640 bytes? And about dimensions, 80x16 is only a cut-out for testing.