Page 1 of 1
Binary File to String of 0s and 1s
Posted: Sat Oct 17, 2015 8:06 pm
by rok
I'm reading a ~500kB binary file via io.open and io.read. This is very fast. But now I need to get every character that was read from the file converted to a string of zeros and ones.
I wrote my own function for that, but it is
very slow.
Code: Select all
function Converter:UnsignedCharToBits(number)
local output = ""
number = number + 1
for i=8, 1, -1 do
if (number - math.pow(2,i-1)) > 0 then
number = number - math.pow(2,i-1)
output = output.."1"
else
output = output.."0"
end
end
return output
end
A guy from
#love IRC suggested I should try the
basen function from here
http://stackoverflow.com/questions/3554 ... -converter but its still very slow. Does anyone know a faster way?
I'm writing a compression/decompression algorithm. For the decompression I probably have to use this method because I am running through all bits and checking for certain combinations.
Would it be faster If I'd try to somehow do it in smaller chunks?
Re: Binary File to String of 0s and 1s
Posted: Sat Oct 17, 2015 8:19 pm
by bakpakin
Just a thought, but maybe you could create a lookup table for every byte with your current code. That would be a lot faster, and your lookup table would only have 256 elements, which is not too big at all. Also, don't use the concatenation operator in a loop like that; its a huge memory hog. Instead use table.concat.
If your making a compression alogorithm, though, love 0.10.0 will have compression and decompression built in.
Re: Binary File to String of 0s and 1s
Posted: Sat Oct 17, 2015 9:06 pm
by kikito
Just a thought: LuaJIT (included in LÖVE by default) has
a bit operations module. You can probably use it to increase the speed of your algorithm significantly.
Re: Binary File to String of 0s and 1s
Posted: Sat Oct 17, 2015 9:53 pm
by jjmafiae
a quick little optimization would be:
and then call the math_pow
Re: Binary File to String of 0s and 1s
Posted: Sat Oct 17, 2015 10:25 pm
by rok
@bakpakin I swapped all of the
.. with
table.concat and made a lookup table like you suggested. It's a bit faster and the code is cleaner. Thank you
@kikito I'll take a look, thanks
@jjmafiae Ok
Re: Binary File to String of 0s and 1s
Posted: Sun Oct 18, 2015 12:37 am
by bobbyjones
jjmafiae wrote:a quick little optimization would be:
and then call the math_pow
Dude you have it backwards that would not work.
that will work.
Re: Binary File to String of 0s and 1s
Posted: Sun Oct 18, 2015 3:09 am
by undef
Aye you sure you have to check every bit?
At 500kb you'd have 4 million bits to check, maybe it suffices to check each byte?
You could dump the binary data in hex like this:
Code: Select all
local file = io.open( filename, "rb" ) -- I think you need the "b" if you're on Windows
local data = file:read"*a"
local hexDump = data:gsub( ".", function( byte )
return string.format( "%02X ", string.byte( byte ) )
end )
print( hexDump )
This should be fast enough, if not you can try and cache the string functions.
Re: Binary File to String of 0s and 1s
Posted: Sun Oct 18, 2015 7:26 am
by rok
Yeah I have to run over every bit
Now you've guys helped me a lot already
This part is very fast now.
Code: Select all
local lookup = require "lookuptable8bits"
-- ...
local t = {}
for i=1, #file_content, 1 do
table.insert(t, lookup[string.byte(file_content, i)+1])
end
local all_content = table.concat(t) -- doing only one concat at the end
-- ...
Time to optimize the other stuff now and it will be perfect, and I think I already know what to do
Re: Binary File to String of 0s and 1s
Posted: Sun Oct 18, 2015 9:16 pm
by jjmafiae
bobbyjones wrote:jjmafiae wrote:a quick little optimization would be:
and then call the math_pow
Dude you have it backwards that would not work.
that will work.
Oh yeah sorry my brain switched off for a second xD
Re: Binary File to String of 0s and 1s
Posted: Mon Oct 19, 2015 6:00 pm
by T-Bone
jjmafiae wrote:bobbyjones wrote:jjmafiae wrote:a quick little optimization would be:
and then call the math_pow
Dude you have it backwards that would not work.
that will work.
Oh yeah sorry my brain switched off for a second xD
Technically, the original one will be faster. The first time you try to call math.pow, the game will crash because you're trying to call a nil value. This is faster than doing tons of pow calculations
