Since this isn't related to the LOVE part of lua, i just dropped it here.
I don't know much about the stuff that goes on "backstage" in LUA, but i noticed something odd the other day...
I were making this "table/nested tables to file function", which basically made the table into a very long string. When i finished the function, I wanted to test the limits - so i made a humongous table! After about 4-5 seconds LOVE crashed, which was odd. I figured that it had to be the string that was getting too long, since the writing process hadn't started yet.
The question to you guys: Does LUA have a "max signs in string limit"... or was it gremlins?
PS. the function was running in a coroutine, and adding each index in the table to a string for each turn!
Silly strings...
- Robin
- The Omniscient
- Posts: 6506
- Joined: Fri Feb 20, 2009 4:29 pm
- Location: The Netherlands
- Contact:
Re: Silly strings...
Can we see your code? I have a suspicion about the problem, which won't occur if you use table.concat, but usually does when concatenating strings in other ways.
Help us help you: attach a .love.
Re: Silly strings...
The code is extremely braided and long, but i'll pick out the important parts:
Reading the table:
The table-to-string code:
I "fixed" it by making a separate file for each index which contains another table...
The important part is that it crashes when the "str" is too long.
By the way - you may all use the table functions if you want to.
Reading the table:
Code: Select all
-- Yada-yada lots of generation code
local str = "local map = {}\n"
local function add( s )
str = str .. s;
end
local num = 1
local function write(n, str)
local path = "maps/" .. name .. "/"
if love.filesystem.isDirectory( path ) then
local success = love.filesystem.write( path .. tostring(n) .. ".lua", str )
return success;
else
local ok = love.filesystem.mkdir( path )
if ok then
local success = love.filesystem.write( path .. tostring(n) .. ".lua", str )
return success;
else
return false, "could not save table"
end
end
end
for x, t in pairs(map) do
add( "map["..x.."] = {}\n" )
for y, g in pairs(map[x]) do
add( "map["..x.."]["..y.."] = {}\n" )
add( table.getString( "map["..x.."]["..y.."]", g ) )
end
add( "return map;" )
write(num, str)
num = num + 1
str = "local map = {}\n"
percent = 25 + (x/#map)*75
-- this is where i reran the thread.
end
end
Code: Select all
function table.getString( base, tab )
local str = ""
local function add( s )
str = str .. s;
end
for id, info in pairs(tab) do
local text = base
if type(id) == "string" then
text = text .. "['" .. id .. "'] = "
elseif type(id) == "number" then
text = text .. "[" .. id .. "] = "
else
text = text .. "." .. id .. " = "
end
if type(info) == "string" then
add( text .. "'" .. tostring(info) .. "'" .. "\n" )
elseif type(info) == "bool" or type(info) == "number" then
add( text .. tostring(info) .. "\n" )
elseif type(info) == "table" then
add( text .. tostring(table.nestedTableString( info )) .. "\n" )
else
add( text .. tostring(info) .. "\n" )
end
end
return str;
end
function table.nestedTableString( tab )
local str = "{ "
local function add( s )
str = str .. s;
end
for k,v in pairs( tab ) do
if type(v) == "string" then
add( "'" .. tostring(v) .. "'" .. ", " )
elseif type(v) == "bool" or type(v) == "number" then
add( tostring(v) .. ", " )
elseif type(v) == "table" then
add( tostring(table.nestedTableString( v )) .. ", " )
else
add( tostring(v) .. ", " )
end
end
add( "}" )
return str
end
The important part is that it crashes when the "str" is too long.
By the way - you may all use the table functions if you want to.
- Robin
- The Omniscient
- Posts: 6506
- Joined: Fri Feb 20, 2009 4:29 pm
- Location: The Netherlands
- Contact:
Re: Silly strings...
It seems I was right.
The problem is, concatenating strings makes a new copy. Thus a = b .. c has three strings: b, c and b .. c. If you concatenate yet another string, you get a = b .. c .. d. Here you get five strings: b, c, d, b .. c and b .. c .. d. When you have one million strings in a table, you will need to create one million (minus one) new strings. What's worse, making strings doesn't take constant time: the longer the strings get, the more bytes need to be copied. So in the end, that explodes (usually because there is no memory left for the program to use).
The problem is, concatenating strings makes a new copy. Thus a = b .. c has three strings: b, c and b .. c. If you concatenate yet another string, you get a = b .. c .. d. Here you get five strings: b, c, d, b .. c and b .. c .. d. When you have one million strings in a table, you will need to create one million (minus one) new strings. What's worse, making strings doesn't take constant time: the longer the strings get, the more bytes need to be copied. So in the end, that explodes (usually because there is no memory left for the program to use).
Help us help you: attach a .love.
Re: Silly strings...
Ah right! Thanks!
- kikito
- Inner party member
- Posts: 3153
- Joined: Sat Oct 03, 2009 5:22 pm
- Location: Madrid, Spain
- Contact:
Re: Silly strings...
It's much better to use table.concat:
This creates 7 strings in memory: the initial a, b, c & d, plus a .. b, a .. c and finally a .. d. It gets worse with every bit added to a.
Using table.concat, you only create a,b,c,d & the final concatenation, plus a table:
With a big number of strings, tables perform much better, memory-wise.
Code: Select all
a = a .. b
a = a .. c
a = a .. d
return a
Using table.concat, you only create a,b,c,d & the final concatenation, plus a table:
Code: Select all
local buffer = {}
table.insert(buffer, a)
table.insert(buffer, b)
table.insert(buffer, c)
table.insert(buffer, d)
return table.concat(buffer)
When I write def I mean function.
- tentus
- Inner party member
- Posts: 1060
- Joined: Sun Oct 31, 2010 7:56 pm
- Location: Appalachia
- Contact:
Re: Silly strings...
Whuh... WHY would it be set up this way? Whats the advantage?Robin wrote:It seems I was right.
The problem is, concatenating strings makes a new copy. Thus a = b .. c has three strings: b, c and b .. c. If you concatenate yet another string, you get a = b .. c .. d. Here you get five strings: b, c, d, b .. c and b .. c .. d. When you have one million strings in a table, you will need to create one million (minus one) new strings. What's worse, making strings doesn't take constant time: the longer the strings get, the more bytes need to be copied. So in the end, that explodes (usually because there is no memory left for the program to use).
Kurosuke needs beta testers
- bartbes
- Sex machine
- Posts: 4946
- Joined: Fri Aug 29, 2008 10:35 am
- Location: The Netherlands
- Contact:
Re: Silly strings...
Advantage? That's just the way it is done. Note that this is because of the way C strings work.
EDIT: Oh right, as kikito's link says, the string objects stay around until the next garbage collection cycle, so yeah, that too.
As this is not a single operation, a .. b .. c is actually a .. (b .. c) (the order I don't know exactly, but you get the point), and each has a result that is then discarded and then there's a wait for the garbage collector, it takes a lot of memory for a short period of time. Due to the way the GC works, in such a loop, it probably won't kick in, or at least not often enough.
EDIT: Oh right, as kikito's link says, the string objects stay around until the next garbage collection cycle, so yeah, that too.
As this is not a single operation, a .. b .. c is actually a .. (b .. c) (the order I don't know exactly, but you get the point), and each has a result that is then discarded and then there's a wait for the garbage collector, it takes a lot of memory for a short period of time. Due to the way the GC works, in such a loop, it probably won't kick in, or at least not often enough.
- kikito
- Inner party member
- Posts: 3153
- Joined: Sat Oct 03, 2009 5:22 pm
- Location: Madrid, Spain
- Contact:
Re: Silly strings...
When I write def I mean function.
- Robin
- The Omniscient
- Posts: 6506
- Joined: Fri Feb 20, 2009 4:29 pm
- Location: The Netherlands
- Contact:
Re: Silly strings...
You're right.bartbes wrote:As this is not a single operation, a .. b .. c is actually a .. (b .. c) (the order I don't know exactly, but you get the point)
Which actually makes it slightly better for repeatedly adding short strings to long strings, but it's still awful of course.Lua 5.1 Reference Manual wrote:The concatenation ('..') and exponentiation ('^') operators are right associative.
Thing is, the options are really this or mutable strings. And mutable strings just don't work so well in languages like Lua, since it probably means you'll be changing the same string in other places unexpectedly as well.tentus wrote:Whuh... WHY would it be set up this way? Whats the advantage?
Help us help you: attach a .love.
Who is online
Users browsing this forum: No registered users and 15 guests