Memory Profiling?

[Sasha264]

LuaJIT's hard memory limit is per process rather than per Lua instance within a process unfortunately, so spreading memory across multiple threads won't raise the limit.

Thank you! Saved some time for me trying that

That being said, most memory use in games tends to be things like textures and other large bits of data, most of which doesn't count towards LuaJIT's Lua-allocated memory limit when you use love objects. There are tons of ways to store large collections of data in memory allocated by love rather than by Lua, and doing so often comes with performance benefits as well.

In games — absolutely! Performance benefits as well, yes!

For example depending on how you structure string concatenation within a loop, it can use a lot more temporary memory than you might expect.

Yes, I solved that problem with something like this. But it was overhead troubles. I just wanted to parse bunch of json files, save them modified in another folder, and that's it, remove that script One time operation, does not care about performance or memory usage, only about simplicity. But it turned out that I must care about memory usage

love 12.

Yay! Great news

[Sasha264]

LuaJIT's hard memory limit is per process rather than per Lua instance within a process unfortunately, so spreading memory across multiple threads won't raise the limit.

Oh, wait! Does this mean that separate love threads gets common lua garbage collection? So if one of my custom threads do a lot of garbage collection it will affect performance of other threads, and most important — main thread, that does graphic work?

[slime]

Garbage collection is local to individual threads / Lua instances, it's just the internal memory address space used by LuaJIT 2.0 that is shared (so in practice the only Lua thing shared is the memory cap).

[Sasha264]

Hm... Interesting, thank you for enlightening, that was counter-intuitive for me =)

[pgimeno]

You are surely mistaken. C structs allocated with ffi.new definitely show up on my collectgarbage("count")

You're right. My recollection was wrong.

For example I tried to parse some heavy json-s with lua on love. (but not so heavy really, just several files about 50 mb in filesize)

Parsing in Lua is best done avoiding generation of substrings, by using string.byte as much as possible. The resulting parser looks really awful, but it's fast and doesn't generate so much garbage.

See for example https://notabug.org/pgimeno/minetest/co ... 0077c9R111 which is a simple parser to escape Lua strings with a special escaping scheme.

It might also help to use an FFI buffer for building strings from parts. But in general, Lua is a horrible language for writing a parser.

[Sasha264]

Looks really as difficult example =)
I used some existing library https://github.com/rxi/json.lua
But if I will need to parse something on my own in Lua, I will remember your advise

[rincewind]

The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/

Hello!
is it same for FreeBSD or just Linux specific?

[pgimeno]

It's Linux specific. I don't know the limit in FreeBSD. In Windows I don't remember if it was 1.5 GB or 2 GB.

[gcmartijn]

Do we always need to set variable to nil for the garbage collector ?

No. Going out of scope is equivalent to setting it to nil.

Sometimes, Löve objects take a long time to be cleaned up by the GC, letting many of them to accumulate and take a lot of memory. In these cases, it's best to explicitly release them with Object:release. I wonder if that's what's happening here.

Did you miss type or does the 32bit version have support for more memory? That would make no sense?

The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/

@pgimeno
I was searching for memory fluctuations inside my code.
And I never set temp tables to nil (and other variables), after you did mention that leaving a function is automaticity (at some point) free up some memory. This make the code more readable (less lines).

But now I did test something, and want to know if this behaviour is oke.

Situation 1: a function create a huge temp table, and the garbage count is now 1024 big.
I did thought that is was not, after your comment.
But is this garbage a problem? at this point ? Or will new object inside the garbage replaces (overwritten), and is in my case 1024 the max garbage I can have.

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

function love.load()
    bla() -- garbage count == 1024 now
end

function love.draw()
    love.graphics.print(
        string.format(
            "Memory: gfx=%.1fMiB lua=%.1fMiB",
            love.graphics.getStats().texturememory / 1024 ^ 2,
            collectgarbage "count" / 1024
        ),10,25)
end

Situation 2: the same, but now I want to manual remove it, it don't work.

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

function love.load()
    bla() -- garbage count == 1024 now
    collectgarbage("collect") -- does nothing
end

Situation 3: works

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
    t = nil -- we don't need it as garbage
end

function love.load()
    bla() -- garbage count == 0.x now
end

I like situation 3 when searching for memory issues or fluctuations in the future.
But will give it other benefits? I have to add multiple lines to my code.

And I don't know why situation 2 did not work, but I don't really use that.
I was thinking about manual garbage collections (turn it off), and only 'collect'/remove the garbage on a frame update (to see what happens).

[pgimeno]

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

`t` does not go out of scope, because it's not local to the function: it's either global or an upvalue. So, it's still in scope when the function ends, therefore it can be read in other functions like love.update or love.draw, so the data must be kept in the variable and can't be cleared by the garbage collector. Only local variables go out of scope when the function ends.

Code: Select all

function bla()
    t = {}
    for i = 1, 100000000 do
        table.insert(t, "hello")
    end
end

function love.load()
    bla()
end

function love.draw()
    love.graphics.print(t[100000000])
        -- prints "hello" because t is visible from here
end

That's yet another case of "globals are evil". Get used to making everything local; it will make the code more readable and prevent some nasty bugs.