Page 1 of 4
Memory Profiling?
Posted: Tue Aug 18, 2020 4:15 pm
by Madrayken
Hi folks,
I'm writing a fairly large RPG (for a Lua project, anyway) and have just started running into a peculiar out-of-memory error far earlier than I expected. As I dive into the reasons for the OOM error, I was thinking it'd be really handy to have a basic 'one number' memory-use stat. Has anyone managed to figure out how to get something like this to work? Is it already somewhere super-obvious that I've missed?
Thanks in advance for any help or advice!
Re: Memory Profiling?
Posted: Tue Aug 18, 2020 4:30 pm
by zorg
Iirc this has been discussed before, unfortunately there's no simple way to count all the main memory used by the application as far as i'm aware.
That said, LuaJIT has a memory limit (from 1-2GB on 64bit to 4GB on 32bit or something like that), whether or not you're intending to use the 32 or 64 bit version of löve. One option for really large mem requirements would be to use 64bit and löve-owned C++-side objects (all the Data objects, like ImageData, SoundData, ByteData - the last one specially if you need misc mem areas for other purposes... you'll need to use the FFI though to write and read from them though.)
I did read about something regarding a luajit fork using a newer garbage collector that makes it possible to use a larger address space but i'm sure it's not included in löve yet... not sure if it can be. 
Re: Memory Profiling?
Posted: Tue Aug 18, 2020 4:53 pm
by Madrayken
Thanks. I feared that might be the case. I've seen external memory profilers (e.g. ones mentioned in
https://www.lua.org/wshop15/Musa2.pdf) but I'd have no idea how to get them to work. For now I'm literally just watching the Activity Monitor app and watching my memory use suddenly jump to 1Gb making the same calls I've been using all over the place for a year. Fun day ahead... ack!
Re: Memory Profiling?
Posted: Tue Aug 18, 2020 10:32 pm
by pgimeno
LuaJIT 2.1 (not a fork, the official one) supports much more memory. Circa 48 bits if my recollection is right. I don't think it's included with Löve yet, though.
Re: Memory Profiling?
Posted: Tue Aug 18, 2020 11:59 pm
by Xii
If you are using Lua tables to store game information, like npc.health=100 or map[pos]=tile_id or whatever, you can significantly reduce your memory consumption by using FFI.
Here is an article detailing how to do that. Using FFI you can define compact memory structures for values that simply do not need the full 64 bits range that regular Lua values have.
Re: Memory Profiling?
Posted: Wed Aug 19, 2020 4:33 am
by grump
Re: Memory Profiling?
Posted: Wed Aug 19, 2020 6:48 am
by ivan
Running out of Lua memory shouldn't happen unless you are doing something terribly wrong.
Make sure you set any unused references to nil otherwise the garbage collector can't release that memory.
Re: Memory Profiling?
Posted: Wed Aug 19, 2020 7:22 pm
by gcmartijn
ivan wrote: ↑Wed Aug 19, 2020 6:48 am
Make sure you set any unused references to nil otherwise the garbage collector can't release that memory.
I need to test this, but what about something like this.
Code: Select all
function add (data)
local rawdata = data
--- do things
return something
end
local x = add({bla})
Do we always need to set variable to nil for the garbage collector ?
Code: Select all
function add (data)
local rawdata = data
--- do things
rawdata = nil
data = nil -- the function to ?
return something
end
local x = add({bla})
Re: Memory Profiling?
Posted: Wed Aug 19, 2020 9:55 pm
by Jeeper
zorg wrote: ↑Tue Aug 18, 2020 4:30 pm
Iirc this has been discussed before, unfortunately there's no simple way to count all the main memory used by the application as far as i'm aware.
That said, LuaJIT has a memory limit (from 1-2GB on 64bit to 4GB on 32bit or something like that), whether or not you're intending to use the 32 or 64 bit version of löve. One option for really large mem requirements would be to use 64bit and löve-owned C++-side objects (all the Data objects, like ImageData, SoundData, ByteData - the last one specially if you need misc mem areas for other purposes... you'll need to use the FFI though to write and read from them though.)
I did read about something regarding a luajit fork using a newer garbage collector that makes it possible to use a larger address space but i'm sure it's not included in löve yet... not sure if it can be.
Did you miss type or does the 32bit version have support for more memory? That would make no sense?
Re: Memory Profiling?
Posted: Wed Aug 19, 2020 11:08 pm
by pgimeno
gcmartijn wrote: ↑Wed Aug 19, 2020 7:22 pm
Do we always need to set variable to nil for the garbage collector ?
No. Going out of scope is equivalent to setting it to nil.
Sometimes, Löve objects take a long time to be cleaned up by the GC, letting many of them to accumulate and take a lot of memory. In these cases, it's best to explicitly release them with
Object:release. I wonder if that's what's happening here.
Jeeper wrote: ↑Wed Aug 19, 2020 9:55 pm
Did you miss type or does the 32bit version have support for more memory? That would make no sense?
The 1 GB limit in 64 bits applies to Linux.
https://stackoverflow.com/questions/351 ... -platforms
http://timetobleed.com/digging-out-the- ... egression/