Garbage going crazy

[grump]

When you run this love file, it will display the current garbage collection counter. It goes up pretty fast.

The program does basically nothing, just counting a few variables up and down, and it calls a bunch of empty functions. The only thing that should produce garbage is the display of the garbage counter.

If you press space, the garbage stops piling up. The difference is this:

Code: Select all

-- lots of garbage
updateStepper2()
updateStepper1()

-- no garbage
updateStepper1()
updateStepper2()

The code looks a bit messy because it's taken from a larger project and reduced to a minimal working example. The real project accumulates many more megabytes of garbage per second, for no apparent reason.

You can also make it stop by removing this part from the code:

Code: Select all

ops[0] = function() end
ops[1] = function() end
ops[2] = function() end
...

But this array of functions is an important part of the project and can't really be changed to something else.

Maybe someone here can tell me what LuaJIT's problem with this is. I'm out of ideas.

[pgimeno]

Without much analysis, I think what you see is that the short loop (1..10) is stopping the trace. Unrolling it solves the issue.

In interpreted mode, FFI sucks. If you add jit.off() at the beginning, you'll see the memory grow with either test case.

This crashes in just a few iterations:

Code: Select all

collectgarbage("stop")

jit.off()

repeat
  local sum = 1LL
  for i = 1, 1000000 do
    sum = sum + 1LL
  end
  print(collectgarbage("count"))
until false

So does this:

Code: Select all

collectgarbage("stop")

repeat
  local sum = 1LL
  for i = 1, 1000000 do
    sum = sum + 1LL
    for j = 1, 10 do
      sum = sum + 2LL
    end
  end
  print(collectgarbage("count"))
until false

[grump]

Without much analysis, I think what you see is that the short loop (1..10) is stopping the trace. Unrolling it solves the issue.

Thanks for checking.

I have these two big components in my project that rely heavily on ffi types, and they both perform well on their own. But as soon as both of them are updated, performance tanks - instead of .5 ms per update, they now require up to 20 ms each, with huge fluctuations and lots of GC activity. It runs okay for a short while, then gets slower and slower until it becomes unusable.

Since you mentioned loop unrolling I tried this:

Code: Select all

jit.opt.start(3, '-loop', 'maxtrace=5000', 'hotloop=100')

Things are MUCH better with these options, framerate stabilised again and memory comsumption stays low. Thanks a lot!

I should try to get a trace log but it's a bit difficult when you use LÖVE.

[pgimeno]

Nice fix.

I should try to get a trace log but it's a bit difficult when you use LÖVE.

Try with: require 'jit.dump'.on() and .off()

[grump]

Nice fix.

It fixed the garbage problem, but there's still huge performance fluctuations between runs. Most of the time it takes a constant 2 ms per update, other times it takes 4-6 ms. Once in a while it only takes .4 ms. It decides how fast it wants to run at every start and sticks to its decision until the next start. I asked in a Lua(JIT) community about these fluctuations that I'm seeing all the time, but the consensus was that I'm just stupid and delusional because it can't happen.

Try with: require 'jit.dump'.on() and .off()

That doesn't work for me.

Code: Select all

Error: [string "boot.lua"]:530: conf.lua:2: module 'jit.dump' not found:

I also tried jit.dump without the require. No dice. No google results either.

[togFox]

I love it when you present evidence to a community and they're calling you stupid and delusional.

[zorg]

It fixed the garbage problem, but there's still huge performance fluctuations between runs. Most of the time it takes a constant 2 ms per update, other times it takes 4-6 ms. Once in a while it only takes .4 ms. It decides how fast it wants to run at every start and sticks to its decision until the next start. I asked in a Lua(JIT) community about these fluctuations that I'm seeing all the time, but the consensus was that I'm just stupid and delusional because it can't happen.

Sounds like the "magical" issues i had with FFI myself, with some audio modules i coded that ran with speeds varying like that... doing casting shenanigans between integers and floats with unions...

Try with: require 'jit.dump'.on() and .off()

That doesn't work for me.

Code: Select all

Error: [string "boot.lua"]:530: conf.lua:2: module 'jit.dump' not found:

I also tried jit.dump without the require. No dice. No google results either.

I think i got auah to give me some precompiled extra luajit stuff earlier that löve didn't have that i could just put into my project and what allowed me to do some debug stuff live; i can try to find it tomorrow when i'm not at work, if you're interested.

[grump]

Sounds like the "magical" issues i had with FFI myself, with some audio modules i coded that ran with speeds varying like that... doing casting shenanigans between integers and floats with unions...

I see it all the time. The more you deviate from standard Lua code, the more it becomes apparent. But it also happens with innocuous Lua code. There is definitely some random element to LJ's heuristics.

I think i got auah to give me some precompiled extra luajit stuff earlier that löve didn't have that i could just put into my project and what allowed me to do some debug stuff live; i can try to find it tomorrow when i'm not at work, if you're interested.

Please do.

Edit: oh, precompiled... if that means binaries compiled for Windows then I can't use it.

[pgimeno]

I asked in a Lua(JIT) community about these fluctuations that I'm seeing all the time, but the consensus was that I'm just stupid and delusional because it can't happen.

FWIW you're not the first one to run into these issues. https://love2d.org/forums/viewtopic.php?f=4&t=88029

Try with: require 'jit.dump'.on() and .off()

That doesn't work for me.

Oops. Maybe it does for me because I have LuaJIT installed system-wide. You need the jit library from the LuaJIT sources. https://github.com/LuaJIT/LuaJIT/tree/v2.0/src/jit (or v2.1 depending on your version)

[grump]

Oops. Maybe it does for me because I have LuaJIT installed system-wide.

Hm, me too. I'll look into it later, thank you.

Just to add a little bit more weirdness:

Code: Select all

local inRange = self.y - y < self.spriteSize
if not inRange then return end

Speed before adding these two lines: 500+ fps
Speed after adding these two lines: 30-40 fps

Because I know LuaJIT is weird, I changed it to this:

Code: Select all

local inRange = self.y - y < self.spriteSize
for i = 1, 100 do
    if not inRange then return end
end

Speed with this loop instead of returning immediately: 500+ fps.

This crap is killing me, lol.