JIT seems to be changing my loop behaviour

[broccoliandlua]

Hi. I've been trying to understand the cause of a bug for a couple days, but I'm simply at a loss. It seems non-deterministic. Adding print statements seems to change its behaviour. And disabling the JIT with jit.off() seems to eliminate it completely. Any idea what's going on here?

I have a game scene, s, which is given a function called update_objects. Please note the count variable declared at the start of the loop. This count is incremented for each object in the scene.

Code: Select all

 s.update_objects = function(self, world)
        local physics_region = self.physics_region
        local objects = self.objects
        local count = 0
        for obj, _ in pairs(objects) do
            count = count+1
            obj:update(world)
            engine.apply_physics(obj)
            if physics_region then
                physics_region:update_object(obj)
            end
        end
        print("update count", count)
    end

PROBLEM: sometimes objects in the scene aren't updated and count is different from the number of pairs in objects.

None of the functions obj:update(world), engine.apply_physics(obj), or physics_region:update_object(obj) alter the scene's objects table in any way. There is no way for that table to mutate during the iteration and post-iteration it reports the correct number of pairs. As mentioned, turning off the JIT eliminates the bug entirely with no other code changes.

[Bigfoot71]

Without knowing what the `objects` table looks like and how it is used elsewhere in the code I can't quite say why this is happening. So without more details I can only assume that the table is modified elsewhere or may be an unanticipated event if you use coroutines (this has already happened to me in a very particular case).

After if indeed it works very well without Lua JIT it is true that in rare cases it can sometimes optimize the code in order to modify its behavior. From what I had read Lua JIT can change the order of execution of certain instructions and can also optimize the use of memory by reusing freed spaces, which can also be a problem in specific cases. Try using a Lua JIT profiler or writing the code differently, but having only this snippet unfortunately I can't say much more.

Maybe someone else could correct me or say more.

Edit:

I just noticed it while wanting to test your code, but I believe that you are using the loop with pairs incorrectly, you are trying to call the key and not the value. https://www.lua.org/pil/7.3.html

An example:

Code: Select all

for key, value in pairs(table) do

This code works with Lua JIT for me:

Code: Select all

local objects = {

    obj_1 = function () print(1) end,
    obj_2 = function () print(2) end,
    obj_3 = function () print(3) end,
    obj_4 = function () print(4) end,
    obj_5 = function () print(5) end

}


update_objects = function(self, world)

        local count = 0

        for _, obj in pairs(objects) do
            count = count+1
            obj()
        end

        print("update count", count)

end

while true do
    update_objects()
end

[pgimeno]

As Bigfoot71 said, there isn't enough code to make heads or tails of whether it's a LuaJIT bug, or a bug in your program. But a LuaJIT bug would not be unseen. viewtopic.php?p=248260

You can disable JIT for a specific function, to alleviate the problem.

[broccoliandlua]

Thanks for the replies. The table is a set, so the pairs are simply game objects (tables) mapped to true. That's why I'm using the key and not the value. However, using the value doesn't change anything. I've tried mapping object ids to objects or objects to themselves. I'm not using coroutines anywhere. And, as mentioned, there is nowhere in the code that this table is modified outside of initialization. I can try to create a sample but due to the nature of the bug I think I'd have to give over my entire codebase.

[GVovkiv]

I think I'd have to give over my entire codebase.

It would be sad to drop entire codebase, if bug is just misplaced table or table object.

[Bigfoot71]

What I would do for you, step by step:

• Print the length of the object table at the beginning and end of the update_objects function to see if it changes

• Try adding a second loop after the first that again iterates over the object table and prints the keys. This will help determine if the problem is related to the objects changed during the loop or if it is related to the loop itself

• Add additional print statements to the loop or in the objects to see if you can determine where the problem is occurring.

• If you are able to reproduce the problem consistently, try running the code with a debugger to see if that helps you identify the cause of the problem, otherwise if you can reproduce the problem but cannot find one still not cause post it to us.

I agree with GVovkiv, it wit would be a shame to throw away all the code if it is only a problem in the management of your tables, which looks rather like it until proven otherwise.

I will add that a problem is (almost) always reproducible and it is even often when you try to reproduce it that you realize your mistake, that has been the case for me many times anyway...

Otherwise if the problem comes indeed LuaJIT as you say then do what PGimeno said.

[slime]

While it could be a problem in user code, it's also true that the version of LuaJIT that comes with love 11.4 has some rare bugs with pairs when it's JIT-compiled, unfortunately. If it does end up being a problem with LuaJIT, pgimeno's suggestion to turn JIT compilation off for a specific function is probably the simplest workaround.

Newer versions of LuaJIT have fixed most or all of those issues, and the next release of love will use the latest LuaJIT version.

[broccoliandlua]

What I would do for you, step by step:

• Print the length of the object table at the beginning and end of the update_objects function to see if it changes

• Try adding a second loop after the first that again iterates over the object table and prints the keys. This will help determine if the problem is related to the objects changed during the loop or if it is related to the loop itself

• Add additional print statements to the loop or in the objects to see if you can determine where the problem is occurring.

• If you are able to reproduce the problem consistently, try running the code with a debugger to see if that helps you identify the cause of the problem, otherwise if you can reproduce the problem but cannot find one still not cause post it to us.

I agree with GVovkiv, it wit would be a shame to throw away all the code if it is only a problem in the management of your tables, which looks rather like it until proven otherwise.

I will add that a problem is (almost) always reproducible and it is even often when you try to reproduce it that you realize your mistake, that has been the case for me many times anyway...

Otherwise if the problem comes indeed LuaJIT as you say then do what PGimeno said.

1. The length of the table does not change. It's the same before and after the function.
2. The loop after the first simply prints out the keys as expected.
3. I've been staring at the code for 30 hours easily now. There's simply no way for it to modify that table.
4. The problem is consistent. It happens as soon as the number of objects in the table in sufficient. And vanishes as soon as JIT is turned off.

I'll just continue with JIT off for that function until the next LOVE release. Thanks, everyone!

[Sasha264]

4. The problem is consistent. It happens as soon as the number of objects in the table in sufficient.

Hmmm, interesting... How many, approximately, is sufficient? It is just good to know =)

Since it is update function I would guess that it is called every frame or some other way regularly. So, the question: does the pattern of missed out objects repeat itself in the next iterations?

[broccoliandlua]

4. The problem is consistent. It happens as soon as the number of objects in the table in sufficient.

Hmmm, interesting... How many, approximately, is sufficient? It is just good to know =)

Since it is update function I would guess that it is called every frame or some other way regularly. So, the question: does the pattern of missed out objects repeat itself in the next iterations?

Sorry for the very delayed response.

Sufficient here is some number in the thousands, although larger numbers seem to make the problem worse.

Yes, it is called every frame.

The pattern typically repeats from some time then changes and repeats again. How long it repeats and how often it changes is random.

Just to confirm I have never encountered this issue again since disabling the JIT.