Iteration order

[Sasha264]

Good day!
Recently I have updated one game from Love 11.2 to Love 11.4.
So far, everything has been good, but I did notice one funny thing:

Code: Select all

local temp = {}
temp["stone"] = 1
temp["scrap"] = 1
temp["organics"] = 1
temp["quartz"] = 1
for id,_ in pairs(temp) do
    print(id)
end

In Love 11.2, it always printed the following keys in the same order (not the order of adding, but some constant order):

Code: Select all

stone
quartz
organics
scrap

However, in Love 11.4, the keys are printed in random order, which changes between runs.
I understand that different order of pairs() iteration is expected behavior in Lua and that I shouldn't rely on it.
And I didn't, but it looks like a major change, which can cause some games to behave differently.
Both versions states that the Lua version is 5.1 when I use print(_VERSION). LuaJIT has been updated?
Is this expected behaviour? What changed from Love 11.2 to 11.4 to cause this?

[Sasha264]

It gets more interesting with numbers keys.
In Love 11.2 and in Love 11.4 this iteration order is the same and constant:

Code: Select all

5
6
7
899
58
-56.3
555.555
1e-21
9999

But when I add some strings to this, they start jumping back and forth (in 11.4):

Code: Select all

5
6
7
899
-56.3
58
a cup
some tea
555.555
1e-21
9999

Isn't it iterating in the order of some hash function? Or hash functions changed in 11.4, like using string addresses or something?

[BrotSagtMist]

Yee luajit has an update.
And since tables have per definition no order that is really pointless to dig into.

[zorg]

And I didn't, but it looks like a major change, which can cause some games to behave differently.

Then those games were coded with an incorrect assumption, and such, should be fixed by their developers/maintainers.
Hard to consider this a breaking change when the thing "broken" was never broken in the first place, always working as intended.

Both versions states that the Lua version is 5.1 when I use print(_VERSION). LuaJIT has been updated?
Is this expected behaviour? What changed from Love 11.2 to 11.4 to cause this?

You probably want to check jit.version instead; luajit won't change the shown lua _VERSION string since it's still (mostly) 5.1 and in all probablility, won't ever be different.

[dusoft]

I understand that different order of pairs() iteration is expected behavior in Lua and that I shouldn't rely on it.

Bingo and there you go. You shouldn't rely on it.
Use ipairs, if you do.

[Sasha264]

You probably want to check jit.version instead;

Yes! Thanks for that hint Now I see the change: LuaJIT 2.0.5 >> 2.1.0-beta3

Then those games were coded with an incorrect assumption, and such, should be fixed by their developers/maintainers.

With that I agree, it should be fixed =)
This error just falls into a category "silent error problem" — the error that can not be reproduced or found by any testing (manual or automatic). And only way to find it — look carefully at the code. After carefully read all documentations. Which makes it kind of the hardest type of error to find
For me the problem was in just one place, where I accidentally relied on some order. And it lead to funny visual effects And I found and fixed it in 10 minutes. But what if that was in 100 places? That wouldn't still be so funny

[Sasha264]

Use ipairs, if you do.

That raises an interesting question, btw: is it safe to use pairs to iterate plain tables (with keys 1, 2, 3, ...), and assume that they will be iterated in correct order? Or the only way to make sure of iteration order is to use ipairs? At the moment it works ok in all versions. But what does language standard tells about that?

If some day some lua implementation start to iterate plain tables with pairs in random order — I am sure it will not be a simple problem

[dusoft]

Use ipairs, if you do.

That raises an interesting question, btw: is it safe to use pairs to iterate plain tables (with keys 1, 2, 3, ...), and assume that they will be iterated in correct order? Or the only way to make sure of iteration order is to use ipairs? At the moment it works ok in all versions. But what does language standard tells about that?

I know that it's an older version's manual, but it's one page and contains well described differences for both pairs() and ipairs():
https://www.lua.org/manual/5.4/manual.html

pairs() uses next() and next()'s order is not defined: "The order in which the indices are enumerated is not specified, even for numeric indices. (To traverse a table in numerical order, use a numerical for.)"

[zorg]

Use ipairs, if you do.

That raises an interesting question, btw: is it safe to use pairs to iterate plain tables (with keys 1, 2, 3, ...), and assume that they will be iterated in correct order?

No. You can't assume that at all; pairs is still undefined in terms of order regardless of what keys you have.

Or the only way to make sure of iteration order is to use ipairs? At the moment it works ok in all versions. But what does language standard tells about that?

The language is explicit about ipairs always iterating in order... as long as your table is a sequence, meaning integer keys only, starting from 1, with no gaps... the iterator itself will start at the key [1] and will stop at the first gap, according to the 5.1 reference.

If some day some lua implementation start to iterate plain tables with pairs in random order — I am sure it will not be a simple problem

pairs always does undefined order, regardless of how whatever lua implementation implemented it (in löve's case, juaJIT by default) - you should never, never rely on pairs EVER having any ordering, regardless whether your tests show it being random or consistent.

If you want to assign order to arbitrary keys, use a second table with integer keys in a sequence from 1 that has, as its values, the keys of the other table... alternatively you might be able to write your own iteration function that works for whatever specific bag of keys you want to support; iirc i have seen things that iterated in terms of lexicographical order on strings, or reverse ipairs, or whatever else.

I know that it's an older version's manual, but it's one page and contains well described differences for both pairs() and ipairs():
https://www.lua.org/manual/5.4/manual.html

Regardless of the latest lua release version being 5.4.6 or whatever, realize that it's irrelevant considering löve uses LuaJIT, which is 5.1 with some additions, so the relevant manual link would be this one: https://www.lua.org/manual/5.1/manual.html

pairs() uses next() and next()'s order is not defined: "The order in which the indices are enumerated is not specified, even for numeric indices. (To traverse a table in numerical order, use a numerical for.)"

That or ipairs.

[BrotSagtMist]

But order in pairs/next should be consistent until changes to the table are made.