Possible problems with locales

[leiradel]

Lua scripts are meant to be distributed in source code format instead of compiled for reasons already discussed in other threads. Just to bring everyone reading this to the same page, the reason is that compiled scripts are not cross-platform.

One problem though is that Lua is also not source code compatible between platforms. It has three sources of problems in this regard because of the use of locale-aware functions:

1. ctype, to check if a character is a letter.
2. sprintf, to convert numbers to strings.
3. strtod, to convert strings to numbers.

The following code showcases the problems:

Code: Select all

os.setlocale('fra_fra')
print(1.5)             -- prints 1,5 (notice the comma)
print(tonumber('1.5')) -- prints nil because the conversion fails
print('1.5' + 0)       -- crashes with a stack trace because the conversion fails

Starting on 5.2, Lua will stop using ctype so the first problem won't be a problem for long. But the other two will remain. For the less experienced people, if you read a config file from disk where numbers use the dot as the decimal separator and use tonumber to convert them to numbers, they will all come as nil depending on the locale.

Although I never faced a problem caused by locale, I can't help but think a player of my game could face problems because his OS has a locale where the decimal separator is a comma.

So the questions are: Have you ever faced a locale-based problem with Lua/LÖVE? What the above code outputs on your system if you remove the os.setlocale line?

Cheers,

Andre

[bartbes]

Here it just seems to ignore the locale, I get normal operation.

[leiradel]

Here it just seems to ignore the locale, I get normal operation.

Good. What is your OS?

[bartbes]

Ubuntu 10.04 x86.

[Robin]

Here no problems either. (Kubuntu 10.04 64-bit)

[vrld]

Here it just seems to ignore the locale, I get normal operation.

Maybe you are missing the locale fra_fra.
My computer ignored this locale, too, but worked (bluescreen, that is) with a different locale.

[kikito]

Maybe I'm missing something here.

This code:

Code: Select all

os.setlocale('fra_fra')
print(1.5)             -- prints 1,5 (notice the comma)
print(tonumber('1.5')) -- prints nil because the conversion fails
print('1.5' + 0)       -- crashes with a stack trace because the conversion fails

Should (IMHO) fail on all environments. Are you trying to say that it works on some systems and not in others?

[vrld]

Should (IMHO) fail on all environments. Are you trying to say that it works on some systems and not in others?

No. If you are missing a locale, the standard "C"-locale is chosen instead, which does not fail.
I have the following locales on my system (Gentoo Linux):

Code: Select all

% locale -a
C
POSIX
de_DE
de_DE.iso88591
de_DE.iso885915@euro
de_DE@euro
deutsch
en_US
en_US.iso88591
en_US.utf8
german

The code prints three times "1.5", but if I set the locale to de_DE, then it prints "1,5", "nil" and then crashes.

[kikito]

Hmm. I see.

So the program 'explodes or runs fine' depending on the locales you have installed.

I think it is not really a problem with locales. It's a problem with the code using the locales.

The program should not make the assumption that a locale has been successfully changed after calling os.setlocale. First, it should check that the change was done successfully. I've found out that the locale is 'returned' by os.setlocale if the first argument is nil.

So in your case you would have to do something like this:

Code: Select all

os.setlocale('fra_fra')
if os.setlocale() == 'fra_fra' then
  -- doing fra_fra-locale related stuff is safe here, the locale is installed
else
  -- fra_fra is not installed so you have to fallback to the 'default'.
end 

[leiradel]

So in your case you would have to do something like this: <snip>

I'd rather avoid doing things depending on the locale. What I'm trying to find out is if the code without the os.setlocale fails on someone's system, which I think it should if the default locale is one that sets the decimal separator to comma.

Once we find a crash I want to know if setting the locale to the standard C locale on that system fixes the problem so that adding os.selocale('C') on love.load will make everything work as expected.

Thanks everyone for your answers so far though I still hope someone will try it and get a crash...

Cheers,

Andre

EDIT: Just to make sure everyone understands what code should be run, it's this one:

Code: Select all

print(1.5)             -- prints 1,5 (notice the comma)
print(tonumber('1.5')) -- prints nil because the conversion fails
print('1.5' + 0)       -- crashes with a stack trace because the conversion fails

Please answer with the results, OS and the system locale. Thanks!