I18N (name pending)

[Elvashi]

here's a copy of the text/language management code from DubSim*, for whatever it may be worth. I don't suggest anyone use this as-is, but its served well enough in DubSim, and my translator hasn't complained about it too much, so I must be doing something right.

Of particular note, it supports genders for both speaker and subject, of 4 possibilities (male, female, both (used for unknown), and neuter/none). It also has a rather nifty token replacement system, which isn't just useful into itself, but means that translators can rather freely rearrange the sentence as they wish (which isn't really possible with otherwise-equivalent format-string approaches).

* I really have to get back to DubSim, maybe someday I'll even finish it...

[leiradel]

I've worked as the coordinator of a team which had to do small translations from time to time. There translations were to localize new pages of the corporate website to the languages of the many countries where we were present. The English content was sent in an Excel spreadsheet, one term or phrase per line in the first column, and we had to write the corresponding Brazilian Portuguese translation into another column.

So far so good, but there were terms and small phrases that were very difficult to translate because of lack of context. For example, the word "Ok" doesn't exist in Portuguese, and we don't have "catch all" word with that meaning. So "Ok" would be translated sometimes to "Fechar" (close in Portuguese), in other times to "Sim" (yes) and so on, depending on the context. But we had to write or call someone to ask what the context was.

I know almost nothing of gettext but it seems from the links in this thread that it supports contexts which from my experience are very important during the translation. If that's true, +1 to gettext format. Another +1 to gettext for the GUI tools that handle gettext format.

But there are three more things I'd love to have in an i18n library. One is the ability to localize any type of asset, not just text. For example, I may choose to use images for the menu options because I use an effect on the text that the font renderer does not provide. Another example are sound files with speech.

The second thing is the way the text is actually localized in the code. Although it's very clear what i18n.getTranslation'mainmenu.options' does, I'd rather not have to type all those characters wherever I need localized strings, though it can be done with just a T=i18n.getTranslation. I also would like to have the "core" language text instead of some "message id" because it makes it easier to visualize what you're doing, i.e.

Code: Select all

doMenu{ T'Start Game', T'Options', T'Quit' }

If more information is needed to get the correct translation (as for the "Ok" example), that could just be part of the string:

Code: Select all

T'nextlevel:Ok' -- translates to Continuar in pt or Ok in en.

The last thing is to use language codes with both the country and the language, i.e. pt_BR, en_US etc. During my time in the position described above, I had to fight the leads because they just couldn't understand that the languages spoken in Brazil and in Portugal are dissimilar in a number of ways, and I'm not talking about just slang. For example, screen is tela in Brazilian Portuguese and ecrã in, uh, Portuguese Portuguese :-P

Cheers,

Andre

[kikito]

I'm ok with everything except 3 things.

First, this is not valid lua:

Code: Select all

T'blah'

But this is:

Code: Select all

'bla':t

Let's use that second one, or similar, as a shortcut.

Second: I think .po files don't have a context. They have a 'comments fields', but it is not mandatory. Or that's what I gathered after a 3-minutes read of the specs.

That's another reason why I was suggesting using a .lua file; nested tables provide a hierarchy very naturally.

Third: the 'message id' thing. is actually a good place for getting context from. Instead of the string 'ok' you get 'btn_label_confirm_deletion'.

Moreover, forcing you to put 'message ids' instead of English text also forces the creation of a 'canonical file' - the en.po/en.lua one. And that is very good.

So I say no to .po (again), or at least postpone until after a lua-based one is done, and I say yes to message ids instead of plain English - although I believe that it is possible to implement something that does both.

[leiradel]

First, this is not valid lua:

Code: Select all

T'blah'

This is a function call to T with a string as the only parameter. Similarly you can call a function with a table as the only parameter with

Code: Select all

func{ x = 1, y = 2 }

Cheers,

Andre

[bartbes]

First, this is not valid lua:

Code: Select all

T'blah'

But this is:

Code: Select all

'bla':t

Interestingly the former is valid lua, and the latter isn't.

[kikito]

Wow.

I didn't know about the foo'String' syntax. I'm freaked out. I've learnt something new today!

Regarding the :t thing, you are right; without parenthesis it doesn't work. Another misstep provoked by my Ruby alter-ego.

I'm not a big fan of having a global object called t (or T) because t is a very common variable name in lua, usually used on table-manipulations. If we are going the foo'string' route, I'd rather call the function l (or L), as in 'localize'.

Edit: Actually, I believe you can achieve a valid "foo".t pseudocall with string.__index. But that would be hacky and non-standard.

[bartbes]

I added a poll, you have 7 days to vote.

[leiradel]

First, this is not valid lua:

Code: Select all

T'blah'

But this is:

Code: Select all

'bla':t

Interestingly the former is valid lua, and the latter isn't.

This can be made to work by changing the string metatable:

Code: Select all

('bla').T

But the required parenthesis make it look ugly.

[leiradel]

<snip>

Second: I think .po files don't have a context. They have a 'comments fields', but it is not mandatory. Or that's what I gathered after a 3-minutes read of the specs.

-1 to gettext.

That's another reason why I was suggesting using a .lua file; nested tables provide a hierarchy very naturally.

I don't believe po files should be used directly by a Lua i18n library. They should be read from the file system and used to build the Lua table(s) that will be used by i18n.getTranslation() so we can have the best of both worlds.

Third: the 'message id' thing. is actually a good place for getting context from. Instead of the string 'ok' you get 'btn_label_confirm_deletion'.

Moreover, forcing you to put 'message ids' instead of English text also forces the creation of a 'canonical file' - the en.po/en.lua one. And that is very good.

I believe we can use anything for the message ids so using btn_label_confirm_deletion or delete_dialog:Delete (or anything else) will be just a matter of taste.

Cheers,

Andre

[leiradel]

I'm not a big fan of having a global object called t (or T) because t is a very common variable name in lua, usually used on table-manipulations. If we are going the foo'string' route, I'd rather call the function l (or L), as in 'localize'.

Actually it will also be a matter of taste. Assuming you call i18n.getTranslation to, well, get a localized string, all you have to do is

Code: Select all

L = i18n.getTranslation