Bundling large libs?

[bs4h]

Hi all,

couldn't figure out how to get Love to love bigger libraries in subdirectores.

The one in question is LuaJSON (https://github.com/harningt/luajson). I want to keep external dependencies away from my own code, so I'd rather not put random stuff in the package's root folder. The problem pops up when https://github.com/harningt/luajson/blo ... a/json.lua asks to require("json.decode"), since I keep the package's json.lua in my own lib/json.lua and require("lib/json").

I've learned that Love's custom loaders ignore package.path.
I absolutely don't want to patch upstream libraries - this doesn't scale well when adding more dependencies or updating.
I'd love to avoid crude hacks, but I'm OK to hack up another custom loader if that'd be the best solution.

Thanks!

[bekey]

-snip-

[bs4h]

There:

Code: Select all

% git clone --depth 1 https://github.com/harningt/luajson.git                 
Cloning into 'luajson'...
[snip]
% cd luajson
% PREFIX=../lib make install  
mkdir -p ../lib/share/lua/5.1
cp -p -r lua/* ../lib/share/lua/5.1
% cd ../lib
% mv share/lua/5.1/* . 
% rm -rf share 
% ls
json  json.lua
% cd ..
% head -1 main.lua
require "lib/json"
% love .
Error: lib/json.lua:5: module 'json.decode' not found:
        no file "json/decode.lua" in LOVE game directories.

        no extension "json.decode" in LOVE paths.
[snip]

[bartbes]

Yeah, until we figure out a good solution for paths (issue #201), the best solution is probably writing a simple loader yourself.

[kikito]

A simple solution for this particular case would be using a different json library. If Lua is not short of one thing, it's of json parsing libraries.

In my work, I spent some time testing different libraries to see which one would suit our needs best. Luajson came up first, but only because we already depended on luarocks and could handle the dependencies with it. That is not the case in LÖVE. As you mentioned, it is = made up of many files, which as you say, cause trouble. I don't believe that is a LÖVE issue, but one of Lua's weak points: you either employ one ugly regular expression trick at the beginning of each file of the lib or in practice you make your library only work from one load point (if you can't use package.path).

In the particular case of Luajson, there is another problem. It depends on lpeg, which is a binary lib. This lib would have to be compiled and bundled differently for all platforms.

json4lua looks like a good candidate. It is a single file with no dependencies. It's pure Lua, so its speed will not match luajson or cjson, but now that we jave luajit in LÖVE the difference shouldn't be astronomical (disclaimer: I have not made any tests myself). But your game would remain cross-platform and your dependencies easier to manage.

[bs4h]

Thanks a lot kikito, that was the perfect answer,

You're right on lpeg dependency! Didn't pay attention. I've looked at JSON4Lua and it's totally fine. I want to rely on JSON only for loading game data (entity, map definitions, etc), for network I'll probably use a more "packed" format (thinking of protobuf or msgpack, perhaps custom).

bartbes, thanks for re-opening the issue on love.filesystem.path. I'll try the custom loader hack for now.

Cheers!

[szensk]

json4lua looks like a good candidate. It is a single file with no dependencies. It's pure Lua, so its speed will not match luajson or cjson, but now that we jave luajit in LÖVE the difference shouldn't be astronomical (disclaimer: I have not made any tests myself). But your game would remain cross-platform and your dependencies easier to manage.

In my tests json4lua is about 5x-11x slower than cjson, where as dkjson is about 3-5x times slower than cjson. Of course, none of them become fully compiled because heavy use of pairs, string.find, string.match, etc.

Results for a 1e6 mixed element table encode and decode:

Code: Select all

$ luajit test.lua
json4lua completed in 2.26s[10.2727x]
dkjson   completed in 0.9s[4.09091x]
penlight completed in 0.77s[3.5x]
ser      completed in 0.53s[2.40909x]
cjson    completed in 0.22s[1x]