Wrapper / abstraction layer mechanism that download needed files on demand

[glitchapp]

Hello, I'm searching for a solution for big projects like some of the one I created to download the files on demand as the player progress through the game.

Someone advised me to implement an abstraction layer / wrapper to accomplish this, I have however no idea where to start and wonder if anyone has attempted that in love2d.

I'll be happy to receive feedback regarding this problem.

Thanks!

[Xugro]

The concept you are searching for is called lazy loading. Here is a sketch in pseudo code:

Code: Select all

function readFile(filename)
    if love.filesystem.getInfo(filename) == nil then
        -- download file
    end

    return love.filesystem.read(filename)
end

This will check if the file is already downloaded. If not it will download the file. Afterwards the file is read.

[MrFariator]

Of note, you may also want to do any such lazy loading on a separate thread, instead of blocking the main thread for N amount of time due to the downloads.

[glitchapp]

The concept you are searching for is called lazy loading. Here is a sketch in pseudo code:

Code: Select all

function readFile(filename)
    if love.filesystem.getInfo(filename) == nil then
        -- download file
    end

    return love.filesystem.read(filename)
end

This will check if the file is already downloaded. If not it will download the file. Afterwards the file is read.

Thanks, I didn't expect it to be that simple. I wonder if you can use that function to download the files from a git repository or it needs to be an html server.

Thanks for the info!

[glitchapp]

Of note, you may also want to do any such lazy loading on a separate thread, instead of blocking the main thread for N amount of time due to the downloads.

That's a good idea but I will be happy if I manage to implement it no matter if it stops the main thread. Once done I will see how can I improve it and maybe attempt to add the loading process to a secondary thread.

[dusoft]

Thanks, I didn't expect it to be that simple. I wonder if you can use that function to download the files from a git repository or it needs to be an html server.

Thanks for the info!

Surely you can download from anywhere, however for Git repo you would like to have some kind of API access to find and fetch the latest version.

[Andlac028]

Thanks, I didn't expect it to be that simple. I wonder if you can use that function to download the files from a git repository or it needs to be an html server.

If you can show raw content of file, you can basically download it over http(s). For example, in github, if you click “Get raw”, you her redirected to url, where you can view raw file, so you can just copy that link and download it from there. Just make sure, you try to download latest version, so that you do not have to change url. Or if you like versioning, you can somehow fetch your desired version and then fetch file based on tag.

[ivan]

I have to admit that fetching files from Github is a pretty cool idea.
There are security implications and HTTPS is not supported out of the box by Love2D.
Also, downloading files could be done without blocking the main thread using luasocket with settimeout(0)
So yes, it can be done but doing it the "proper" way is not simple.
Also you cannot easify replace any of your game's Lua files especially if it is packaged or fused.

[Andlac028]

There are security implications and HTTPS is not supported out of the box by Love2D.

Actually, it is now not hard to use https, as it will be implemented in 12.0 via lua-https and you can also use prebuild library from here for older versions (just put it in folder with your application and require it)

Also for files, I think that you can store them in some data directory (writable by love.filesystem.write), correctly set append path in conf, so it will override files from source (not sure if applicable also for main.lua and conf.lua)

[glitchapp]

There are security implications and HTTPS is not supported out of the box by Love2D.

Actually, it is now not hard to use https, as it will be implemented in 12.0 via lua-https and you can also use prebuild library from here for older versions (just put it in folder with your application and require it)

Also for files, I think that you can store them in some data directory (writable by love.filesystem.write), correctly set append path in conf, so it will override files from source (not sure if applicable also for main.lua and conf.lua)

I agree that security is important, however, first I need to make it work and then will take care of using https or make it work on a different thread.

Further information about this issue is here: https://codeberg.org/glitchapp/fish-fil ... e/issues/6

The size of the assets which reached a peak of 2,4gb has been reduced thanks to the implementation of a webp library and a good compromise in the audio quality and now is around 600mb. 600mb is still too big and a barrief for many people and this project would benefit a lot from a lazyloading mechanism since many people won't beat most of the levels or would only test the game.

The lazyloading.lua file will be added on the next update, any contributions are welcome.