Page 1 of 3
Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 12:34 am
by Marty
I'd like to enumerate all files within a directory, recursively. Those files are not located in the root folder of the game or the save directory.
I've learnt this can be done in traditional lua using LFS, but it seems to be quite challenging to get it to work in LÖVE. Is there an easier way?
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 12:56 am
by pgimeno
I've found this but I don't know how well it is supported across platforms:
https://github.com/spacewander/luafilesystem
I don't think there's a built-in way to do it with Lua.
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 2:42 am
by zorg
For one, you could drag an arbitrary folder onto your project, and mount it; then you could enumerate its contents as well as its subdirectories';
Alternatively, "Hack" PhysFS through luaJIT's FFI to mount your whole filesystem, and use love.filesystem.enumerate.
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 6:59 am
by raidho36
Short answer is "you can't". LÖVE is designed as a video game framework and video games don't need to access files outside root and save directories. It's done this way partly to stop novice developers from polluting gamers' filesystems, inadvertently or out of disregarding good practices. Especially on Mac, iOS, Linux and Android, which have strongly defined system file hierarchy, as opposed to Windows which only cares that you don't put garbage in Windows and Program Files folders but otherwise allows any arbitrary file structure.
As others said, if you hook up to external libraries you can bypass this limitation. But you really shouldn't, because as mentioned above, you will probably screw it up for someone's OS.
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 9:47 am
by grump
You can use LuaJITs ffi API to access the filesystem functions of the OS, if you know a bit about C programming.
This one
kind of works, both for Windows and POSIX systems, although it's not very good code. It works for simple use cases. There's also at least 2 LFS implementations in pure ffi code (no external lib required) on github, but both had several issues when I tried them, including crashes on one OS or the other.
Here's a quick test how zorg's idea works to mount the root directory. I don't know about the intricacies of PhysFS - I just tested this quickly in Linux and it seems to work fine.
Code: Select all
local ffi = require('ffi')
ffi.cdef [[
int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]
ffi.C.PHYSFS_mount('/', 'root', 1)
local items = love.filesystem.getDirectoryItems('root')
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 1:14 pm
by zorg
grump wrote: ↑Thu Feb 15, 2018 9:47 am
Here's a quick test how zorg's idea works to mount the root directory. I don't know about the intricacies of PhysFS - I just tested this quickly in Linux and it seems to work fine.
Code: Select all
local ffi = require('ffi')
ffi.cdef [[
int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]
ffi.C.PHYSFS_mount('/', 'root', 1)
local items = love.filesystem.getDirectoryItems('root')
Note to myself: Finish the darned thing and release it as a library
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 5:00 pm
by Marty
raidho36 wrote: ↑Thu Feb 15, 2018 6:59 am
But you really shouldn't, because as mentioned above, you will probably screw it up for someone's OS.
This is not for a game and will be used only internally.
zorg wrote: ↑Thu Feb 15, 2018 2:42 am
For one, you could drag an arbitrary folder onto your project, and mount it; then you could enumerate its contents as well as its subdirectories';
grump wrote: ↑Thu Feb 15, 2018 9:47 am
Here's a quick test how zorg's idea works to mount the root directory. I don't know about the intricacies of PhysFS - I just tested this quickly in Linux and it seems to work fine.
Thank you very much. I'm trying this for test purposes on Windows 10 now:
Code: Select all
local ffi = require('ffi')
ffi.cdef [[
int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]
function love.load(args)
ffi.C.PHYSFS_mount("C:\\Windows", "root", 1)
local items = love.filesystem.getDirectoryItems("root")
print(table.concat(items, ", "))
end
It throws an error: "Cannot resolve symbol 'PHYSFS_mount'. The specified procedure could not be find.[]
What can I do to make this work on Windows?
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 5:32 pm
by raidho36
Try loading PhysFS dll via FFI.
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 6:22 pm
by grump
love.dll has PhysFS statically linked. I don't know why it doesn't find the export via ffi.C on Windows.
I'm not sure about possible bad side effects, and there might be a better way to do this, but this works for me:
Code: Select all
local ffi = require('ffi')
local l = ffi.os == 'Windows' and ffi.load('love') or ffi.C
ffi.cdef [[
int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]
l.PHYSFS_mount('C:\\', 'root', 1)
print(table.concat(love.filesystem.getDirectoryItems('root'), '\n'))
Re: Enumerate files outside of root / save directory, recursively
Posted: Thu Feb 15, 2018 7:56 pm
by Marty
grump wrote: ↑Thu Feb 15, 2018 6:22 pm
I'm not sure about possible bad side effects, and there might be a better way to do this, but this works for me:
Code: Select all
local ffi = require('ffi')
local l = ffi.os == 'Windows' and ffi.load('love') or ffi.C
ffi.cdef [[
int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]
l.PHYSFS_mount('C:\\', 'root', 1)
print(table.concat(love.filesystem.getDirectoryItems('root'), '\n'))
Yes, this works for me, ffi.load('love') did the trick.
Thank you so much.
EDIT: So here is my working function. I could not use isDirectory, because it was always returning false. I think this is an issue with PhysFS. I solved it by calling recursion in any case and trying to get directory items of files, too.
Code: Select all
function string.ends(s, e)
return e == "" or string.sub(s, -string.len(e)) == e
end
local function get_files_recursively(dir)
local result = {}
if not string.ends(dir, "/") then dir = dir + "/" end
local files = love.filesystem.getDirectoryItems(dir)
for i = 1, #files do
local subFiles = get_files_recursively(dir .. files[i] .. "/")
local subFilesCount = #subFiles
for j = 1, #subFiles do
table.insert(result, subFiles[j])
end
if subFilesCount < 1 then
table.insert(result, dir .. files[i])
end
end
return result
end
Disclaimer: I know it can be improved, but this won't be used in a game anyways.