Enumerate files outside of root / save directory, recursively

[Marty]

I'd like to enumerate all files within a directory, recursively. Those files are not located in the root folder of the game or the save directory.

I've learnt this can be done in traditional lua using LFS, but it seems to be quite challenging to get it to work in LÖVE. Is there an easier way?

[pgimeno]

I've found this but I don't know how well it is supported across platforms: https://github.com/spacewander/luafilesystem

I don't think there's a built-in way to do it with Lua.

[zorg]

For one, you could drag an arbitrary folder onto your project, and mount it; then you could enumerate its contents as well as its subdirectories';

Alternatively, "Hack" PhysFS through luaJIT's FFI to mount your whole filesystem, and use love.filesystem.enumerate.

[raidho36]

Short answer is "you can't". LÖVE is designed as a video game framework and video games don't need to access files outside root and save directories. It's done this way partly to stop novice developers from polluting gamers' filesystems, inadvertently or out of disregarding good practices. Especially on Mac, iOS, Linux and Android, which have strongly defined system file hierarchy, as opposed to Windows which only cares that you don't put garbage in Windows and Program Files folders but otherwise allows any arbitrary file structure.

As others said, if you hook up to external libraries you can bypass this limitation. But you really shouldn't, because as mentioned above, you will probably screw it up for someone's OS.

[grump]

You can use LuaJITs ffi API to access the filesystem functions of the OS, if you know a bit about C programming.

This one kind of works, both for Windows and POSIX systems, although it's not very good code. It works for simple use cases. There's also at least 2 LFS implementations in pure ffi code (no external lib required) on github, but both had several issues when I tried them, including crashes on one OS or the other.

Here's a quick test how zorg's idea works to mount the root directory. I don't know about the intricacies of PhysFS - I just tested this quickly in Linux and it seems to work fine.

Code: Select all

local ffi = require('ffi')
ffi.cdef [[
	int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]

ffi.C.PHYSFS_mount('/', 'root', 1)
local items = love.filesystem.getDirectoryItems('root')

[zorg]

Here's a quick test how zorg's idea works to mount the root directory. I don't know about the intricacies of PhysFS - I just tested this quickly in Linux and it seems to work fine.

Code: Select all

local ffi = require('ffi')
ffi.cdef [[
	int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]

ffi.C.PHYSFS_mount('/', 'root', 1)
local items = love.filesystem.getDirectoryItems('root')

Note to myself: Finish the darned thing and release it as a library

[Marty]

But you really shouldn't, because as mentioned above, you will probably screw it up for someone's OS.

This is not for a game and will be used only internally.

For one, you could drag an arbitrary folder onto your project, and mount it; then you could enumerate its contents as well as its subdirectories';

Here's a quick test how zorg's idea works to mount the root directory. I don't know about the intricacies of PhysFS - I just tested this quickly in Linux and it seems to work fine.

Thank you very much. I'm trying this for test purposes on Windows 10 now:

Code: Select all

local ffi = require('ffi')
ffi.cdef [[
	int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]

function love.load(args)
	ffi.C.PHYSFS_mount("C:\\Windows", "root", 1)
	local items = love.filesystem.getDirectoryItems("root")
	print(table.concat(items, ", "))
end

It throws an error: "Cannot resolve symbol 'PHYSFS_mount'. The specified procedure could not be find.[]

What can I do to make this work on Windows?

[raidho36]

Try loading PhysFS dll via FFI.

[grump]

love.dll has PhysFS statically linked. I don't know why it doesn't find the export via ffi.C on Windows.



I'm not sure about possible bad side effects, and there might be a better way to do this, but this works for me:

Code: Select all

local ffi = require('ffi')
local l = ffi.os == 'Windows' and ffi.load('love') or ffi.C

ffi.cdef [[
	int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]

l.PHYSFS_mount('C:\\', 'root', 1)
print(table.concat(love.filesystem.getDirectoryItems('root'), '\n'))

[Marty]

I'm not sure about possible bad side effects, and there might be a better way to do this, but this works for me:

Code: Select all

local ffi = require('ffi')
local l = ffi.os == 'Windows' and ffi.load('love') or ffi.C

ffi.cdef [[
	int PHYSFS_mount(const char *newDir, const char *mountPoint, int appendToPath);
]]

l.PHYSFS_mount('C:\\', 'root', 1)
print(table.concat(love.filesystem.getDirectoryItems('root'), '\n'))

Yes, this works for me, ffi.load('love') did the trick.

Thank you so much.


EDIT: So here is my working function. I could not use isDirectory, because it was always returning false. I think this is an issue with PhysFS. I solved it by calling recursion in any case and trying to get directory items of files, too.

Code: Select all

function string.ends(s, e)
   return e == "" or string.sub(s, -string.len(e)) == e
end

local function get_files_recursively(dir)
	local result = {}
	if not string.ends(dir, "/") then dir = dir + "/" end
	local files = love.filesystem.getDirectoryItems(dir)
	for i = 1, #files do
		local subFiles = get_files_recursively(dir .. files[i] .. "/")
		local subFilesCount = #subFiles
		for j = 1, #subFiles do
			table.insert(result, subFiles[j])
		end
		if subFilesCount < 1 then
			table.insert(result, dir .. files[i])
		end
	end
	return result
end

Disclaimer: I know it can be improved, but this won't be used in a game anyways.