The Filesystem - A Rant

[yintercept]

Incoming Rant.

Theres been something thats been bothering me for a while now, something that really grinds my gears.
Every time I've gone to develop a side tool I've been forced to work *around* the file system.

I understand this was decided a while back but I can't help but feel in my gut that there should be some sort of toggle or flag to
turn it off.

From what I've seen theres a lot of FUD over read/write access in Love2d.
It makes little sense that we should be restricted in file/directory access because a select few POTENTIAL individuals might misuse said tools.
And I've found with most things in life, the desire to protect users from themselves cripples the usefulness of the thing in question.

It goes to the heart of what love2d is supposed to be. Strictly for Games, or can it be for other things, like Tools? After all, isn't tool development an essential part of the history, and practice of game development?

The sky is not in fact falling, contrary to the doomsayers in all the threads making 'helpful' suggestions about read/write access restrictions 'just in case.'

But from my own experience and what other legitimate users have encountered, all it's done is get in the way..constantly. This is what I've consistently encountered, and isn't even a complete list:

Can't write a shape editor tool that allows opening an external image or directory to import an image,
..because of file system restrictions.

Can't write a level editor tool that allows exporting unless I'm doing copytoclipboard serialization which is arguably worse.
..Because of file system restrictions.

Because love2d loads from main and only main, and because I'm too early in development to put a main menu in,
...I can't include tools IN the game folder itself.


I'm not typically an incendiary individual, but the idea that people will misuse anything you give them is fearmongering by a small vocal minority of people who treat others like children. It's not that people won't, it's that it doesn't matter if they do. A tools job isn't to protect it's users from themselves, least of all when it significantly impedes a core element of development: tool creation.
People will always misuse tools, and while some decisions might make it easier for them to do it, making it *harder* for them is a downward spiral which leads to constantly working on further and further defenses against misuse, all while reducing functionality for the intended users.

Because of this, I and others are left with the following kludges when using love2d:

- make a seperate folder for the tool, copy ALL my assets into the tool before using it. Now anyone I share the
tool with will also have to follow this convoluted step.

- while anyone using tools built with love2d have to copy all their assets in, THEY ALSO can't hit any sort of
simple 'save here' or 'export' button. No, instead they have to

- dig up an obscure save folder

- OR take a serialized string of data which the tool copies into their clipboard, go to their assets folder,
create a new file, name it, open it, copy paste, and then save the result.

- alternatively, put a big ugly early menu in the game to host the tools, resulting in them being tied to
my own work and requiring more work to share or reuse elsewhere.

The reality is love2d isn't big enough to worry about anything worse than script kiddies, and more importantly,
any tool made must necessarily be crippled in order to prevent a select few dumb*sses from misusing it, which
holds everyone else, legitimate users, and development, hostage to a select few individuals misuses.

Because of people crying about 'the sky falling' if outside directory access is enabled' the following things have become unnecessarily complicated to make in love2d:

- GUI editors/builders
- Physics editors/collision shape editors
- Tile terrain/editors (tiled does this one competently)
- Any other sort of tool or editor

These things would help the userbase tremendously and people who might develop them are entirely put off
by, as I said, what is essentially fear mongering. This decision, to restrict directory access may have seemed to make sense in
light of a vocal minority calling for it, however well meaning they may have been, but essentially it went through because
not enough people spoke up to say "this is going to stunt the community and tools development."

For the tl;dr crowd - we need external file access if we want to do anything more serious or build on Love. The decision to restrict access should be reversed.

[slime]

I plan to expand the filesystem API to accommodate more situations in LÖVE 12.

However, it is not simply a decision. There's not a line or few of code I can delete to make everything work as you want. The reason why it works like it does right now is not only to protect end users from LÖVE game developers. As with most things, it's more complicated than it might seem at first glance.

LÖVE uses PhysFS under the hood for most love.filesystem APIs. PhysFS is pretty core to LÖVE because it enables the virtualized filesystem where you can transparently read from a zipped .love or an unzipped source folder, and the game's save directory, with relative filepaths. PhysFS itself has some API restrictions where it can only write to a single directory at a time, and all open files must be closed before that directory can be changed.

If LÖVE gets more location-based file writing capabilities in the future, that restriction will need to be addressed somehow.

On top of that, operating themselves are (for good reason) very sensitive these days to what locations apps are allowed to write to, and even read from, without explicit permission dialogs the user must read and accept. Mobile operating systems have always been like that (for example iOS is completely sandboxed, where apps cannot read from any location outside designated places that the app owns), but many restrictions are in place on modern desktop operating systems, these days.

The most recent macOS version, for example, automatically shows a popup to end-users the first time a given app wants to read from or write to the Desktop folder. Same with the Documents folder, and a host of others. The user may deny permission to the app, and then the app needs to handle that situation gracefully. Apps have limited-to-no write access to most locations – including places where the user might have put the app themselves (so apps do not have guaranteed write access to next to their own executable). The situation on Windows is similar in terms of write access, but not quite so restrictive yet in terms of read access I think.

The one single place that all apps in all operating systems LÖVE supports have permanent guaranteed read and write access to, is the app's own folder in the operating system's equivalent of the AppData directory. I'm really glad that has been the default location of the save directory since forever, otherwise there would be a bunch of headaches related to LÖVE games that don't think about read/write-permissions running into ever increasing OS filesystem security. It has also made porting LÖVE games to different platforms a lot simpler than it otherwise would be.

On the topic of porting, I think it's pretty important to make filesystem APIs work on as many platforms and in as many countries as possible, by default (for example I'd consider it a failure if a user in Russia had more trouble running a LÖVE game or tool than a user in England, because the Russian user has a user folder and other folder names in Cyrillic). Obviously there are situations where full paths are useful, but I'd like to provide good alternatives that should be preferred by default, when possible.

Again, all that being said I plan to expand love.filesystem's access capabilities for LÖVE 12, those are just some of the things I have to think about and write code to deal with while doing so.


I should also mention you can leverage love.system.openURL to open the user's save directory in Explorer (or the operating system's equivalent program) with a button or other UI in your game/app.

[yintercept]

As usual I'm proven wrong by the more cool headed among us!

[ivan]

On Windows, I don't think that Love2D can "protect" the end-user when we already have access to io. os. and ffi... really it's the operating system's job to look after itself.

love.filesystem does a pretty god job at keeping our games cross platform. There are some idiosyncrasies, since the AppData folder is treated like the source directory, leading to stuff like:
love.filesystem.getRealDirectory('myfile.txt')

What if the AppData folder and the game's source directory both contain 'myfile.txt'? I understand that the API is like that for a reason, it's just not as clear and straightforward when compared to something like io.open or LFS.

So yes, yintercept your frustration is totally understandable and you can use io.open to access files from elsewhere. Just be advised that different operating systems could be more strict about your read/write permissions.

[zorg]

I could also reiterate yet again my stance on even windows where i need to do things like run the takeown(ership) command on literally everything (*.*) everywhere because i don't want the OS telling me what i am and am not allowed to delete. Same thing with it specifying one specific folder on one specific file for apps to write to (the user folder, appdata/docs/game saves within it) when in my use case (single-user use-case), i have apps on another drive with a different letter and i'd rather they write into their own folder for convenience's sake; no reason to have them fill up various locations on my C: drive... i think the thing ticking me off specifically is that i don't have the option for what i want, not necessarily that the OS works like this by default... but i digress.

In the case of PhysFS, i've already written a library that may be intrusive, may not work on OS-es (desktop ones, mobile probably doesn't work at all, but haven't tested anything other than windows myself) depending on their version or just permissions of many kinds (filesystem and/or user permissions), but it works for me.

https://github.com/zorggn/love-fml

By default, it separates the save folder from the source base folder, and even mounts all filesystem roots (in the case of windows i mean) it can find into another mountpoint in the virtual filesystem. The downside of this is twofold; firstly, the singular active writefolder is still a problem if you'd want to write to different locations in paralell, since that's not possible; secondly, mounting the real FS roots, i think, means that any update afterwards to the real FS won't be tracked in the virtual FS, hence re-mounting the whole thing would need to be a periodic thing, either manually or automatically (and as slime said, you'd need to close all open handles if you'd have those in those specific virtual "folders")... i might be wrong on the second point there though since the save folder does track modifications as in newly created files and folders. :v

However, the downside of lua's io is that it has issues with specific characters in paths; if i could solve that issue with PhysFS' helper functions (OS-specific converters to and from utf-8, ucs-2, ucs-4, whatever) and the io functions would accept such paths, then i'd say that it would be a golden solution.... because otherwise, not only could you not save files with non-"ascii" names (locale-dependent i mean), but nor could you navigate into such folders.

(Also, i should mention that the reason i don't consider LFS as a solution here is sorely because it would need to be compiled separately.)

There are some idiosyncrasies, since the AppData folder is treated like the source directory, leading to stuff like:
love.filesystem.getRealDirectory('myfile.txt')

What if the AppData folder and the game's source directory both contain 'myfile.txt'? I understand that the API is like that for a reason, it's just not as clear and straightforward when compared to something like io.open or LFS.

By default, the save folder (in appdata on windows) has precedence when you have the same path to a file in both locations, and if you want the source folder to take precedence always, this can be set in conf.lua with setting t.appendidentity to false; however, it is not treated like the source directory, only in the sense that both are mounted in the virtual filesystem to the same place.