LÖVE

I need to save a very large table to file so that I can read it back in later. Bitser doesn't work for very large tables. I have tried to roll my own saving routines based on modified versions of this sort of code, but whether I gradually concatenate the values into a string and then save it to file, or append the values to file one by one, the results are really painfully slow. The ptr-dump module is advertised as doing this kind of job for very large tables, so I think it might be worth a go. The problem now is that I can't work out how to use it. I have successfully used other modules like anim8, and it was just a question of copying the .lua file into the same directory as my main.lua, and using require, but I can't find any version of doing that which works with this module. The documentation suggests that if I have for example done then I should be able to call but I'm being told I'm calling a nill value.

Can someone please explain how to install this module, or explain any other way they know that will work to get a LÖVE program to save and later reload a very large table?

Looking into the source (and docs) it seems to be 'dump.tostring'. Note that the 's' is not capital.
You could also potentially look into executing your code async with threads.

Lua in love should be able to access all installed modules. You can also navigate to modules directory and copy one into your project directory.

That was so brain-dead of me! Thank you so much for spotting my silly mistake!

Sadly, although this now "works", it seems that although this library is much faster than my own code, it also crashes when attempting to serialise very large tables.

If I'm reading correctly, your system is basically saving a set of scripted draw calls, correct? You'd probably be able to save a ton of space by just serializing user input instead of all of the love.draw() information and then test it a few times to make sure the replays come out deterministically. Your game already knows how to do all the drawing instructions on its own, so you're saving a ton of unnecessary data.

As an example, Doom's demo format (https://doomwiki.org/wiki/Demo) comes out to about 140 bytes per second, or about 8KB a minute. At that format you'd hit about 2 hours of gameplay or so before you even have 1MB of data to work with.

If your game has any kind of random behaviors, you can serialize and store them in a table for later use with love.math.getRandomSeed() : https://love2d.org/wiki/love.math.getRandomSeed

You might still run into issues like you're seeing right now if your data tables get really humongous, but it might be an easier fix overall to work with less data for your replays rather than chase down a fix for handling the bigger data set. Even when the set size gets too big, you could probably find the crash point and split the table up before it becomes a problem.

I'm running into some similar issues with gigantic meshes for my 3D toolset, and in those cases it seems like it's more of a warning to not use those models since they're too dang big, haha.