LÖVE

Hi.
My problem is: garbage collector is ruining performance. With 'best' setting, it freezes the game once in several seconds. Originally it freezed the game for a moment very often, which made the character movement jittery.

Why I think it's GC:
1. It's unpredictable
2. When I disable the code which generates garbage, everything runs smoothly.

What generates garbage:
Unloads of chunks of blocks. Faster movement -> more chunk unloads -> more freezes.

What I tried (unsuccessfully):
1. Various ways of calling 'collectgarbage()" method.
1.1 "Step" every frame - makes things worse than original.
1.2 "Step" with some number - even worse.
1.3 "stop" and then only doing "steps" on frames - very bad. (not sure if it was this variant which caused 20-second interrupts)
1.4 "stop" when expensive code is executed, "restart" or "step" otherwise - not helpful
2. Reusing object in pool. When individual object is measured, it's slightly faster then creating new objects. Unfortunately, there are many of them. All the memory is quickly polluted with the huge pool. GC doesn't need to collect blocks, but it probably still checks them. Anyways, performance becomes very-very bad, and it makes usage of third way of 'improvement' impossible because pool size grows bigger then the allowed lua ram.
3. Distributing garbage generation across frames. Instead of unloading all chunks at once, only one chunk is freed per frame. It prevents small stops on every chunk border, postponing it to some random 1-2 seconds freeze in the future. This 'best' performance is still very irritating.

Option to reduce game quality is not really an option, it contains only a small portion of what I wanted to implement, and it already have troubles. Option to avoid GC by making structures in FFI is also probably not an option, it will make things much more complicated and probably will ruin flexibility and extensibility.

.love file is attached.

So, my question: am I missing something? Is there anything else I can try before I give up?

Hi and welcome to the forums!

Apart from the game's controls, which seems like the most extreme antithesis to being "tight" (the player character floating like there's no gravity, and also slow speedup and slowdown), the code's pretty hard to read (to me, anyway);

- What is getTurnTime used for? (which gives you back the current delta times a hundred)
- Does collectgarbage even support the "setpause" and "setstepmul" parameters?
- I wouldn't define functions inside love.update or love.draw (getScreenPrinter, printer even)
- The renderer does a lot of draw calls; i'd not use love.graphics.rectangle, and i'd use SpriteBatch-es to draw repeating tiles using one or more tilemaps.
Probably a fair deal more issues i'm not willing to pore over.

You can try using both SpriteBatches for less draw calls, and love.thread to offload chunk loading/saving from the main thread, allowing smoother gameplay.

Ok. I agree, controls aren't in a playable state. Actually, it's not a direct control, holding arrow keys only increases the acceleration applied to the main entity, and then common physics is applied. Not being bound to the surface by the gravity and no speed caps are needed for debug, so probably it's not the good time to rewrite that. I guess that collisions with the surface on each 1-block height increase are also very irritating.

- 'turn time' is for having integer times and intervals. Now it's used by a couple of 'growing' blocks. In case of many such blocks, the code will need to check a couple of ints per frame and notify the listeners, instead of checking many float times or each block individually checking its time.
- "setpause" and "setstepmul" are supported. When you set them to 800, gc hell begins. Also 'collectgarbage' throws an error when something unsupported is passed to it.
- Good point, no need for creating functions each frame. I've read that if values from outside the functions are not used, new function will not be created each time, however the test shows that in this particular code piece this optimization doesn't happen.
- Excessive rectangles are in the unfinished 'inventory' drawing. Can be hidden with 'i'. (actually, I should have hidden it by default when I posted the .love, sorry). Blocks are already drawn with SpriteBatch. Also several draw calls are added by drawing text to show stats.

Threads will not help with gc and object construction, and can add data transfer/sync overhead, however I guess it's worth trying since generating/disk operations can be also expensive. I'll probably try that.

Actually, I'm slightly feared that even after all the optimizations, and offloading to threads, and other complicated tricks, at some moment I'll be forced to make a choice between abandoning the project or rewriting it in a language with more pluses in its name, and all the current efforts will be thrown into a trash can.

Your chunk code reads data from the hard drive and parses it when it loads in a new chunk, and it serializes and saves data to the hard drive when it unloads an old chunk.

It does both of those quite frequently when you move. This will be very slow regardless of what language you use (and even if it was pausing due to garbage generation, it's quite feasible to reduce garbage generation to a point where collection doesn't impact performance noticeably).

Loading a file is something like 10,000,000 times slower than doing a math operation, for reference.

EDIT: Even with chunk file loading/saving disabled, your chunk generation code also takes multiple milliseconds per call for me, and seems to be called multiple times per frame every second or two when moving around. (It took roughly 40 ms in one example I measured, to execute the chunk unloading/loading). There's a lot of room to optimize that code.

no_login_found wrote: ↑Sun Jan 07, 2018 1:01 pm Actually, I'm slightly feared that even after all the optimizations, and offloading to threads, and other complicated tricks, at some moment I'll be forced to make a choice between abandoning the project or rewriting it in a language with more pluses in its name, and all the current efforts will be thrown into a trash can.

I don't have too much experience with chunks and spritebatches in LÖVE but I'm pretty sure that something like
your game can be made in it, even without using seperate threads. I'm currently working on my first chunk based project (a tile
map editor in LÖVE) and the results I get atm are really promising. After some tests I managed to get 25x25
chunks -> 36x36Tiles (32x32px) per chunk (810k tiles) drawing simultaneously with ~630 drawcalls at 600FPS without
updating the chunks and ~350FPS while continuously updating 5-15 chunks. And that still needs to be optimized.

Your game runs at about 50 fps at launch and then goes down all the way to 5-10 fps after digging
a couple minutes.. and that seems to be odd because you have way less chunks and spritebatches to update and draw.
I can't really tell whats going on in your code but for me it seems like there is a major problem
with the way you render and/or save and load the chunks/blocks/whatever.

Edit: yey, got ninja'd.

MadByte wrote: ↑Sun Jan 07, 2018 4:49 pm After some tests I managed to get 25x25
chunks -> 36x36Tiles (32x32px) per chunk (810k tiles) drawing simultaneously with ~630 drawcalls at 600FPS without
updating the chunks and ~350FPS while continuously updating 5-15 chunks.

Impressive numbers. Not sure how you achieved them. Probably that's because not many modifications are done to each vertex data?
I've made a couple of test projects, drawing overlapping half-transparent 16x16 random tiles with modified colors (which correspond game rendering of several block layers with transparent liquids and block damage indicated by shifting the color to red spectre). And I can draw only 250k at 45 fps with c++/opengl and only 80k at 40fps in LOVE with SpriteBatch . The game I've posted draws about 18k at 60fps, which is kind of consistent with the test project .

You can do all blocking hdd/network stuff in separate thread.
I wrote thread-class with callbacks for same problems.

no_login_found wrote: ↑Sun Jan 07, 2018 7:02 pm Impressive numbers. Not sure how you achieved them. Probably that's because not many modifications are done to each vertex data?
I've made a couple of test projects, drawing overlapping half-transparent 16x16 random tiles with modified colors (which correspond game rendering of several block layers with transparent liquids and block damage indicated by shifting the color to red spectre). And I can draw only 250k at 45 fps with c++/opengl and only 80k at 40fps in LOVE with SpriteBatch . The game I've posted draws about 18k at 60fps, which is kind of consistent with the test project .

Sorry for the late answer.. You're right, I'm not updating every single tile all the time because that would indeed cause performance problems. I just update the chunks which actually get changed.That's exactly what you need to do as well, there is no reason to update everything multiple times per frame.

Here is my (very early stages) editor project if you're intrested.

MadByte wrote: ↑Tue Jan 09, 2018 10:35 pm Here is my (very early stages) editor project if you're interested.

Thanks. Now I see how you are doing it. I've checked its performance, it never goes below 900fps without 'f1 debug', and with 'f1 debug' it never goes below 400fps. (However, I'm sure making my code utilize this approach is not the top priority performance improvement.)


I've checked what's causing those freezes while moving in already generated chunks.
Now it seems clear that they are caused by file access.
When the game stops, CPU usage drops to 0 for a couple of seconds, and for the same couple of seconds HDD access LED shines brightly.

So, as many of you advised, I've moved file i/o to separate threads.
Unfortunately, it didn't help.
When disk is busy, main thread is also locked.
Why?

I'm not loading any assets/lua files. They are all preloaded at the very beginning.
I'm not using any other file access in main thread. Not even checking for file existence or getting save directory.
I'm not waiting for threads to return anything. If channel:pop() returns nothing, code just goes further. In case I've misunderstood the nature of pop(), I've tried to use getCount() before it, and noticed no difference.
I've tried love.filesystem, io, and C functions with ffi (in case some of them operate on single thread), issue still happens for all of them.
I've even disabled print() in case it's secretly writing log file somewhere and can't do it because disk is busy, with no effect.
I've check other applications and they work fine while the game is hanging, so it's not some global OS freeze affecting all apps and threads, including main game thread.
I've searched in google to check file access issues with threads, and they say on forums that you can even write to the same file from different threads at the same time (if you don't care for consistency).
I've checked if LOVE is really using threads and didn't fallback to some evil GIL threads, and it can go beyond single core CPU percentage with ease.

What am I doing wrong, what am I missing? I'm sure I can improve disk performance by using fewer bigger files instead of multiple small one,and using simple binary format instead of current verbose txt one, but that will not give me an answer why file threads lock main thread.

Attached new .love version. Changed worldgen to provide a long straight tunnel for easy uninterruptible movement for long distances. Generation is still slow in main thread (but even if file loading is not used, chunk saving starts to causes issues after certain point), so if you want to eliminate gen time from the test, generation first run is needed . I'm checking with 1GB generated data(90k files).

I've discovered I was shooting myself in the foot all the time. The worst freezes were caused with love.filesystem.createDirectory in main thread to ensure save directory for chunks exists. Fixed it, and it's better now.
Process Explorer helped to see thread stack when it was in state wait:wrResource. Debugging process with visual studio showed different stack, however it also was related to files. Unfortunately, they don't show script function names, so it's not clear which lua code is calling those functions.
The thread freezes aren't completely gone. Now they happen in state wait:Executive, stack showing thread-related function names and functions like lua_open or lua_close. Not sure what that means, I create threads only once (or am I shooting myself in the foot again...). Will continue investigating this.