Fuzz testing and CI with headless love

[technomancy]

Hi folks!

I haven't seen much talk about automated testing with Love, so I wrote a blog post about what I do in my game Bussard. It can be a bit tricky to figure out how to run a Love program in a headless environment for continuous integration, and since my game is open source I thought it might be helpful for others to learn from. I also have a neat (well, I think it's neat) setup for fuzz-testing (throwing random input at my program by inspecting the currently active key bindings) that might be useful for others. It's already found over a dozen bugs in my own codebase.

https://technomancy.us/180

Feedback welcome!

I also use the GitLab CI feature to continually run my tests (fuzz tests, regular tests, and luacheck linting) against git master and all incoming patches. It's really easy to set up and quite handy at helping people get into the habit of testing: https://gitlab.com/technomancy/bussard/pipelines

[Jack5500]

Wow, that looks great. Never thought about automated testing in Löve, but the way you set it up I could see it working for me as well. Thanks!

[technomancy]

Thanks! I think the key to making well-testable code is making your input handling work against a table of key bindings. If you're doing it in a declarative way, it's easier to automate.

Of course, it's questionable how helpful the fuzz testing would be in other games; my game is somewhat unusual since it embeds a multi-mode text editor and in-game Lua console. There are about 150 keybindings (with some redundancy) across all the different modes. Anyway, I had a lot of fun with it.

[Pangit]

That is a very impressive os - now all it needs is a text editor. Thanks for sharing your implementation of fuzz testing.

* sorry for the joke. I had to before someone else did.

Seriously though that is a great project. Very creative.

[davisdude]

[1] Yes, I know I just set myself up for the old "Bussard is a great OS, it just lacks a decent text editor" joke. Honestly I am just waiting for someone to come along and implement a vim mode in-game; if any player thinks they can do better than the built-in editor they are welcome to try!

That is a very impressive os - now all it needs is a text editor.

In all seriousness a Vim mode would be pretty cool if somebody would want to take that on... Interesting write-up