Coverage reports in LOVE

[CaptainMaelstrom]

Has anybody successfully created coverage reports working in LOVE?

I've tried luacov (https://keplerproject.github.io/luacov/) but been unsuccessful. I can set the require paths so that the luacov lib loads ok, I can start it and know it's collecting stats, but I haven't been able to figure out how to get it to print a coverage report file. Even with the supplied tick command. When I try to love.filesystem.write(luacov.stats, 'cov.txt'), I end up getting errors from other parts of the library (hook.lua).

I'll keep trying, maybe end up rolling my own coverage reports like I did for tests. But if anybody has any insight on generating coverage reports for LOVE code I'd be happy to hear them.

[ivan]

Love2D uses LuaJIT so you have to make sure that tool is compatible with LuaJIT.
I don't see LuaJIT mentioned anywhere in the docs.
Generally speaking, profiling with LuaJIT is hard because it's super optimized and some hooks are not triggered at all.
I would love to see a better way to profile Love2D, until then you have to use an inferior tool like my profiler.
Hope that helps.

[Fuzzlix]

.. but I haven't been able to figure out how to get it to print a coverage report file.

If i remember correctly, it writes/append to a somehow binary datafile and you have to run luacov from commandline afterwards to generate a text file.

[raidho36]

Generally, for debugging you want to use interpreted Lua. Just swap out the DLL.

[JoshGrams]

LuaCov works fine for me under Windows in 11.2. I'm doing:

Code: Select all

require('luacov.runner').init({
	runreport = true,
	deletestats = true,
	exclude = {
		"^tests/.*"
		-- etc...
	}
})

Then after running my tests I just call love.event.quit as usual, and it writes the coverage report to luacov.report.out (I don't have to do anything special to make that happen). I haven't messed with this setup in ages so I don't remember if I had to do anything special or hack luacov at all to get it running (I don't think so).

[CaptainMaelstrom]

ivan and raidho36, I hadn't considered the added difficulty of testing/covering LuaJIT so thanks for making me aware of that extra gotcha.

JoshGrams, thanks for showing me that. I found more init options here:
https://keplerproject.github.io/luacov/ ... ml#modules

I'm now setting my own statsfile and reportfile paths so I can find them both. My stats file has lines like this:

Code: Select all

57:extends/debug.lua
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 336 0 336 336 336 0 336 336 0 0 0 0 0 0 336 

But my report file is:

Code: Select all

==============================================================================
Summary
==============================================================================

File  Hits Missed Coverage
--------------------------
Total 0    0      0.00%

I haven't excluded anything so I expected all files to get reported on. So I wonder why it's not showing me a report like:

Code: Select all

==============================================================================
test.lua
==============================================================================
 1 if 10 > 100 then
*0    print("I don't think this line will execute.")
   else
 1    print("Hello, LuaCov!")
   end

I must be missing some other init option. Anybody know how to interpret the stats file? Do you think swapping the .dlls like raidho36 suggested will help my situation? I'd rather test LuaJIT since that's what I'm deploying.

[raidho36]

LuaJIT and vanilla Lua are functionally identical. That is, if you don't use LuaJIT-specific functionality (such as C data) you should be fine with just swapping the DLL. Beyond running slower, there will be no difference in the way the code runs.

Well actually there will be subtle differences. For instance, LuaJIT will use CPU instructions for math operations where applicable, and those don't follow IEEE as a rule. Particular pitfall I've experienced is that using min or max with NaN as one of operands will not ignore the NaN and may return it if it was the right operand of the binary CPU instruction, and order of these operands is pretty random in compiled code.

[pgimeno]

Maybe executing it with jit.off() helps?

[ivan]

I'd rather test LuaJIT since that's what I'm deploying

When profiling, you will ALWAYS be profiling the Lua code, not the underlying binaries.

[CaptainMaelstrom]

I went to my love2d directory and swapped the existing LOVE 11.3 file called "lua51.dll" with the "lua5.1.dll" downloaded from:
https://sourceforge.net/projects/luabin ... p/download

Note: I'm using the 64-bit zipped Windows version of LOVE 11.3.

The report file looks the same -- empty, but now my print statements are not printing to console. Did I swap the right dlls?

I also tried jit.off() and my report file is still empty. I tried this after reverting the .dlls.

I really don't want to give up on coverage reports, so if anyone else has any ideas of things to try or alternatives to luacov even, I'm all ears.

Thanks all.