Over two frames of lag in mouse movement

[cloudberry]

First of all, thank you for taking your time to read this. I've been stuck on this for a while and would appreciate any help you can give. I have read related forum posts but noone appears to have done more thorough testing on this issue.

Attached is a minimal example to reproduce the issue I am having (trackingTest.love). Open the .love file and move the cursor over the window; there is a small box following its movements. If you record this at the framerate of your monitor (vsync is enabled), you will notice that the position of the box is always between two and three frames behind the position of the cursor. The example really draws three overlapping boxes, but that is just to show that the positions given to me are consistent across love.draw, love.update, and the mouse event. The box appears to move very little during the first two frames of me moving the cursor, which leads me to believe that either the mouse event or the rendering is two frames behind the physical input, but that is just a hypothesis.
If you disable vsync or use the version with vsync disabled I attached (trackingTest-no-vsync.love), or if you use a monitor with a higher refresh rate, the problem will be a lot less noticeable, but it will still be two or more frames of lag (record at proportionally higher frame rates to see this), just that the frame time is a lot shorter. Given the love2d update/rendering pipeline as I understand it, I would expect no more than a single frame of lag between the mouse event updating the box position and the box being drawn at the new position on screen. Other rendering engines I tested while debugging this do not have more than a single frame of delay between mouse movement and updated rendering.

What is going on here? If anyone has any pointers as to what is happening, and perhaps even how to fix it, I would be delighted to know. Thank you again for reading.

[pgimeno]

With vsync enabled, a 1 frame delay, or maybe more, is expected. See https://love2d.org/forums/viewtopic.php ... 09#p195509

With vsync disabled, you'll have to complain to your OS makers or your graphics card makers, because for me (Linux, nVidia) it works fine.

By the way, in order to obtain what you seem to want with the boxes, you should use alpha = 1 for the first rectangle, alpha = 1/2 for the second, and alpha = 1/3 for the third; that will give grey if they overlap. It's the same if you want to show a mix of three images at the same time. If there were a fourth, it would use alpha = 1/4, and so forth.

[cloudberry]

I am using Linux (Linux Mint 20.2, also nVidia graphics), and this is what I recorded. The delay was not 1 frame as I said, but 2-3 frames with VSync. And with vsync off, once again, it is still 2-3 frames, just that the frame time is much lower.

The boxes were just meant as a demo of the issue; not much effort was put into them and it's unrelated to the issue I am having.

Edit: Perhaps I should clarify. I had read the issue you linked before, and I, indeed, expect a delay of one frame, but not more than that. No other graphics framework (and program) I tested in a similar way has had a delay of more than a single frame, so I figure it must be an issue in love2d.

[Xii]

That does indeed seem sickeningly laggy. There should not be that much delay getting from mouse to Lua.

[grump]

Here's my colorful explanation that's probably wrong somewhere:

gpu_pipeline.png (50.66 KiB) Viewed 5236 times

Frame 0: backbuffer and frontbuffer are empty, nothing on screen
Frame 1: backbuffer filled, front buffer empty, nothing on screen
Frame 2: first time something is shown on screen - the things from 2 frames back

And that, kids, is why rendering things at the mouse cursor position has a 2-frame lag (and why reading pixels from the GPU is a bad idea)

[cloudberry]

Frame 0: backbuffer and frontbuffer are empty, nothing on screen
Frame 1: backbuffer filled, front buffer empty, nothing on screen
Frame 2: first time something is shown on screen - the things from 2 frames back

This could make sense as an explanation. However, I don't know why the GPU wouldn't execute the code and push to the screenbuffer within the same frame; the GPU processing itself (and filling of the backbuffer) is independent of VSync, only the push to the screenbuffer has to wait for the monitor. If it worked as shown, this would also be a universal issue, yet I only observed this behaviour when using love2d.

[BrotSagtMist]

Huh so far i have understood it any lag is caused because the calculations begin right after a frame was pushed to the monitor and is significantly faster than a frame, then the program stays idle for almost an entire frame until this now old calculations are drawn to the monitor.
If you turn this around by adding a sleep that fills up the entire sleep gap the lags should significantly reduce.
Try this mod, any keys toogle:

lag.love

(715 Bytes) Downloaded 155 times

[pgimeno]

What other applications have you checked with? Were they GL? Did you try the other applications with the same monitor/graphics card/drivers/OS you're using?

[cloudberry]

What other applications have you checked with? Were they GL? Did you try the other applications with the same monitor/graphics card/drivers/OS you're using?

I have reproduced this issue with other applications on my own (Linux) machine, but we've reproduced the problem on multiple Windows machines as well. The easiest way for me and others to test this behaviour is to use applications with canvases that can be moved by the cursor; you can just grab a fixed point such as a corner and watch the canvas lag behind as you drag your mouse cursor around. Of the ones that would be easy to reproduce it in, I have recorded the behaviour of canvas dragging in GIMP (a GTK app) and Krita (a Qt app), and I recorded the window dragging behaviour in my own desktop environment (Cinnamon). None of these applications displayed a delay greater than a single frame in any situation. All these applications use OpenGL behind the scenes as far as I am aware.

[grump]

Not really comparable.

Here's an example once in love, with a bit of ffi to make sure there's no buffering of the mouse coordinates:

Code: Select all

local ffi = require('ffi')
local SDL = ffi.os == 'Windows' and ffi.load('SDL2') or ffi.C
ffi.cdef('uint32_t SDL_GetGlobalMouseState(int*, int*);')
local mouseX, mouseY = ffi.new("int[1]"), ffi.new("int[1]")

function love.draw()
    SDL.SDL_GetGlobalMouseState(mouseX, mouseY)
    local wx, wy = love.window.getPosition()
    local mx, my = mouseX[0] - wx, mouseY[0] - wy
    love.graphics.rectangle('fill', mx - 16, my - 16, 32, 32)
end

And the same thing with raylib:

Code: Select all

#include "raylib.h"

int main(void) {
    SetConfigFlags(FLAG_VSYNC_HINT);
    InitWindow(800, 600, "raylib mouse lag");
    SetTargetFPS(60);

    while(!WindowShouldClose()) {
        int mx = GetMouseX(), my = GetMouseY();

        BeginDrawing();
        ClearBackground(BLACK);
        DrawRectangle(mx - 16, my - 16, 32, 32, WHITE);
        EndDrawing();
    }

    CloseWindow();
}

There is no perceivable difference.

Edit: I take it back! I tested on a machine that didn't do vsync properly. It indeed looks like the love version lags more with vsync on. I don't wanna make a video, maybe someone else.