Is there something like Canvas:overwriteOldImageData() ?

[no_login_found]

Canvas size=screen size
If I call Canvas:newImageData() each frame, it just goes over 2GB and crashes after several seconds.
If I follow it with collectgarbage(), it doesn't crash, but it's extremely slow.
How can I access canvas contents each frame without creating huge imagedata each frame?

[raidho36]

Short answer: you can't. The image data is on the GPU, and to access it on the CPU you have to duplicate it from GPU RAM to system RAM, which is slow, for obvious reasons. Also I suspect that it's slow both with and without "collectgarbage", a single object creation and deletion times are negligible, so is performance impact of having a single extra Lua instance to be processed by a GC.

You can however do pretty much anything as long as the data never has to leave the GPU (or enter it from the outside) for no performance penalty on top of normal computational costs. Now, you can't draw a canvas to itself. You can however use two canvases as front and back buffer for this purpose.

[PGUp]

Canvas:clear()
Then set the canvas and draw things to it again

[no_login_found]

Fortunately, I realized that now I need only 1 px and not every frame, so I can go with newImageData without huge overhead.

However I did some tests to see what is slow and how slow it is:

Reading 1px
----------------------
glReadPixels 8ms
Canvas:newImageData 8ms

Reading whole screen of pixels
----------------------------------
glReadPixels 12ms
Canvas:newImageData 15ms
Canvas:newImageData + collectgarbage 53ms
direct call to ImageData:__gc causes unexpected errors in other functions, probably corrupts something?

So we can clearly see that getting data from canvas is slow, but memory issues are causing +300% more delay for big images when default gc is not enough to collect them until out-of-memory crash.

Also tried to implement requesting data to PBO, but unfortunately adding related functions to ffi is much more complicated than just adding glReadPixels. Tried to use glewInit() and it returns success, but there's still no functions in any namespace. Probably that requires much more knowledge of related libs.
By the way, forum engine insists that opengl abbreviations like PBO or FBO are 'too common' and refuses to search for them.

[zorg]

The forum can only search for keywords with lengths of at least 4 characters, which is indeed a bummer, but i do recall some talk about this before, and i believe there would be issues if they enable the 3 character minimum limit.

[raidho36]

Calling collectgarbage triggers the GC to do a full round of garbage collection - it is a slow operation and is slower if you have more live Lua objects.

I suggest you try accomplish your task without reading pixels.

[no_login_found]

I was able to get pointers to missing opengl functions with wglGetProcAddress on win.

Now it takes 10^-5 seconds to get pixel from PBO (not sure how inaccurate such small values are), with glReadPixels as soon as drawing is complete and reading from PBO only when needed.

Probably it can be optimized more by manually creating framebuffer with params optimized for reading as opposed to 'for render', but at the same time I know that nvidia drivers are good at overriding hints when the usage doesn't match them, so it's also possible that the canvas was automatically optimized for reading.

Tried if I have the same issue with missing functions on linux, and there was no issue: just ffi.load("GL") is enough. It ran 19 fps without requesting pixels and 17fps with requesting pixels. But since it was on VM, I can't really make any conclusions.
Tried it for android, and it says there's no "libGL.so". Made it fallback to Canvas:newImageData if opengl loading wasn't successful. Anyways, android is to slow to run the game, so I tried it just out of curiosity.

Not sure if it's good enough, there's still a chance that I'll be forced to use the solution which doesn't involve reading pixels from framebuffers.

-- creating pbo

Code: Select all

      ffi.GLL.glGenBuffers(1, PBO_id)
      ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
      ffi.GLL.glBufferData(ffi.GL.GL_PIXEL_PACK_BUFFER, memsize, nil, ffi.GL.GL_STREAM_READ)
      ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)

-- requesting data from canvas

Code: Select all

    love.graphics.setCanvas(zBuffer)
    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
    ffi.GL.glReadPixels(love.mouse.getX(),love.mouse.getY(), 1,1, ffi.GL.GL_RGBA, ffi.GL.GL_UNSIGNED_BYTE, nil)
    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
    love.graphics.setCanvas() 

--getting data from pixel buffer object

Code: Select all

    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, PBO_id[0])
    zImage2 = ffi.cast("unsigned char *", ffi.GLL.glMapBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER,ffi.GL.GL_READ_ONLY))
    ffi.GLL.glUnmapBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER)
    ffi.GLL.glBindBuffer(ffi.GL.GL_PIXEL_PACK_BUFFER, 0)
    id = zImage2[0]

-- getting missing functions

Code: Select all

    
    if ffi.GL and ffi.os == "Windows" then
      ffi.cdef[[
          typedef int (__stdcall *PROC)();
          typedef char* LPCSTR;
          PROC wglGetProcAddress(LPCSTR name);
      ]]

      local nameHolder = ffi.new("char[100]")
      ffi.GLL = {}
      local function loadProc(definition, name)
        ffi.copy(nameHolder, name)

        ffi.GLL[name] = ffi.cast(definition, ffi.GL.wglGetProcAddress(nameHolder))

        local asInt = ffi.cast("int",ffi.GLL[name])
        if asInt >=-1 and asInt <=3  then
          print("wglGetProcAddress returned nothing for "..name)
          io.stdout:flush()
          error("wglGetProcAddress returned nothing")
        end
      end


      loadProc("void (*)(GLsizei, GLuint *)", "glGenBuffers")
      loadProc("void (*)(GLenum, GLuint)", "glBindBuffer")
      loadProc("void (*)(GLenum target, GLsizeiptr size, GLvoid* data, GLenum usage)", "glBufferData")
      loadProc("void* (*)(GLenum target,  GLenum access)", "glMapBuffer")
      loadProc("GLboolean (*)(GLenum target)", "glUnmapBuffer")
    else
      ffi.GLL = ffi.GL
    end

[raidho36]

What are you doing with that one pixel anyway?

[no_login_found]

Checking which ui element is under the mouse. Approximately like it's described here https://en.wikibooks.org/wiki/OpenGL_Pr ... _selection, but now my implementation is much more simple since I'm not using stencil buffer, just painting a rect with color=element id to another canvas. Probably it will be more useful when/if I have overlapping non-rect shapes with holes.

[grump]

I'm not using stencil buffer, just painting a rect with color=element id to another canvas. Probably it will be more useful when/if I have overlapping non-rect shapes with holes.

That's a straightforward and obvious way to implement it, but beware: reading pixels from the color buffer stalls the rendering pipeline and can be a very expensive operation, depending on the rendering hardware and the complexity of your scene. If at all possible, it's usually faster to have a simple representation of your UI shapes instead, e. g. by using a region data structure or polygon approximations.