Accessing LÖVE C++ internals via luajit FFI

Questions about the LÖVE API, installing LÖVE and other support related questions go here.
Forum rules
Before you make a thread asking for help, read this.
Post Reply
User avatar
s-ol
Party member
Posts: 1080
Joined: Mon Sep 15, 2014 7:41 pm
Location: Milan, Italy
Contact:

Accessing LÖVE C++ internals via luajit FFI

Post by s-ol »

Hey all,
I want to stream the rendered output of love2d to other applications, directly on the GPU, using texture-share-vk. It provides a C API that I can use easily using the luajit C FFI and the following works perfectly:

Code: Select all

local ffi = require 'ffi'

ffi.cdef [[
// gl.h
typedef int GLint;
typedef unsigned int GLuint;
typedef unsigned int GLenum;

// texture_share_ipc.h
typedef enum ImgFormat {
    R8G8B8A8,
    R8G8B8,
    B8G8R8A8,
    B8G8R8,
    Undefined,
} ImgFormat;

// texture_share_gl_client.h
typedef enum ImageLookupResult {
    Error = -1,
    NotFound = 0,
    Found = 1,
    RequiresUpdate = 2,
} ImageLookupResult;

typedef struct ClientImageDataGuard ClientImageDataGuard;

typedef struct GlClient GlClient;

typedef struct GlImageExtent {
    GLint top_left[2];
    GLint bottom_right[2];
} GlImageExtent;

bool gl_client_initialize_external_gl(void);

struct GlClient *gl_client_new(const char *socket_path, uint64_t timeout_in_millis);

void gl_client_destroy(struct GlClient *gl_client);

enum ImageLookupResult gl_client_init_image(struct GlClient *gl_client,
                                            const char *image_name,
                                            uint32_t width,
                                            uint32_t height,
                                            ImgFormat format,
                                            bool overwrite_existing);

int gl_client_send_image(struct GlClient *gl_client,
                         const char *image_name,
                         void*, // GLuint src_texture_id,
                         GLenum src_texture_target,
                         bool invert,
                         GLuint prev_fbo,
                         const struct GlImageExtent *extents);
]]
local tvs = ffi.load("texture_share_gl_client")

local width, height
local canvas
local client, image

function love.load()
  width, height = love.graphics.getDimensions()
  canvas = love.graphics.newCanvas(width, height)

  assert(tvs.gl_client_initialize_external_gl(), "no init")
  client = assert(tvs.gl_client_new("/tmp/vk_server/vk_server.sock", 1000))
  image = assert(tvs.gl_client_init_image(client, "love2d", width, height, ffi.C.R8G8B8A8, true) > 0)
end

local angle = 0

function love.update(dt)
  angle = angle + 5 * dt
end

function love.draw()
  love.graphics.setCanvas(canvas)
    love.graphics.clear(0, 0, 0, 0)
    love.graphics.translate(width/2, height/2)
    love.graphics.rotate(angle)

    love.graphics.setColor(1, 0, 0, 1)
    love.graphics.rectangle("fill", -150,-150, 300,300)
  love.graphics.setCanvas()

  tvs.gl_client_send_image(
    client, "love2d",
    canvas:getHandle(),
    0x0DE1, -- GL_TEXTURE_2D
    false, 0, nil
  )

  love.graphics.reset()
  love.graphics.setBlendMode("alpha", "premultiplied")
  love.graphics.draw(canvas)
end
...there's just one small gotcha: I had to add canvas:getHandle() by patching love2d:

Code: Select all

--- old/src/modules/graphics/wrap_Texture.cpp.pre	2025-03-20 12:46:59.334954849 +0100
+++ new/src/modules/graphics/wrap_Texture.cpp	2025-03-20 12:46:23.144789811 +0100
@@ -300,6 +300,14 @@
 	return 1;
 }
 
+int w_Texture_getHandle(lua_State *L)
+{
+	Texture *t = luax_checktexture(L, 1);
+	void* handle = (void*) t->getHandle();
+	lua_pushlightuserdata(L, handle);
+	return 1;
+}
+
 const luaL_Reg w_Texture_functions[] =
 {
 	{ "getTextureType", w_Texture_getTextureType },
@@ -323,6 +331,7 @@
 	{ "isReadable", w_Texture_isReadable },
 	{ "getDepthSampleMode", w_Texture_getDepthSampleMode },
 	{ "setDepthSampleMode", w_Texture_setDepthSampleMode },
+	{ "getHandle", w_Texture_getHandle },
 	{ 0, 0 }
 };
Now I don't think this extremely narrow use case warrants adding an API like that to love2d, but at the same time it would be very nice to be able to portably distribute a "texture_share_vk.lua" library.

I was wondering if there could be some way to access the C++ function using the C FFI, but there are two hurdles:
  • love2d internals are written in C++, luajit FFI is C-only
  • liblove.so has internal symbols stripped
The first could be ignored by just dealing with mangled C++ names. The second is a build system configuration concern, but force-exporting all symbols also prevents linker optimization (i believe?) so this is not ideal either. When I build the arch package it also creates a debug package with symbols, and that already contains the function I need, but these symbols are not available by default.

Code: Select all

nm -sC /usr/lib/debug/usr/lib/liblove-11.5.so.debug | grep 'Canvas.*getHandle'
000000000009f960 t love::graphics::opengl::Canvas::getRenderTargetHandle() const
000000000009f980 t love::graphics::opengl::Canvas::getHandle() const
000000000009f990 t non-virtual thunk to love::graphics::opengl::Canvas::getHandle() const
Has anyone struggled with this before? Are there any other potential approaches I've missed? Is there a case for making this kind of integration easier in love2d? Discuss!

PS: If anyone wants to try my example, note that texture-share-vk is Linux only and the generated C headers are invalid without a ptach. The patch shouldn't be needed though as the FFI doesn't load the original headers directly. You can use the OBS source plugin provided by the author to visualize the stream by entering the name I've hardcoded as "love2d" in the example above.

s-ol.nu

Code: Select all

print( type(love) )
if false then
  baby:hurt(me)
end
RNavega
Party member
Posts: 456
Joined: Sun Aug 16, 2020 1:28 pm

Re: Accessing LÖVE C++ internals via luajit FFI

Post by RNavega »

Couldn't you use the color attachment of the main framebuffer instead?
The below is for Windows only. Getting the "name" (more like ID) of GL_COLOR_ATTACHMENT0 of the main framebuffer (Edit: actually whatever framebuffer that is currently bound, which is assumed to be the main one as the program has just started) always yields "0", so I wonder if it's possible to even hardcode it when using it with that gl_client_send_image().

Code: Select all

io.stdout:setvbuf('no')
local ffi = require('ffi')


-- Thanks to microsoft Copilot AI for speeding this up...
ffi.cdef([[
    // https://github.com/malkia/luajit-winapi/blob/master/ffi/winapi/headers/common.lua#L104C3-L106C32
    // Cheating by making it 'const' or else LuaJIT doesn't do auto-conversion from a Lua string.
    typedef const char CHAR;
    typedef CHAR *LPSTR; //Pointer
    typedef LPSTR LPCSTR; //Alias

    // https://github.com/malkia/luajit-winapi/blob/master/ffi/winapi/headers/common.lua#L28C3-L29C34
    typedef void VOID; //Alias
    typedef VOID *LPVOID; //Pointer
    // https://github.com/malkia/luajit-winapi/blob/master/ffi/winapi/windows/opengl32.lua#L6
    typedef LPVOID PROC; //Alias

    // https://github.com/malkia/luajit-winapi/blob/master/ffi/winapi/windows/opengl32.lua#L54
    PROC wglGetProcAddress(LPCSTR lpszProc);

    typedef uint32_t GLuint;
    typedef int32_t GLint;
    typedef uint32_t GLenum;
    void glGetFramebufferAttachmentParameteriv(GLenum target, GLenum attachment,
                                               GLenum pname, GLint *params);
    typedef void (*type_glGetFramebufferAttachmentParameteriv)(GLenum target, GLenum attachment,
                                                               GLenum pname, GLint *params);
]])

local TYPEOF_GLINT_PTR = ffi.typeof('GLint[1]')

local SDL = (jit.os == "Windows") and ffi.load("SDL2") or ffi.C

-- OpenGL binding from malkia's UFO:
-- https://github.com/malkia/ufo/blob/master/ffi/OpenGL.lua
local libs = ffi_OpenGL_libs or {
   OSX     = { x86 = "OpenGL.framework/OpenGL", x64 = "OpenGL.framework/OpenGL" },
   Windows = { x86 = "OPENGL32.DLL",            x64 = "OPENGL32.DLL" },
   Linux   = { x86 = "libGL.so",                x64 = "libGL.so", arm = "libGL.so" },
   BSD     = { x86 = "libGL.so",                x64 = "libGL.so" },
   POSIX   = { x86 = "libGL.so",                x64 = "libGL.so" },
   Other   = { x86 = "libGL.so",                x64 = "libGL.so" },
}
local gl = ffi.load(libs[ffi.os][ffi.arch])

local proc = gl.wglGetProcAddress('glGetFramebufferAttachmentParameteriv')
local glGetFramebufferAttachmentParameteriv = ffi.cast('type_glGetFramebufferAttachmentParameteriv', proc)

-- https://github.com/KhronosGroup/OpenGL-Registry/blob/main/api/GLES2/gl2.h#L348
GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME = 0x8CD1
-- https://github.com/KhronosGroup/OpenGL-Registry/blob/main/api/GLES2/gl2.h#L351
GL_COLOR_ATTACHMENT0 = 0x8CE0
-- https://github.com/KhronosGroup/OpenGL-Registry/blob/main/api/GLES2/gl2.h#L331
GL_FRAMEBUFFER = 0x8D40


function love.load()
    local tempName = TYPEOF_GLINT_PTR()
    -- Uncomment to see 'tempName' change to something else.
    --local c = love.graphics.newCanvas(320, 240)
    --c = love.graphics.newCanvas(320, 240)
    --c = love.graphics.newCanvas(320, 240)
    --love.graphics.setCanvas(c)
    glGetFramebufferAttachmentParameteriv(GL_FRAMEBUFFER,
                                          GL_COLOR_ATTACHMENT0,
                                          GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME,
                                          tempName)
    print('Pointer: ' .. tostring(tempName))
    print('Texture ID: ' .. tempName[0])
    error('stop')
end
User avatar
s-ol
Party member
Posts: 1080
Joined: Mon Sep 15, 2014 7:41 pm
Location: Milan, Italy
Contact:

Re: Accessing LÖVE C++ internals via luajit FFI

Post by s-ol »

Thank you, that's a great idea!
On Linux it seems I can just link to libGL statically, not sure if that is somehow illicit, it's working for me: :awesome:

Code: Select all

local ffi = require 'ffi'

ffi.cdef [[
// gl.h
typedef int GLint;
typedef unsigned int GLuint;
typedef unsigned int GLenum;

void glGetFramebufferAttachmentParameteriv(GLenum target, GLenum attachment,
                                           GLenum pname, GLint *params);

// texture_share_ipc.h
typedef enum ImgFormat {
    R8G8B8A8,
    R8G8B8,
    B8G8R8A8,
    B8G8R8,
    Undefined,
} ImgFormat;

// texture_share_gl_client.h
typedef enum ImageLookupResult {
    Error = -1,
    NotFound = 0,
    Found = 1,
    RequiresUpdate = 2,
} ImageLookupResult;

typedef struct ClientImageDataGuard ClientImageDataGuard;

typedef struct GlClient GlClient;

typedef struct GlImageExtent {
    GLint top_left[2];
    GLint bottom_right[2];
} GlImageExtent;

bool gl_client_initialize_external_gl(void);

struct GlClient *gl_client_new(const char *socket_path, uint64_t timeout_in_millis);

void gl_client_destroy(struct GlClient *gl_client);

enum ImageLookupResult gl_client_init_image(struct GlClient *gl_client,
                                            const char *image_name,
                                            uint32_t width,
                                            uint32_t height,
                                            ImgFormat format,
                                            bool overwrite_existing);

int gl_client_send_image(struct GlClient *gl_client,
                         const char *image_name,
                         GLuint src_texture_id,
                         GLenum src_texture_target,
                         bool invert,
                         GLuint prev_fbo,
                         const struct GlImageExtent *extents);
]]
local tvs = ffi.load("texture_share_gl_client")
local gl = ffi.load("GL")

local width, height
local canvas
local client, image

function love.load()
  width, height = love.graphics.getDimensions()
  canvas = love.graphics.newCanvas(width, height)

  assert(tvs.gl_client_initialize_external_gl(), "no init")
  client = assert(tvs.gl_client_new("/tmp/vk_server/vk_server.sock", 1000))
  image = assert(tvs.gl_client_init_image(client, "love2d", width, height, ffi.C.R8G8B8A8, true) > 0)
end

local angle = 0

function love.update(dt)
  angle = angle + 5 * dt
end

GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME = 0x8CD1
GL_COLOR_ATTACHMENT0 = 0x8CE0
GL_FRAMEBUFFER = 0x8D40
GL_TEXTURE_2D = 0x0DE1

function love.draw()
  love.graphics.setCanvas(canvas)
    love.graphics.clear(0, 0, 0, 0)
    love.graphics.translate(width/2, height/2)
    love.graphics.rotate(angle)

    love.graphics.setColor(1, 0, 0, 1)
    love.graphics.rectangle("fill", -150,-150, 300,300)

    local name_ptr = ffi.typeof('GLint[1]')()
    gl.glGetFramebufferAttachmentParameteriv(
      GL_FRAMEBUFFER,
      GL_COLOR_ATTACHMENT0,
      GL_FRAMEBUFFER_ATTACHMENT_OBJECT_NAME,
      name_ptr
    )
  love.graphics.setCanvas()

  tvs.gl_client_send_image(
    client, "love2d",
    name_ptr[0],
    GL_TEXTURE_2D,
    false, 0, nil
  )

  love.graphics.reset()
  love.graphics.setBlendMode("alpha", "premultiplied")
  love.graphics.draw(canvas)
end
RNavega wrote: Thu Mar 20, 2025 9:37 pm ...always yields "0", so I wonder if it's possible to even hardcode it when using it with that gl_client_send_image()
I tried this and it crashes - the "0" here means there is no color attachment on the primary framebuffer. Within the Canvas this works great though and solves my use case without patching löve.
The original question regarding the C++ APIs is still valid to a degree though, for example with this approach I can't access the texture name/id of non-canvas textures, like videos.

s-ol.nu

Code: Select all

print( type(love) )
if false then
  baby:hurt(me)
end
User avatar
slime
Solid Snayke
Posts: 3179
Joined: Mon Aug 23, 2010 6:45 am
Location: Nova Scotia, Canada
Contact:

Re: Accessing LÖVE C++ internals via luajit FFI

Post by slime »

s-ol wrote: Thu Mar 20, 2025 12:25 pm Is there a case for making this kind of integration easier in love2d? Discuss!
https://github.com/love2d/love/issues/1640

That being said, love 12 has 3 different graphics backends rather than just OpenGL. Hard-coding opengl internals into an app won't work there. Some graphics APIs (such as Vulkan) have more than one texture handle per texture as well, e.g. a texture image memory object and a texture view object.
s-ol wrote: Fri Mar 21, 2025 10:09 am The original question regarding the C++ APIs is still valid to a degree though, for example with this approach I can't access the texture name/id of non-canvas textures, like videos.
Videos use up to 3 different love textures internally (e.g. for YCbCr content). There isn't just one single id to access. Video doesn't inherit from Texture.
RNavega
Party member
Posts: 456
Joined: Sun Aug 16, 2020 1:28 pm

Re: Accessing LÖVE C++ internals via luajit FFI

Post by RNavega »

s-ol wrote: Fri Mar 21, 2025 10:09 am On Linux it seems I can just link to libGL statically, not sure if that is somehow illicit, it's working for me: :awesome:
Thanks for testing. It probably is illicit 8)

I see that you moved it to the inside of love.draw(), so I'm curious: does the canvas texture ID change frequently?
I was originally thinking of something like this:

Code: Select all

local my_canvas
local my_canvas_texture_id

function love.load()
    my_canvas = love.graphics.newCanvas(...)
    love.graphics.setCanvas(my_canvas)
    -- Get the canvas ID during startup.
    (...)
    my_canvas_texture_id = name_ptr[0]
end
...I'm assuming that the texture ID stays the same for the duration of the graphics context, so it'd only need to be queried once and then reused.

Edit: I also wonder if something similar could be done on the other backends, but in that case that texture-share-vk thing would have to support them as well.
User avatar
s-ol
Party member
Posts: 1080
Joined: Mon Sep 15, 2014 7:41 pm
Location: Milan, Italy
Contact:

Re: Accessing LÖVE C++ internals via luajit FFI

Post by s-ol »

slime wrote: Fri Mar 21, 2025 11:11 am https://github.com/love2d/love/issues/1640
Ah yes, that would be exactly what I had hoped to find in liblove.so, but would not have dared to feature-request flat out :ultrahappy:
RNavega wrote: Fri Mar 21, 2025 4:49 pm I'm assuming that the texture ID stays the same for the duration of the graphics context, so it'd only need to be queried once and then reused.
That's absolutely correct and exactly what I ended up doing when I put the code into an importable library:

https://git.s-ol.nu/lua-texture-share-vk/
https://luarocks.org/modules/s-ol/texture-share-vk

I also added an example for receiving the canvas.
RNavega wrote: Fri Mar 21, 2025 4:49 pm Edit: I also wonder if something similar could be done on the other backends, but in that case that texture-share-vk thing would have to support them as well.
As the name implies, texture-share-vk supports vulkan, but since vulkan doesn't have very much global state I don't think your idea for finding the texture name would work. You also need a couple extra parameters, here's the C signature:

Code: Select all

int vk_client_send_image(struct VkClient *vk_client,
                         const char *image_name,
                         VkImage image,
                         VkImageLayout orig_layout,
                         VkImageLayout target_layout,
                         VkFence fence,
                         VkOffset3D *extents);
For Mac and Windows, you'd want to wrap Syphon and Spout respectively. texture-share-vk is not widely adopted on Linux and could use some help but if it were a bit closer to feature parity with the other two, I think it would be very nice to have a cross-platform native wrapper combining the three APIs.

s-ol.nu

Code: Select all

print( type(love) )
if false then
  baby:hurt(me)
end
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 0 guests