Effil: easy and efficient multithreading

[mihacooper]

Hello, comrades!

Recently we released a Lua library to support OS multithreading - Effil.
Here is github repo and small project overview.
Comparing to other threading solutions this is easier to use and has more features:

• Manageable threads: pause/resume and cancel
• Tables transmittable between threads + metatables
• FIFO channels

Short example:

Code: Select all

effil = require "effil"

function foo(functor)
    for i = 1, 100 do
        functor(i)
    end
end

storage = effil.setmetatable(
                { val = 0 },
                {__call = function(t, val) t.val = t.val + val end }
)

thread = effil.thread(foo)(storage) -- run separate OS thread
thread:wait() -- wait for thread completion

print(storage.val) -- prints '5050'

For right now there is no any specifics for Love2d but it still can be very helpful for your projects.
In case of interest I will try to make Effil build compatible with Love2d objects or even embed it into engine!

[zorg]

Comparing to other threading solutions this is easier to use and has more features:

• Manageable threads: pause/resume and cancel
• Tables transmittable between threads + metatables
• FIFO channels

Have you compared it to löve's own threading solution, love.thread?

[mihacooper]

if you mean performance comparison then not, we didn't compare it yet. But if you look at our docs you'll see the difference in usage. Effil's API much more convenient and I would say it's more powerful (look at list in the first message).

[zorg]

Okay, but i can already see some points i'm not personally agreeing with:

• Needing to use your effil.table class/type instead of just passing through either a lua table, a Data type native to löve, or a pointer to a C type that i can define with the FFI.
• Runner.step seems like it needs to use the debug functionality of lua, inherently slowing processing to a crawl in comparison to what could be achieved speed-wise. (Except for when using 0, probably)

I can see that your implementation adds a few management-related functions that love.thread doesn't have explicitly; status, get, cancel, pause/resume; as well as kinda being a hybrid between threads and coroutines, since it has yield and sleep (though löve does implement love.timer.sleep, and yielding could be achieved by channels)

Now don't misunderstand me, your lib looks amazing, it's just that i'm not sure it's something that's needed with this framework specifically, since it already implements similar functionality. (Not to mention that needing to build so/dll files means it's that much less cross-os compatible, the same gripes i have with imgui)

That said, i'm just one person, so i might be wrong on many fronts; others should also share their views on this lib.

[mihacooper]

Thanks for feedback, zorg!

I will try to answer to you:

1. Needing to use your effil.table class/type instead of just passing through either a lua table, a Data type native to löve, or a pointer to a C type that i can define with the FFI.

   • effil.table is concept of Lua table shared between threads. There is no possibility to use regular Lua tables for that: of course you can serialize/deserialize them but that's not efficient and that doesn't allow to really share tables (have access from several threads).
   • And I don't really get why you may need FFI + C pointer for that purpose

2. Runner.step seems like it needs to use the debug functionality of lua, inherently slowing processing to a crawl in comparison to what could be achieved speed-wise. (Except for when using 0, probably)

   • Yes it uses debug hook, so yes it affects performance. That's why you can disable that feature)

3. I can see that your implementation adds a few management-related functions that love.thread doesn't have explicitly; status, get, cancel, pause/resume; as well as kinda being a hybrid between threads and coroutines, since it has yield and sleep (though löve does implement love.timer.sleep, and yielding could be achieved by channels)

   • I prefer to call that like high level threading API, not hybrid. We are implementing high level, easy-to-use library - that's our aim.

4. i'm not sure it's something that's needed with this framework specifically, since it already implements similar functionality.

   • And here you may be right! I'm not saying that it's something that should replace standard Love2d threading, of course not! But but for me Effil is more convenient (I'm using it in my project), so it can be helpful for people as well.