Post-0.10.0 feature wishlist

[MachineCode]

Maybe I am missing something here - I am not suggesting a change to Lua, but a new framework data object similar to imageData, that makes threads much more versatile for various data operations where copying data via channels would be non optimal. I know FFI can do this already, as you say, but it is not quite as user friendly as the other parts of love2d that are so clean and simple.

I know that when you pull a string out of the data object that string is unique to the thread and duplicated, but this is an improvement because the thread that wants to import the data only creates a new instance as required - which may be an arbitrary subset of the data posted. Another thread can post data into the object without knowing how or when it will be used. I am assuming the the lock is implemented by the insert/extract method on the data object - tricky code like that is probably best only done by people who know what they are doing.

So, lets say I have a thread that pulls in a block of data from a network socket and I want to make this available to some worker threads that treat this data like random access memory. Using a data object, the data would be inserted as a string, and then threads that need access would just use the read function on the data object as required. As you say, ffi could solve this, however ffi does expose the programmer to issues with memory access violations. A framework data object keeps the love2d coder safe in the sandbox. I really like the channel implementation because it is so elegant and clean, so really I am just wishing for a shared data object to compliment channels.

[raidho36]

You will wind up copying same amount of data while doing more work and losing LUA-table convenience. I don't see the benefit.

[zorg]

You will wind up copying same amount of data while doing more work and losing LUA-table convenience. I don't see the benefit.

Well, you can't send nested tables to threads through channels anyway.

[MachineCode]

"You will wind up copying same amount of data while doing more work and losing LUA-table convenience. I don't see the benefit."

If you are dealing with bulk data which is in bytes the overhead of putting it in tables is high. The worker thread may be doing something like reading bytes and using them to impliment a Virtual machine. You only have to write the data once to the data object. The worker thread might only access 10% of the data as bytes - so the arrayData:read_byte(index) is a small overhead to remove the need to duplicate the whole data as a table or even string. In cases where you are pipelining or sharing a data block with multiple client threads, the advantage accumulates. Every time you start a new worker thread that needs access to the same data, you only want to pass a reference to the data object. This ends up being efficient.

There are two useful tools for threads. Channels, and shared Data Objects. Both are really useful depending on how you design your program.

I come from a DSP background where data flow and efficiency is always prime concern, so this seems like a good idea to me.

[raidho36]

Game engines is an area where things are usually optimized out of the ass as it is.

To me it looks more like you have irregular approach to worker threads. They're not supposed to individually choose which data to process. Instead, master thread feeds data to worker threads, exactly the amount the worker is supposed to process. So in that regard it doesn't even makes sense to implement this, because that would be duplicating already existing functionality.

Just in case it's not immediately obvious: you can't use data directly on the shared memory location. LUA has to copy it into its state to use it. So any operation on such object results in copying a chunk of data. This is already implemented, in a better fashion.

[zorg]

Just in case it's not immediately obvious: you can't use data directly on the shared memory location. LUA has to copy it into its state to use it. So any operation on such object results in copying a chunk of data. This is already implemented, in a better fashion.

Thank the stars LuaJIT's ...oh look, i wasted my 1000th post on replying here... too bad.FFI enables this though, sharing access to data at some mem location, without having to copy lua state; Would be bad if i had to copy over a dozen buffers with hundreds of values each, value-by-value, every second...

[MachineCode]

OK, I am not trying to derail love2d or waste anyone's time here - all I can do is present what I think is a reasoned and logical argument for a feature that would be an addition to the framework and help ordinary programmers deal with threads safely.

Firstly, a data object as outlined is almost a subset of what already exists with imageData. If I was happy with redefining the 4 rgba bytes I could just use an image data object in an unconventional way. Based on this, I suspect the feature would be low cost for the devs.

Secondly, as zorg has said - an ffi can give this ability with LuaJIT. Can you still use LuaJIT if targeting an iOS device? From an efficiency point of view accessing a value from an ffi data is likely to be very fast, whereas from a data object you need to call the ArrayData:read() function which will have some overhead, but you don't have to step outside the sandbox.
Using the ffi for sharing sets of data still leaves the problem of how to update a block of vales without letting any other thread access that set during the update. That, I believe, is solved by adding the string insert/extract function to the data object and implementing the lock in the data object function - where it is done by the devs who understand how the whole framework operates.

raidho36 - why do we care about orthodox ways of using threads? As long as the technique we use preserves data integrity and avoids race conditions the programmer can design their program as they see fit. Here is an example of an application that seems to benefit from having a data object -

I want to implement a Virtual Machine in Lua code that executes bytecode tokens. I have 64K of various procedures all written in bytecode tokens - sort of like a set of library functions. This VM accepts input strings and processes them using parts of the 64k bytecode library. So I might want to spawn dozens of these VM threads as required, and each one needs the same 64K bytecode library - which doesn't change very often. Without using ffi, how could I avoid sending this library block to every newly spawned thread? Also, depending on the input presented to each VM, it might only actually use 10% of the 64K bytecode library. Duplicating it in full in each spawned thread would be a waste. I don't necessarily need the whole 64k bytecode library in each thread's Lua state - I just need to have access to it like its an array and the slight overhead of a function call to retrieve a byte is negligible.

Anyway, I have tried to make a reasonable argument here, so have a think about it. I know love2d is mostly popular for writing games, but it is such a great framework that lends itself to unconventional applications like virtual machines and processor emulations - which may seem to be unorthodox from a game tool perspective.

[slime]

Can you still use LuaJIT if targeting an iOS device?

Yes, LOVE uses LuaJIT (including the FFI) on iOS. JIT compilation is disabled on iOS and Android though, so while the FFI will still work it will be much slower than it normally is on desktop platforms.

[Zireael]

Will there be a way for LOVE2D to support svg or other vector graphics out of the box?

[Fuzzlix]

Hi.

I played around with my own love.run a little bit to lower the cpu load of my idleing game.
For loop time calculations are 2 functions responsible: love.timer.getDelta() and love.timer.step().

It looks like, getDelta() returns the time between the last 2 step() calls? In this case the result of getDelta() would be outdated by one loop! Shure it is not a big issue but i cant get any information about the actual time spent since the last step() command and sometimes i need this information.
I suggest a new pair of functions which may be used in addition or as a replacement to the existing ones:

func1(): returning the time since the last step() command.
func2(): returning the time since the last step() command AND executing step().

Inside your love.run() you call...

Code: Select all

love.update(func2())

...to get the time spent since the last love.update(dt) and reset the timer automatically.
In the next love.run() loop you can call func1() as often you wish to get the time spent since the last love.update() call without the 1-loop-delay the getDelta() function has.