LuaPd - Real-time audio synthesis with Pd (Pure Data)

[myQwil]

Github Repository: https://github.com/myQwil/luapd

luapd.zip

(5.14 MiB) Downloaded 440 times

Pd is an open source visual programming language for multimedia. It is a major branch of the family of patcher programming languages known as Max (Max/FTS, ISPW Max, Max/MSP, etc), originally developed by Miller Puckette.
More info about Pure Data can be found here: https://puredata.info/

LuaPd is a Lua C API Library that allows users to incorporate Pd patches into their projects.
A simple example might look something like this:

Code: Select all

if love.system.getOS() == 'OS X' then
	package.cpath = './?.dylib;'..package.cpath
end
require('luapd')

function love.load()
	inChannels ,outChannels ,sampleRate ,queued ,bitdepth ,ticks ,buffers =
	1          ,2           ,44100      ,false  ,16       ,1     ,33

	pd = PdBase()
	pd:init(inChannels, outChannels, sampleRate, queued)
	pd:computeAudio(true)
	patch = pd:openPatch('pd/test.pd')

	local size = PdBase.blockSize() * ticks
	sdata  = love.sound.newSoundData(size, sampleRate, bitdepth, outChannels)
	source = love.audio.newQueueableSource(sampleRate, bitdepth, outChannels, buffers)
end

function love.update()
	while source:getFreeBufferCount() > 0 do
		pd:processShort(ticks, sdata:getPointer())
		source:queue(sdata)
		source:play()
	end
end	

The zip includes a few LÖVE examples to help you get started:

major-minor:

major-minor.jpg (24.49 KiB) Viewed 17474 times

Change the scale of a piece of music to minor, dorian, phrygian, etc.

freqmod:

freqmod.jpg (39.87 KiB) Viewed 17474 times

A frequency modulator.
mouse horizontal movement changes the modulation frequency
mouse vertical movement changes the modulation index
mouse wheel changes the carrier frequency

If you find that the audio is stuttering a lot, you can open love/pdmain.lua and increase the number of ticks or buffers. Or you can decrease these amounts if you find that there's too much of a delay. Disabling vsync allows you to further decrease the delay, though it may have the side effect of screen tearing.

All feedback is welcome and appreciated. Let me know if you run into any issues or find any bugs and I'll try to address them as quickly as possible

[zorg]

One potential thing you could do is have the audio generation part running on another thread; that way, neither visual stuff (including potential vsync), nor the OS scheduler will mess with the processing, and it might work better with less latency as well.

Also, i might incorporate this into my composer project eventually, so cheers for the implementation!

[myQwil]

Cool, I look forward to seeing your composer project
The first time I tried using a thread, I don't think I had a firm enough grasp over how they worked, but I've recently given it another shot and it seems to be working really well. It allows for a much smaller delay. I'll update the zip as soon as it's ready.

EDIT: after further testing, the difference seems to be pretty negligible. I think that's because the delay was never really that bad to begin with and any delay I was personally experiencing was due to an audio server (JACK) adding its own delay of about 42.7ms. If I turn off JACK and just use pulseaudio, the delay is hardly noticeable. I'll probably still add the thread version as an extra example but it's probably not worth completely switching over to that method.

[Felix_Maxwell]

This is so cool, great work! I've been using Max/MSP /Pd for like 12 years so this is quite neat to see. It opens up possibilities of creating music production tools with Love as the GUI. I've also been considering using luasocket to communicate with SuperCollider, though the overhead of having the user install SC is kind of a barrier.

Again, really excellent work. If someone were to port the audio for their game to a Pure Data patch , they would have infinite flexibility as far as audio goes (not to knock love's audio engine of course, I know it's super advanced and feature-packed)

[Nixola]

EDIT: after further testing, the difference seems to be pretty negligible. I think that's because the delay was never really that bad to begin with and any delay I was personally experiencing was due to an audio server (JACK) adding its own delay of about 42.7ms. If I turn off JACK and just use pulseaudio, the delay is hardly noticeable. I'll probably still add the thread version as an extra example but it's probably not worth completely switching over to that method.

I think this is the first time in my life I've heard of JACK having more latency than Pulse; how are they both configured?

[zorg]

Cool, I look forward to seeing your composer project
The first time I tried using a thread, I don't think I had a firm enough grasp over how they worked, but I've recently given it another shot and it seems to be working really well. It allows for a much smaller delay. I'll update the zip as soon as it's ready.

EDIT: after further testing, the difference seems to be pretty negligible. I think that's because the delay was never really that bad to begin with and any delay I was personally experiencing was due to an audio server (JACK) adding its own delay of about 42.7ms. If I turn off JACK and just use pulseaudio, the delay is hardly noticeable. I'll probably still add the thread version as an extra example but it's probably not worth completely switching over to that method.

It was mostly a windows consideration anyway (where OALS does add about 50 ms latency to anything you do with löve)

[myQwil]

I think this is the first time in my life I've heard of JACK having more latency than Pulse; how are they both configured?

It's the "Pulseaudio JACK Sink" setup, so it's still using pulse but then additionally being sent through JACK.
Also it had a lot to do with the size of the buffers that I was telling JACK to work with, and I was able to make the delay smaller by lowering the frames/period and periods/buffer settings

[myQwil]

I recently discovered that while the Windows version was working on my end, it most likely hasn't been working for anyone else on Windows unless they had MSYS2 with a specific set of packages already installed. I worked out which libraries were needed by luapd.dll and hopefully it should now work for everyone else as well. The updated zip has been added to the original post.

[blurryface]

What a great job. I've been working with Pure Data and Lua for a while and this library it is just amazing.
But I have a question: Is it only work in Love? Or we can use this library only with a Lua code?

[myQwil]

What a great job. I've been working with Pure Data and Lua for a while and this library it is just amazing.
But I have a question: Is it only work in Love? Or we can use this library only with a Lua code?

In theory, it should be able to work with anything written in Lua. One thing to bear in mind is that this library doesn't do its own audio output. It generates the samples, but it's up to you to send those samples to an audio device.

Love has the QueueableSource, which makes it easy to output the audio samples. Any Lua-oriented game engines that support real-time audio generation will have their own unique audio interfaces, but probably relatively similar solutions.