Procedural Audio Clicks

[Parameterized]

I'm trying to implement procedural audio in my game, but having trouble getting rid of the clicking that I assume is caused by sudden changes in the waveform. At sound clip lengths that are multiples of 0.5, there's no clicking between them (if you keep the frequency the same.) Looking at the waveform seems to show that there are no sudden changes, so is the way love handles audio clips cutting off some of the beginning or the end if it's not a multiple of 0.5? Or maybe source:isPlaying() is inaccurate at most clip lengths? I've tried adding extra to the end of the clip and using the time since the last clip played to loop, but that didn't seem to help.

[zorg]

I would guess that you have two issues.

One is that you're not setting the source to loop the waveform, rather, you just restart it in love.update very time it's not playing.

Second is that whether you do restart it or not, if the length is not related to the frequency and the sampling rate of the SoundData (or to be more precise, the function with which you generate the samplepoints), it will clip.

Take math.sin for example, if you don't buffer a "whole" period, there will be a sharp cut in the produced waveform.


You could use iterOffset with Source:tell and Source:seek, but that kind of defeats a purpose, since you'd always need to seek to a point where the Source's sounddata will align with the end of it.


tau = 2*pi
sin(x * tau) has periodicity of 1.0
sin(1 Hz * tau / 44100 Hz) has periodicity of 44100
sin(440 Hz * tau / 44100 Hz) has a period of 2205/22 (Thanks, WolframAlpha)
So, for 440Hz, the first "restart" of the periodic waveform, and since this is sine, the second zero-crossing, is at the 2205th samplepoint.

Calculating this for any frequency one might set gives us a corrected buffer length, using that we should get noless popping (if we loop the source as well, we might get more popping with restarting the source "manually" in love.update still, as well as when we mess with the buffersize or the frequency via the GUI)

Now, this actually doesn't fix it, since there are frequencies that can't be chopped up in a way that it would neatly repeat with one static buffer size, worse yet, it might fit, but only after a ludicrously large number of samplepoints. (And you can't tell what value might give back what period)



You know, one neat solution would be the following:

User sets the length (in seconds) -> Code turns that into samplepoints, but transforms that to a whole multiple of the sampling rate.
something like math.floor(length*rate) should work. -> create a sounddata with this length

User sets the pitch (in Hz) -> Divide it into two parts

First part is the base frequency of the waveform generating function, computed so the waveform will be periodic with regards to the length of the buffer. (meaning that the distance between all samplepoints, -including- the last and the first, is equal)
math.floor( calculatedLength / frequency) -- calc. length is the value we got above.
The above is actually just the nearest multiplier of the frequency we want, so you'd generate the samplepoints with
func(i * frequency * firstPart / rate) -- 2pi or tau is a "normalizer" of the sin function, it's not used in the general form, but you would use it if func was a trig. function. You do want the values to be periodic in [0,1*calculatedLength]!

Second part is the adjustment factor, that you feed to the source using Source:setPitch (a multiplier), so that it sounds more exact.
(calculatedLength / frequency)/ floor(calculatedLength / frequency) -- the second expression is firstPart in the one above.

This has an added bonus: If you set the pitch, and the first part is not changed, you can just setPitch even if playback is still happening (with the Source being set to loop), and it will be more seamless than you could muster with any other (vanilla löve) method.

With some numbers:

calcLength = math.floor(0.2 seconds * 44100 Hz) --> 8820 samplepoints
firstPart = math.floor(8820 samplepoints / 440 Hz) --> 20
So you fill the SoundData with (example: sine wave) math.sin(i * 20 * math.pi * 2 / 44100)
secondPart = (8820/440) / math.floor(8820/440) --> 1.002272...
So you'd set Source:setPitch(1.0022727272727272727272)

So, if my math checks out, this should work better.
((44100 Hz) / ((8820 smp / 440 Hz)/floor(8820 smp / 440 Hz)) / (20 cycles / 0.2 seconds) == 440 Hz)

Note that we could, for efficiency's sake, adjust the size of the soundData by dividing with the integer multiplier of the wanted frequency, but that somewhat invalidates the length slider... that said, filling a gigantic buffer is somewhat slow, using smaller ones is better, especially when the content of it doesn't change.

Finally, we could also just store one "static" period of a sine wave, and only use setPitch to multiply it, but it will sound horrendous if one pitch shifts it too far from 1.0 (Also don't set it to 0.0)

Now, i don't want to toot my own horn, so i'll say that others, including me, did experiments with this kind of thing before, either in similar ways, generating sounddata and playing it back, or, with Queuable sources, that one can push data to in realtime.
Here's my latest test project in the latter area. This probably works too slow on IOS (And on android too at the moment) to be of use, but on computers, it's fast enough to be worth using.

[Parameterized]

So it's not stopping at a zero-crossing that makes it pop? Do you know how something like this works/could be translated to l2d?

[zorg]

Well, yes and no.
I mean, one whole period of a sine wave has two zero-crossings, but the slopes are different, so even if you start with a samplepoint of value 0, if you "wrap" the waveform at the first zero-crossing, then that's not really a sine wave (it's an abs(sin) one! ), and it will sound different, more complex.
But as i said, the popping can come from many places. That's one, another is the unpredictability of the game loop (i.e. love.update, and to go back one level, love.run itself) You can never rely on that being 100% precise, since it won't be.
Even if you solved the above two issues (which are somewhat in the order of easy to hard), there's still one bigger issue.
If you could feed the sound card data in realtime (which you can with the project i linked in my previous post), the smaller/shorter the buffer, the more times the cpu needs to upload those chunks onto the soundcard... meaning that if this process is not fast enough, you will get buffer underflows, which is basically a way of saying that the sound card suffers from resource starvation... or that you're trying to drink faster than how your cup is being filled (yay metaphores )

What you linked is a neat shadertoy
Since that's doing the hard work in glsl, it's a bit obfuscated, but yes, i can somewhat work out how it works.
There are two melody lines, and they are mapped to the left and right channel in a 3:7 and 7:3 ratio.
The notes are defined per-pitch first, then per-time, with delays specified by those D() functions, i believe.
In english, "wait for x time, then play this or that note".
It's actually how some types of music notation works.

As for translating it to lua, i have no idea!
I would need to spend time understanding it more if i would want to do that... which i don't.
But hey, feel free to look at the source of the thing i linked above, or even this; not the cleanest code, but i believe i did pepper it with comments, so it shouldn't be that hard... if you already have prior knowledge about some audio stuff.