Microphone Support for LÖVE!

[zorg]

Well, some of them are optimized to such a degree that's not really attainable with pure lua (nor with ffi even); one'd either need to use an external lib (either something hand-written that'd expose functions that access SIMD functionality, or big libs, e.g. intel integrated performance primitives) to get such speeds.

But yes, i do aim to test out how many simultaneous voices i can squeeze out of löve.

[Dr. Peeps]

Code: Select all

function love.update ( )
	if inputs[ 1 ]:getSampleCount ( ) > inputs[ 1 ]:getSampleRate ( ) / 20 then
		local data = inputs[ 1 ]:getData ( )
		...

Can I ask what the "/ 20" is for in the above if statement? Is it basically limiting the getData() calls to 20 times per second?

Also, the parameters for the startRecording() function are: samples, sampleRate, bitDepth, channels. For the samples parameter, does it simply mean the recorder will fetch only that many samples with each call to getData(), and then stop recording?

[raidho36]

The division by 20 calculates an offset. It will not start playing until it has 1/20th of a second worth of buffered sound. If you start playback as soon as you get any readout from the microphone, there will only be exactly enough samples to play before the next readout, assuming it happens exactly at the moment when buffer runs out. Which in reality simply doesn't happen and the audio stutters because the next readout will often be slightly late and the playback buffer will underrun and stop, causing an audible gap. I can live with that for testing purposes, but it'll give bad impression to people who are actually going to use it.

The "samples" argument says you want the recording buffer to be this long. It acts as a circular buffer so that translates to about "keep this many samples recorded". If you fail to read out the data and the buffer fills up, it'll keep recording but it'll overwrite itself and you'll lose the data at its tail; since it's practically a circular buffer, self-overwriting doesn't cause any audio problems.

[Dr. Peeps]

The "samples" argument says you want the recording buffer to be this long. It acts as a circular buffer so that translates to about "keep this many samples recorded". If you fail to read out the data and the buffer fills up, it'll keep recording but it'll overwrite itself and you'll lose the data at its tail; since it's practically a circular buffer, self-overwriting doesn't cause any audio problems.

That makes it sound like the buffer will magically start overwriting itself before I'm ready. The buffer is static until I call getData() on it again, isn't it? My understanding was that if I don't call getData(), no data will be gotten.

[raidho36]

You are correct, the data stays in the buffer until you call getData. To explain input buffer overflow behavior in more comprehensible terms, when it fills up completely, whenever new sample is added at its head, one sample is discarded at its tail.

[Dr. Peeps]

You are correct, the data stays in the buffer until you call getData. To explain input buffer overflow behavior in more comprehensible terms, when it fills up completely, whenever new sample is added at its head, one sample is discarded at its tail.

Yes, I think I understand. With LÖVE's getData() then, if I called the function nearly immediately after the previous call (instead of waiting for a fresh buffer), you're saying the resulting buffer could be almost the same as the previous - but the whole thing might be moved over by one sample, with the first sample gone and a new one now at the end? Is that how it works?

I appreciate the explanations.

[raidho36]

No, it will simply return nil if it couldn't read from the input, which it can't unless it reports there being some samples buffered. When it does reads, it also automatically clears the buffer, so you never get the old data from it, it's always newly captures samples (if any). If you fail to read from it for longer than the buffer duration and it fills up, it will only keep the latest samples, however many can fit, with oldest being discarded.

[Dr. Peeps]

No, it will simply return nil if it couldn't read from the input, which it can't unless it reports there being some samples buffered. When it does reads, it also automatically clears the buffer, so you never get the old data from it, it's always newly captures samples (if any). If you fail to read from it for longer than the buffer duration and it fills up, it will only keep the latest samples, however many can fit, with oldest being discarded.

Ah, that does sound more useful.

[Dr. Peeps]

Code: Select all

inputs[1]:start(buffersize, 11025, 16, 1)
...
-- update loop
if inputs[1]:getSampleCount( ) == buffersize then
      data = inputs[1]:getData ( )
end

I've been trying to find the optimal sample rate / buffer size for my app, and I'm getting some interesting results. In the above code, if I set buffersize to 345 (or below), I will get audio updates 30 times per second, which is what I'd like to achieve. If I raise buffersize to 346, the updates drop to 20 times per second. (This is all based on an overall game update of 60 fps of course.) I wonder why 345 seems to be the cutoff, when 11025 / 30 is actually 367.5 samples.

Also, if appears that no matter how low I set buffersize, it will never fetch audio data as fast as 60 times per second .... Is there a minimum sample count that the software won't go under?

Obviously I'm getting hung up on fiddly numbers here, and I don't want to hardcode any of these values. I just want to ensure the optimal fetch rate of 30 times per second. Any thoughts on how to manage this?? I wonder if there will be differences across different hardware, as well.

Note that I'm not interested in doing any audio playback or recording here; I'm just monitoring the microphone and responding with visual feedback. The faster the response time, the better.

[raidho36]

OpenAL internally updates at under 50 frames per second. The amount of samples updated per frame is usually at cyclically repeating irregular intervals, e.g. 342 on odd frames and 558 on even frames.

Also you're reading the data out wrong. You're waiting on entire buffer to fill up before reading. What you should do is immediately read as soon as there is any input in the buffer. That should produce SoundData at the same rate as AL's internal update rate.