Microphone Support for LÖVE!

[Noacos]

Hello. I'm curious how could I get the microphone audio levels (sorry for my terminology) to use (for example) for a visualizer.

[vrld]

I'm assuming that by "audio level" you mean the loudness of the sound bites. To understand how this can be computed it is worth looking into how that sound bite is represented. With math.

So: a [wiki]SoundData[/wiki] object represents a short signal that describes the vibration of the microphone membrane (which is basically all that sound is: vibration over time) at any given time. Mathematically, we have a function that maps a time to the amount of vibration: sound(time) = amount of vibration, or shorter: \(s(t) = v\). Note that v can be positive or negative, depending on whether the membrane swings up or down. A SoundData object basically stores the values of s(t) for discrete values of t (for example t=0, t=0.00002, t=0.00004, ...)

The loudness of a sound can be measured by the Amplitude, which wikipedia defines as "a measure of its change over a single period". This isn't really helpful though, and indeed there are many definitions of how the change is actually measured. In my opinion, "Peak amplitude" and "Root mean square amplitude" are the most suitable, so we will use that.

Peak amplitude is the maximum absolute value of the signal, that is: \(\text{peak amp} = \arg\max_t |s(t)|\)
In Code:

Code: Select all

function peakAmplitude(sounddata)
    local peak_amp = -math.huge
    for t = 0,sounddata:getSampleCount()-1 do
        local amp = math.abs(sounddata:getSample(t)) -- |s(t)|
        peak_amp = math.max(peak_amp, amp)
    end
    return peak_amp
end

Root mean square amplitude is "defined as the square root of the mean over time of the square of the vertical distance of the graph from the rest state." In our case, we can assume the "rest state" to be 0; then the distance is just the value of the signal. Mathematically this means: \(\text{rms} = \sqrt{\frac{1}{T}\sum_{t=1}^T {(s(t))}^2}\) - we have to sum the square of every value, divide by the number of items and then take the root:

Code: Select all

function rmsAmplitude(sounddata)
    local amp = 0
    for t = 0,sounddata:getSampleCount()-1 do
        amp = amp + sounddata:getSample(t)^2 -- (s(t))^2
    end
    return math.sqrt(amp / sounddata:getSampleCount())
end

Now, if you want to know the volume distribution across different frequency bands (as seen in equalizers), you will need to process the signal with a Fourier transform. But this is a rather complicated topic...

edit: forgot the mean part in root mean square

[Noacos]

Thank you for your quick answer. So I read your answer few times, I have done a small research and for now I just store absolute value of the first sample. I don't need an equalizer, I just want a game using similar mechanics as http://kanako.dk/clients/fun/WOOORRK!/. Do you think that this is a good approach?

[Manyrio]

Hey I just discovered this awesome library.
I find it very cool ! I have just one question, after playing with it during the last hours, I wanted to know how do we get the sounddata created by the library ?
(Sorry for bad english, I'm french)

[zorg]

Hi,
as you can see from this example, for example, the callback's second parameter, data, is the SoundData.
The poll method creates them, as seen here.

[Dr. Peeps]

I'm just looking at the new RecordingDevice class implemented in 0.11 (and excited about it!). Does anyone have some simple example code? The link given doesn't seem to contain anything (http://hastebin.com/revohalixi.lua). Thanks in advance.

[raidho36]

It used to - but it's gone. You'll notice the link redirects you to the front page, which is a blank paste.

Code: Select all

local inputs
local output

function love.load ( )
	inputs = love.audio.getRecordingDevices ( )
	print ( inputs[ 1 ]:getName ( ) )
	inputs[ 1 ]:start ( )

	output = love.audio.newQueueableSource ( inputs[ 1 ]:getSampleRate ( ), inputs[ 1 ]:getBitDepth ( ), inputs[ 1 ]:getChannels ( ) )
end

function love.update ( )
	if inputs[ 1 ]:getSampleCount ( ) > inputs[ 1 ]:getSampleRate ( ) / 20 then
		local data = inputs[ 1 ]:getData ( )
		output:queue ( data )
		print ( ( "%d samples captured" ):format ( data:getSampleCount ( ) ) )
		if output:getDuration ( "samples" ) > data:getSampleCount ( ) then
			output:play ( )
		end
	end
end

[Dr. Peeps]

Code: Select all

	inputs = love.audio.getRecordingDevices ( )
	inputs[ 1 ]:start ( )

Thanks! Seems to work.

Hmm, the source lists RecordingDevice:startRecording() as a method, but that's not called in this example - just RecordingDevice:start().

[zorg]

It may have been renamed. Check the merged pull requests.

[Dr. Peeps]

getData removes currently recorded samples from input and puts them into new SoundData

This works well, but .... What is the likelihood of having a variant of getData that doesn't create a new SoundData object with every call, instead "refilling" an existing buffer with new data?

This is only in the interest of keeping object creation / garbage collection / memory fragmentation down. (In the above example I can see that data pointer climbing ever higher the longer the program runs ...) In my case, my app is constantly monitoring the microphone (not just recording it for a short period), so creating the same object over and over seems wasteful ... but I'm new to both LÖVE and Lua and perhaps don't know what I'm talking about.