More granular control of music loops and using tracker files

[manadream]

Hello all,

I've been developing on Love for the last 6 months or so and I love it (no pun intended). However, I have so far been unable to accomplish what I am wanting to do in regards to controlling music, specifically controlling the looping of it, and I am also wondering if anybody knows of a way to use tracker files (.xm in this case) with more control such as being able to turn on and off tracks, etc.

So with the looping issue, I have tried what various other people have said in this forum, which is to use :tell and :seek on the music source object to get and set the play position in order to control how the music loops. The issue I have with this is that it is not fine grained enough for me to get the music to loop perfectly. What I have noticed is that the :tell("samples") call gives me samples in 1024 increments instead of the exact value at the time that I call the function. So I'll get something like this when I call it each frame (notice the repeated values):

2046
2046
3070
3070
4093
5117
5117
6140
7164
7164
8187
9210
9210
10234

This is not fine grained enough for me to control the music precisely and I get skips and jumps and overall inconsistent looping when I try to loop based on samples. So my question is, why is this coming back in these increments? Is there a way to increase the granularity of the value returned? This is when playing from a wav file.

On the same topic, I create my music in a tracker, and so I have .xm files that I can play the music from, but it doesn't seem to respect the loop settings saved in the xm file, it just loops the entire thing instead of looping back to the pattern specified in the tracker file. So I still have the same problem with loops here. I am also wondering if anyone knows of a way to control tracker things in love like muting tracks or other things one could do with a tracker.

Basically, what I am trying to do falls under these two categories:
1. Have songs with intros that don't loop the intro
2. Have songs with multiple parts that the game transitions to based on what's happening (i.e. a song with a chill part and a part that is more intense when you encounter enemies, both of which have transitions between the parts so the music flows from one mood into the other smoothly)

Anyone have any input as to how I can accomplish this?
Thanks in advance!

[raidho36]

You can manually decode your audio stream and feed it into a queueable source, to produce seamless transition between different points on the track. As for tracker music formats, LOVE uses a simple tracker player which doesn't have many features. You can however hook up a different player via FFI.

[zorg]

First of all, :tell depends on your vsync setting since there's only one "game loop", so if that's enabled, you can't get the granularity you could otherwise; but even with that, there's no guarantee you can exactly control the loop down to the samplepoint with :tell and :seek only.

As raidho said, using a short sounddata as a buffer (and a decoder since you don't want to load your whole song into memory as uncompressed samplepoints), and a queueable source, you can, since you need to manually go over all samplepoints, and when you reached a specific smp count, which is the loop end point, you can set it back to the loop start point (with seeking the decoder back there) and it will work completely seamlessly.

As for more fine grained control with tracker formats, while löve does use libmodplug, which would allow such things, it's not utilized by löve due to how the devs wanted the implementation to basically mirror the non-notational formats as well, effectively treating all sound formats the same, like if they were bitstreams.

That said, tracker file loop points should be respected, and if they're not, then either libmodplug has a bug, or your xm file has the looping defined wrong.

On a final note, i've been working on an app, soon to be a lib, that can load and play back tracker files with more options, although it only supports s3m files at the moment (and cave story/organya files)... it's not as correct as libopenmpt though (which löve should really update to ), but it works.

[manadream]

Thanks to you both for the information. It was very helpful.

So I modified the code in this thread to meet my needs for looping, and that worked perfectly.

Code: Select all

mSource = love.sound.newDecoder('path/to/music/song.mp3', 2048) -- Default, but this should be smaller, imo.
qSource = love.audio.newQueueableSource(mSource:getSampleRate(), mSource:getBitDepth(), mSource:getChannelCount(), 8) -- default
smpCounter = 0
yourLoopPoint = 6942804

-- in update
if qSource:getFreeBufferCount() then
    local Buffer = mSource:decode() -- You'll get a 2048 (by default) sized SoundData
    if yourLoopPoint == smpCounter + Buffer:getSampleCount() then
        -- queue and seek
        qSource:queue(Buffer)
        mSource:seek(0) -- or whatever your starting point would be
        smpCounter = 0
    if yourLoopPoint > smpCounter+Buffer:getSampleCount() then
        -- just queue the whole buffer
        qSource:queue(Buffer)
        smpCounter = smpCounter + Buffer:getSampleCount()
    elseif yourLoopPoint > smpCounter then
        -- only queue part of the buffer, and then seek to the starting point (or to the loop start if you have such a thing) with the Decoder.
        local temp = love.sound.newSoundData(yourLoopPoint - smpCounter, mSource:getSampleRate(), mSource:getBitDepth(), 
    mSource:getChannelCount())
        for i=0, (yourLoopPoint - smpCounter - 1) do
            temp:setSample(i, Buffer:getSample(i))
        end
        qSource:queue(temp)
        mSource:seek(0) -- or whatever your starting point would be
        smpCounter = 0
    end
end

However, I have another problem, which I realize I didn't fully specify I was trying to do this in the initial post. With this method, I can't seem to seek forward in the docoder. What I am trying to do here is I have a wav file that has two different parts that I want to loop and transition between. So the main song is from the beginning to a certain sample, and looping that is no problem. But when I want to seek the decoder past the loop point into a part of the wav file that has the next section of music, it returns a nil buffer when I call :decode after calling :seek on the decoder. Does the decoder need to have decoded up to that sample point before it can decode the samples after that point?

Also, in regards to this:

That said, tracker file loop points should be respected, and if they're not, then either libmodplug has a bug, or your xm file has the looping defined wrong.

I double checked this and yeah, it does not seem to be respecting my loop points in the xm file. I created the file in MilkyTracker, which is a really popular tracker, so I have doubts that the file it creates does not have looping defined correctly. But perhaps that is the case. Do you have a tracker that you have used and experienced the looping work with? I tried playing the xm file with :setLooping on true and false. True just loops the whole thing, not respecting my loop points, and false stops at the end of the song where it would normally loop in the tracker. So it treats it like a normal audio file essentially.

Thanks again for the help!

[raidho36]

I can't seem to seek forward in the docoder.

Have I seriously forgot to add seek to decoder? No, it can't be.
*double checking the source code*
*crafting a demo to make sure*
Decoders have ":seek" function, works the same* as for streaming sources (they're basically a decoder + qsource but transparent to the game code). The wiki somehow doesn't mention it, it's an oversight in documentation. That said, you'd have better luck with separate files that have exactly defined starting and ending points; decoders can't guarantee that the playback will start at exactly specific sample you seek to.

Well, I just created a wiki page for Decoder:seek so that's that.

*the same except you can't specify "samples" as timing unit - decoders don't accept positions in samples. Steaming sources convert samples to seconds when you seek by sample. This difference is probably a development oversight. By which I mean it's totally my fault. But nobody called this out in however long that been a thing so yeah.

[manadream]

Oh, I see. Then my problem is that I'm calling seek with samples instead of seconds. That's super helpful to know, thanks. I'll change that and see how it works.

[raidho36]

Also mind the OpenAL's internal refresh rate of 50 Hz. This puts a lower limit on how short a buffer you can use with queueable sources. Since manual playback is affected by FPS hiccups which can cause buffer underflow and stutter, it's advisable to put that into a separate thread running at a consistent slow speed.

[pgimeno]

Decoders have ":seek" function, works the same* as for streaming sources

Couldn't it support specifying the seek position in samples, just as streaming sources do, for consistency?

[raidho36]

Couldn't it support specifying the seek position in samples, just as streaming sources do, for consistency?

I suppose it could, if someone implemented it. It should be just a few lines of code, most of which can be copy-pasted from /modules/audio/openal/source.cpp. But, again, decoders internally don't support this, so beyond maintaining consistency between unrelated APIs there's no reason to do it, so there isn't any kind of pressure.

[zorg]

The issue is probably that some codecs might encode into formats that aren't exactly samplepoint-accurate (e.g. mp3), so you can't really seek to a specific point (and even if you could, it would take extra internal logic to decode a larger chunk, and either count the smp-s and memcopy a slice of the array into another memory location that'd be returned, or return extra parameters denoting the extra cruft on both ends of a buffer... both of which would make things more complicated.

(Which manadream already mostly coded in, in their reply to my previous post...)

Anyhow...

However, I have another problem, which I realize I didn't fully specify I was trying to do this in the initial post. With this method, I can't seem to seek forward in the docoder. What I am trying to do here is I have a wav file that has two different parts that I want to loop and transition between. So the main song is from the beginning to a certain sample, and looping that is no problem. But when I want to seek the decoder past the loop point into a part of the wav file that has the next section of music, it returns a nil buffer when I call :decode after calling :seek on the decoder. Does the decoder need to have decoded up to that sample point before it can decode the samples after that point?

Usually, seeking a bitstream format will net you what you want (mp3, ogg, wav), that is, you can seek in both directions, and when you :decode, it should decode from approximately where you seeked to (with tracker formats, it depends on libmodplug), so it's a bit strange that for a wav file, if it fails to work...

I double checked this and yeah, it does not seem to be respecting my loop points in the xm file. I created the file in MilkyTracker, which is a really popular tracker, so I have doubts that the file it creates does not have looping defined correctly. But perhaps that is the case.

Do you have a tracker that you have used and experienced the looping work with? I tried playing the xm file with :setLooping on true and false. True just loops the whole thing, not respecting my loop points, and false stops at the end of the song where it would normally loop in the tracker. So it treats it like a normal audio file essentially.

If your xm file has the correct order number at 0x42 hex (66 decimal) in the file, as two bytes, then it saved the restart position; I wrote a test module in OpenMPT, which also saves that correctly, but löve/libmodplug ignores it, it seems.

Interestingly enough, the same test module exported to .it, .s3m and .mod formats did loop correctly, so it's just xm files for some reason...

(More and more i feel like i should fork the source and replace libmodplug with libopenmpt; at least that's actively developed and is way more accurate than what is currently used... maybe when löve's on github)