[hwsmm]

This PR lets the framework use SDL3 as an alternative audio backend. It requires no change to the game (osu!).

To use it, change AudioDriver to SDL3 in framework.ini.

Limitation:

• It is capable of using FFmpeg as a decoder, but currently BASS is being used as a decoder since FFmpeg bundled with the framework lacks audio decoders.

[smoogipoo]

This is super cool, though I don't see osu! using this in the near future if ever. I'd see this implemented as an additive nuget package, and exposed through some way that isn't the framework config. For example, it could be included in HostOptions or simply as a virtual method in Game.

[peppy]

I'd actually like to experiment with this and see how good support is for the upcoming changes I want to make with WASAPI initialisation. So I don't want to throw this out. Moving away from bass would be a huge consideration, but I wouldn't throw it away.

I'm going to mark this as a draft as I don't see it getting reviewed or merged anytime soon, but I still useful to have around as a reference for what is involved in making this work, and potential performance / latency cross-checking in a future.

[hwsmm]

I'll maintain this until some of you have anything to do with this, mostly because I am now so used to playing the game with these patches, I can't really go back to BASS...
FWIW, on Linux with Pipewire at very small buffer, hitsound latency was almost on par with patched wine which osu! linux players mostly use. It produces some artifacts, so shouldn't be default, though.

[peppy]

@hwsmm thanks for keeping this up-to-date. curious on your thoughts on the move to sdl3 – does this help in any way on the audio side of things? i don't want to promise anything but we have internally discussed the potential of moving to sdl3 as an audio provider at some point.

curious on how wasapi is looking specifically, as i read that sdl3 was going to add better support, potentially replacing my attempt at initialising for lower latency (#6088) which ended up having compatibility issues and limited results.

[hwsmm]

SDL3 Audio doesn't give any improvements over SDL2 for the usecase of osu!framework in my opinion.
From what I see, improvements in SDL3 audio is mainly to let developers reduce burden of writing audio subsystems by themselves, and my implementation already has mixers and everything, so they are kind of meaningless here and even add a bit of overhead in some cases.

SDL has a good WASAPI implementation, but they don't provide any means to adjust buffer sizes (https://github.com/libsdl-org/SDL/blob/main/src/audio/wasapi/SDL_wasapi.c#L635 -- only uses default '0' value).
Buffer length usually ends up being around 10ms (or more, should be plenty good?) on most systems I tested on. I am not sure how better it could be compared to BASS DirectSound.
We can make use of this API to determine lowest possible latency, but it's only available on Windows 10 or higher. I can probably write a PR (or a patch for our SDL builds?) to SDL for this if anyone wants, but I guess you can also just implement it within C# and pass the value to BASSWASAPI.
Though I didn't know such API existed until now. I don't really play osu! on Windows, anyway..

However, the problem that hinders real low latency lies in .NET GC overhead. As I wrote in the PR description and the previous comment, audio stutters a lot if I force low latency on Linux, and it should be the same on Windows, too.
Any value below 10ms seems to cause underruns in audio, and it's very noticeable if you go back to main menu from song select in game.
I remember that the same happened in your WASAPI testing discussions on Discord, and this is probably because WASAPI callback exists in managed land.
I even wrote a simple C library out of boredom to handle SDL audio callback in unmanaged land, and it solved most of audio stutters issues on very small buffer on Linux (around 1ms), but you won't want this since it requires a couple of additional native libraries.

So what I am trying to say here is, that even with the Windows API above, it's going to be pretty hard to get the stable low latency audio experience unless audio is handled in completely unmanaged land.
The best solution might be making a native library for BASSWASAPI callback if you only consider Windows, but I would like to see any kind of support for Linux since there are some players (including me) not wanting to make a complete switch to lazer because patched WINE provides better latency.

---

... if you ask me what's the point of this PR then:

1. Better support for Linux audio servers
2. Audio is at least fine in gameplay at very low latency, it's not so good in song select, though.. (Not saying that audio is always bad, it sounds nice around 10ms)
3. Can get rid of BASS, a closed source library, without introducing new native libraries except for libswresample, which is a part of ffmpeg

[smoogipoo]

However, the problem that hinders real low latency lies in .NET GC overhead.

That's, honestly, the first time I'm hearing of this. I would imagine that code returns in a fraction of a microsecond so I wouldn't expect the GC to have that sort of impact on it, but I suppose it would make sense for super small buffer lengths.

If it is GC and not e.g. the cost of context switching, there's a few things I'd try:

• Wrapping in GC.TryStartNoGCRegion()/GC.EndNoGCRegion()?
• Using UnmanagedCallersOnly.
• Using SuppressGCTransition.

Perhaps, all at the same time.

[hwsmm]

I guess my comment was a bit ambiguous. I meant that the audio callback returns fast, but the problem is supposedly 'stop-the-world' effect of GC.
Whenever STW occurs, SDL fails to get audio data in time from the audio callback located in managed land.
Please correct me if I misunderstood your message, I have only used C# for less than an year at this point.

Here are my test results for things you mentioned:

1. GC.TryStartNoGCRegion()/GC.EndNoGCRegion() pair mostly fixes the problem, except for longer stutter from time to time.
2. UnmanagedCallersOnly is already applied to the audio callback.
3. SuppressGCTransition doesn't seem to help. Only one native method is used in the callback, anyway..

[TheComputerGuy96]

It would be nice to combine all of this audio component work and make a new open-source BASS-compatible audio library
that everyone could use (not just C# applications) in a separate repository (I'm personally going to call it Treble until a better name can be found)

[hwsmm]

First of all, this PR is not compatible with BASS. It just implements o!f audio components in C# and SDL, while the current o!f audio backend is written in BASS API.
Which means, this PR calculates current track position by converting data position to milliseconds in C#, but current o!f calls BASS_ChannelGetPosition (and some more) to do the same.
It would be very time consuming to re-expose internal APIs in a new project and to make it compatible with BASS and its addons (o!f uses BASSMIX, BASSFX and BASSWASAPI), and BASS is still maintained well.

I do love open source, and it's true that I wrote this PR to get rid of a closed source lib from osu!framework, but there is only little point to both audio library consumers and us in maintaining a new audio library when there are mature open-source audio libraries out there. Better contribute to them if they are missing some features. It could be another example for xkcd new standard in my opinion.

Just consider this as a random audio backend for a game engine. Unity, Unreal and Godot, they all have their own audio backend, but they are not standalone, right?

[hwsmm]

I've just sent a mail!

Forgot to add in the mail. I am planning to implement decoder in .NET side, because it adds a bit of complexity and may cause link problems with FFmpeg.

[hwsmm]

Starting from https://discord.com/channels/188630481301012481/589331078574112768/1396539637861847141, I copied most part from this PR to the new library, and I got most things working... on my PC.

As a side note, I was actually a few hours before (pre-)releasing my native library here when smoogi sent me that message since it had been pretty stable lately, so I was not motivated to write it again in NativeAOT, but I ended up deciding to do because it required much less effort because I already had some in C#.

I honestly got surprised with Satori. When I finished the basic implementation and tested with plain NativeAOT, the result was not good on very small buffer (around 1ms).

However, with Satori, it still had some underruns, but was very acceptable than plain NativeAOT and this PR.

Anyway, here are some points to know:

• It is LGPL because SoundTouch.Net (LGPL v2.1) is statically compiled.
• BASS is the default decoder because it needs to work for now. It can also use FFmpeg 7 with libswresample and audio decoder if it is available and BASS init code is removed from SDL3AudioManager.cs.
• Decoder is integrated because I want to keep API stable for my another native library (not public yet).
• Android probably works (not tested yet), but no iOS because I don't have any Apple devices. Too cumbersome to figure out xcframework when I can only use CI for Apple toolchain.
• Satori is not available on macOS x86_64 (due to build failure - this upstream issue suggests that the issue has been fixed in new XCode 16 releases but no luck for me) and Android (don't know how)
• Repository structure is pretty rough right now. I may fix in the future, but it works for now (repo).
• You can set an environment variable SDL_AUDIO_DEVICE_SAMPLE_FRAMES to adjust latency. 128 should work well on Windows with a good audio driver.

How to try:

1. git clone -b sdl-audio https://github.com/hwsmm/osu-framework.git
2. Apply this commit on top of upstream osu
3. cd osu && ./UseLocalFramework.sh
4. cd osu.Desktop && dotnet run
5. Edit AudioDriver in framework.ini to SDL3

This thing is still very early, so it may be (completely) broken in some cases. I'd be grateful if you can try and let me know if it works well.

[hwsmm]

@smoogipoo I am not expecting an immediate response, but here is a mention in case you didn't enable notification on this thread.

[smoogipoo]

This is still in my emails but very low priority at the moment. I've been waiting on a conclusion to the discussion in #705 + #6628 because @peppy's already reviewing that.

[Rudicito]

@hwsmm I tested your changes. Note that I'm using 2025.816.0-lazer, not the latest version, because your framework is a bit outdated.

I'm on Linux (Fedora 42).

I tried with this beatmap. Note that I’ve set a 10 ms audio offset, since that’s what I use in my regular lazer client, that I deactivate beatmap hitsound, and that I use argon pro.

With BASS:

With SDL3 (without set SDL_AUDIO_DEVICE_SAMPLE_FRAMES):

With SDL3 (SDL_AUDIO_DEVICE_SAMPLE_FRAMES = 128):

With SDL3 (SDL_AUDIO_DEVICE_SAMPLE_FRAMES = 64)

For SDL3 and SDL_AUDIO_DEVICE_SAMPLE_FRAMES set to 64 and 128, the hitsound feels more responsive, feel like I was often clicking early. Didn't encounter audio issues with these values. In the editor, I feel like the hitsound are early.

With SDL_AUDIO_DEVICE_SAMPLE_FRAMES set to 32, sound start crackling, didn't try playing a beatmap.

Hope this help, you can ask me to test things if you want.

Maybe I should have test with no audio offset? I can do it if you want.

[hwsmm]

@Rudicito Hey, thanks for testings! Reporting issues you encounter while using it is sufficient for now (you may use whatever offset you want). sdl-audio-alt branch is now updated, so you can probably update your game.

[unknowwiiplayer]

The following was written with AI assistance based on my own testing and findings.

I am not an audio developer, so I cannot evaluate the architectural or maintenance concerns involved in merging this work. My contribution here is empirical: I am an experienced taiko player, and input-to-hitsound latency is the main reason I normally avoid osu! and prefer OpenTaiko with ASIO on Windows.

I tested the newer sdl-audio-alt work against osu! 2026.711.0-lazer on CachyOS with PipeWire.

To get the current game compiling, I used:

• the sdl-audio-alt branch of hwsmm/osu-framework
• the existing AudioFilter compatibility patch
• a small additional adaptation for the newer effects in AfToggleSection.cs

On the osu! side, the additional adaptation was small: six newer AddEffect() / RemoveEffect() calls were migrated to GetNewEffect(), Apply(), and Remove().

The result compiled successfully.

I was then able to play stably with PipeWire running at a 48-frame quantum at 44.1 kHz.

The gameplay difference was not subtle. Input-to-hitsound response felt dramatically tighter, and osu! felt better to me than it ever has before, including my experience playing on Windows. I only play taiko, so that is the mode I can confidently evaluate.

I understand that a working local build does not address all of the architectural, maintenance, mobile, or cross-platform concerns involved in upstreaming an alternative audio backend.

However, it does demonstrate that the newer SDL3 audio work can still run on the current game, and that its practical benefit for latency-sensitive Linux players can be enormous.

I would really like to see this work revisited and given a concrete path toward an officially supported low-latency audio option, even if it initially remains experimental or Linux-only.

I can provide the exact osu! diff and testing details if they are useful.

[hwsmm]

I am sorry, but osu! project does not accept any kind of AI assistance (including documentation things). Please take a look at this section: https://github.com/ppy/osu-framework#contributing

I just updated my patches in the sdl-audio-alt comment. It's not a big deal anyway, since I still play the game with my backend and fix issues as I find them.

Also I moved the old branch to sdl-audio-legacy, since I have not been touching it for long time.