vercidium
patreon
Raytraced Audio Update - New Features and New Approaches
🕑 Added 2025-04-19 07:55:44 +0000 UTCPatreon limits videos to 8 Mbps. The full resolution video and 65 screenshots are available here
Sector's Edge is now using the latest Raytraced Audio system and it sounds incredible. There was no nice way to do this, I had to delete all the old audio code from Sector's Edge and connect it to the new system, one function at a time. My GitHub stats show 6071 lines added and 21519 lines deleted.
The environment is fully destructible + buildable, so you can test out different room shapes and sizes. You'll also be able to place sound sources around the map and trigger them via the UI or keyboard shortcuts.
In the bottom left you'll be able to see:
wind ambience direction
reverb stats - the diagram shows the size of the room (blue box), and how much energy is returning back to you (how full the box is)
how 'outside' you are
update time - how long it takes to re-trace all 960 rays and update reverb / ambience / muffling / permeation stats for all sounds
You can press escape and hover over active sounds to view graphs for:
LOS (green) - line-of-sight muffle strength
PER (orange) - permeation energy
You can also click the X to delete the sound, or click the lock icon to keep the graphs visible.
Patrons will get access to this demo first, but there's still a few more things to do:
ability to change maps and weather
improve ambience directionality - ability to hear rain from multiple directions
fly camera
setup sound sources and trigger them
As I'm working on this, I'm thinking of ways to improve (or completely replace) this raytracing system. I've outlined them below:
Ray Sets
There are some environments where the majority of the rays stay near the player rather than travelling outwards to discover more parts of the world:
One option to improve this is to use the results of the last raytrace to determine the 'best' and 'worst' rays, in terms of data gathered. For example if two rays are fired in a similar direction and they both discover the same sound sources and produce the same reverb information, then the 2nd ray is obsolete. The first ray should be given 2x the influence (since its doing the work of 2 rays), and the other ray should be fired in another direction.
The other option is to slightly randomise the reflection angle of each ray, and then over time pick the direction that caused the ray to cover the most distance.
Both options would require dividing the rays into two sets:
the 'base rays', these never change to ensure all directions are still checked
the 'supporting rays', which explore different paths based on the heuristics gathered by the base rays
As the environment around the player changes, the rays would then automatically adapt to cover the most ground possible.
Voxel Flood Fill
It's relatively expensive to cast green line-of-sight rays to calculate how muffled each sound source should be. With more active sound sources, more rays need to be cast, and the system runs slower.
One approach I've considered for a long time is to group sound sources together if:
they have line-of-sight with each other
they are close to each other
For example when firing a minigun, each bullet makes a sound when it hits a surface. Each bullet might collide with a different surface (the minigun is inaccurate), and each sound must be raytraced to determine how muffled it is.
I'm already grouping sounds together per-player, for example footsteps, gunfire and proximity chat all share the same raytracing stats because they originate from the same player.
But to improve this further, I'm considering a new flood fill approach, which is completely separate to the raytracing approach. Both approaches will be supported, so you can choose the best one for your game.
Each sound would be positioned on the voxel grid, and the flood fill would expand outwards from your player in search of them.
Each time it steps outwards from your player, the 'distance' number increases. When a sound is discovered, the current distance number is compared against the straight-line distance to the sound. In the diagram above, the green sound is 5 units away from your player, and the distance number is 5, and 5 / 5 = 100% clarity so the sound is not muffled.
But if there's something between you and the sound, it'll take more steps to reach it:
In this case it took 9 steps to reach the sound source, so it should be slightly muffled. The more steps it takes, the more muffled the sound should be.
This system would require tweaking, as the sound above shouldn't only be 5 / 9 = 55% clear, as that's a pretty extreme reduction given it's only a small wall between you and the sound. Here's a few more examples:
The same reverb stats as the raytracing approach (returning ray percent, returning ray length) can be calculated in this approach too. Every time the flood fill finds a non-empty block, it would check the current distance number against the straight-line distance from your player. If they're the same, then there's line of sight with your player, and the usual reverb stats can be calculated from that.
I'm not sure how permeation would work with this. Maybe a separate close-range flood fill could be used to calculate how much 'mass' there is between you and each sound source. I'm also not sure how ambience would work, maybe raycasting is needed for that.
In theory this approach should run faster, because less voxels are visited and we don't waste time trying to find sounds that are clearly enclosed in a separate room.
Vector Fields
Another potential improvement for the raytracing approach is to use vector fields to 'guide' rays away from dead ends.
The vector field will start empty (each point in the field would have no direction), and as a ray travels throughout the world, it'll alter/push the vector field direction to either the left or right. The next ray that travels along the same part of the vector field would then be slightly pushed to the side.
This is all guesswork - I'd have to run a simulation - but my theory is that over time the rays will spread away from commonly visited areas, and be forced to explore the path less travelled. This might even cause them to bend around corners.
Another approach would be to produce the vector field from the voxel data. Empty space would have no direction, and voxels would point towards the empty space, causing rays to be repelled like a magnet away from walls. This might just be a complicated version of the flood fill approach, but I'd like to test it anyway and see how it goes.
Miscellaneous
I faced a few strange rendering issues with textures when trying to upgrade SE to the latest Vercidium Engine version. The worst issues was flickering textures in the distance, as if they had no mipmaps. The problem ended up being the normal textures, but while I was debugging I ended up with this strange edge-detection effect. Not sure how it works but it might be useful for effects later on!
I also fixed a memory leak (the game allocated 300,000 buffers for indirect rendering) and cleaned up a lot of other messy engine code. I'll merge that into the main branch once things are stable.
So far 1156 people have filled out the https://vercidium.com/audio form! I'll share the survey results here soon.
Comments
Vercidium
It's not out of the picture, I haven't used compute shaders before so I'm keen to try them out. There's just a lot more to try / test / optimise before that
J19
Is a compute shader really out of the picture here? I mean, if it runs fast enough as is, that's wonderful. I'm just always tempted to jump to compute shaders lol
Vercidium
Thank you, yeah the reverb needs to be fixed. Until I have materials working I'll allow you to: - pick the reverb presets for small rooms / large rooms / outdoors - control the formula for interpolating between presets based on: - the percentage of rays that return - the length of the returning rays I'm currently using the 'factory_smallroom' and 'dome_saintpauls' presets which have a lot of echo. I should be using stone presets here as most of this map is concrete.
Max Quagliotto
This looks very promising! Looking forward to the progress of raytraced audio. One question though -- I notice that the environment has a LOT of reverb, even with there being some parts where you're partially walking outside in some parts. That seems to be an issue still with many games in my opinion. Is it because of the materials on the walls and their absorption coefficients? I would think that many other materials would have slightly higher absorption so as to reduce the RT60 in an environment.
w9w9bc
I would love to see this feature released and commercialized in the form of an Unreal version, a WISE version. I'll be rooting for you.
Vercidium
I’m aiming to have this demo ready for people to try next weekend. Then the next step will be building a C# DLL for it, and send it to a handful of devs and see how the integration goes This first DLL version won’t handle the underlying audio API (OpenAL, fmod, etc). Instead it will return the muffling / permeation / etc properties for each sound, that the devs can then use to apply their own filters to their sounds. That way I’m not diving into unreal/Godot specifics yet, and I can focus on the raytracing integration first
w9w9bc
Cool, when will I be able to share it? I'd love to test it out.
