vercidium
patreon
Modern Dithering with OpenGL
🕑 Added 2024-09-20 13:20:30 +0000 UTCI revisited the low poly forest renderer and noticed there was terrible colour banding at night time:
If you're having trouble seeing it, here's the same image with the contrast increased:
This looks terrible, and ruins the immersion when large bands of colour move across the screen (it looks worse in real time).
This happens because most games and monitors use 8-bit colour. This means each red, green and blue component of a colour can store a value in the range 0-255. You may have seen colours represented in this format, e.g. RGB (50, 112, 255).
This might not sound like many colours, but 255 x 255 x 255 = 16.5 million unique colour combinations can be made. So why does the image above have these ugly bands of colour?
Colour banding is most visible in dark monochrome environments. If the brightest pixel at night time is RGB (0, 0, 7) and the darkest pixel is RGB (0, 0, 0). That means we only have 8 different shades of blue to work with.
If we interpolate from 0 to 7 across a 1920px monitor, we get 8 distinct bands of colour:
I've increased the brightness and contrast here to make the bands more visible
Games that use pixel art create the illusion of gradients using a technique called dithering. This technique uses the same amount of colours (in this example 8 colours), but arranges them in a specific pattern:
This illusion works best when the pixels are small. If we look at another close-up example, we can see the pattern and the illusion is broken:
The Problem
It's difficult to apply this dithering pattern to real-time rendering because the final result is composed of many different colours that are arranged in unpredictable patterns. In the images above we know ahead of time that we're blending between black and blue, which is why we can arrange the pixels in this pattern.
But 3D games often have lighting, fog and other postprocessing effects applied afterwards, so the colour of the final image can't be predicted.
We can use a post-processing dithering shader to apply dithering the final image, but it is expensive as it requires sampling pixels in a large radius around the current pixel.
However, we can fake dithering using a different approach.
The Solution
When rendering a 3D game, you can either render directly to the screen, or to an off-screen framebuffer.
A framebuffer is a technical term for an image that we can render 3D objects to. When we create a framebuffer, we can control how many colour components it stores (e.g. R, RG, RGB, RGBA) and how many bits each colour component has (e.g. 8, 16, 32).
If we were to render directly to the monitor, we'd be stuck rendering to an 8-bit RGB image because that's the only format the monitor can display. But OpenGL offers over 300 different formats, and they each have their own use case:
RGB8 - the most common format, it stores 8 bits for each RGB component
R16 - a greyscale image with 16 bits. I use it for dynamic exposure, so each pixel can store a higher-precision brightness value
RG8 - the red and green components represent pitch and yaw. I use it for normal maps
For our framebuffer, we'll use the RGB16 format. 16 bits are stored for each red, green and blue component, which means 65536 unique values can be stored in each. That's 8x more precision than an 8-bit number, and can be combined to create 65536 x 65536 x 65536 = 281 trillion unique colours. That's 16975370x more colours!
Sadly - unless you're using a HDR monitor - we'll have to convert it back to 8-bit colour when displaying it on the screen. So we're back at square one with the same colour banding:
16-bit image converted to an 8-bit image (so you can view it on your monitor)
The trick to solving this is to add random noise to the entire image.
This noise slightly adjusts the value of each colour in the 16-bit framebuffer. When mapping these 16-bit colours down to 8-bit colours, the magic happens and this dithering effect appears:
Contrast is increased here so the dither pattern is visible
Here's the fragment shader, which reads from the 16-bit framebuffer via the dither_sourceTex sampler, and writes to your monitor via the gColor variable.
If we tried this with an 8-bit framebuffer, the effect wouldn't work. We'd just end up with bands of colour with noise over the top:
The magic happens when squashing 16-bit colours down to 8-bit colours.
Final Result
Here are some before-and-afters of 8-bit colour vs 16-bit dithered colour. The code is available on GitHub for paid Patrons. Run the forest project on the main branch to see it in action. The key files are:
DitherShader.cs applies the random noise
TextureCollectionScene.cs creates the RGB16 framebuffer
The BlitToScreen function in SequenceForestRender.cs controls the dither strength
These images are best viewed in full-resolution on Dropbox, as the banding re-appears when Patreon resizes and compresses the images.
These images are best viewed in full-resolution on Dropbox, as the banding re-appears when Patreon resizes and compresses the images.
Comments
Vercidium
Thank you, that means a lot to hear
Trace
i can't tell you how much i appreciate how detailed you are, i can see the effort
