Furkan Gözükara
patreon
Comprehensive Different Version and Precision FLUX Models Speed and VRAM Usage Comparison
🕑 Added 2024-08-15 03:25:49 +0000 UTCI just updated the automatic FLUX models downloader scripts with newest models and features. Therefore I decided to test all models comprehensively with respected to their peak VRAM usage and also their image generation speed.
Automatic downloader scripts : https://www.patreon.com/posts/109289967
Patreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.
Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord
Please also Star, Watch and Fork our Stable Diffusion & Generative AI GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)
Testing Results
All tests are made with 1024x1024 pixels generation, CFG 1, no negative prompt
All tests are made with latest version of SwarmUI (0.9.2.1)
These results are not VRAM optimized - fully loaded into VRAM and thus maximum speed
All VRAM usages are peak which happens when finally decoding with VAE after all steps completed
Below tests are on A6000 GPU on massed Compute with FP8 T5 text encoder - default
Full tutorial for how to use locally (on your PC on Windows) and on Massed Compute (31 cents per hour for A6000 GPU) is at below
SwarmUI full public tutorial : https://youtu.be/bupRePUOA18
Testing Methodology
Tests are made on a cloud machine thus VRAM usages were below 30 mb before starting SwarmUI
nvitop library is used to monitor VRAM usages during generation and peak VRAM usages recorded which usually happens when VAE decoding image after all steps completed
SwarmUI reported timings are used
First generation never counted, always multiple times generated and last one used
Below Tests are Made With Default FP8 T5 Text Encoder
flux1-schnell_fp8_v2_unet
Turbo model FP 8 weights (model only 11.9 GB file size)
19.33 GB VRAM usage - 8 steps - 8 seconds
flux1-schnell
Turbo model FP 16 weights (model only 23.8 GB file size)
Runs at FP8 precision automatically in Swarm UI
19.33 GB VRAM usage - 8 steps - 7.9 seconds
flux1-schnell-bnb-nf4
Turbo 4bit model - reduced quality but VRAM usage too
Model + Text Encoder + VAE : 11.5 GB file size
13.87 GB VRAM usage - 8 steps - 7.8 seconds
flux1-dev
Dev model - Best quality we have
FP 16 weights - model only 23.8 GB file size
Runs at FP8 automatically in Swarm UI
19.33 GB VRAM usage - 30 steps - 28.2 seconds
flux1-dev-fp8
Dev model - Best quality we have
FP 8 weights (model only 11.9 GB file size)
19.33 GB VRAM usage - 30 steps - 28 seconds
flux1-dev-bnb-nf4-v2
Dev model - 4 bit model - slightly reduced quality but VRAM usage too
Model + Text Encoder + VAE : 12 GB file size
14.40 GB - 30 steps - 27.25 seconds
FLUX.1-schnell-dev-merged
Dev + Turbo (schnell) model merged
FP 16 weights - model only 23.8 GB file size
Mixed quality - Requires 8 steps
Runs at FP8 automatically in Swarm UI
19.33 GB - 8 steps - 7.92 seconds
Below Tests are Made With Default FP16 T5 Text Encoder
FP16 Text Encoder slightly improves quality and also increases VRAM usage
Below tests are on A6000 GPU on massed Compute with FP16 T5 text encoder - If you overwrite previously downloaded FP8 T5 text encoder (automatically downloaded) please restart SwarmUI to be sure
Don't forget to select Preferred DType to set FP16 precision - shown in tutorial : https://youtu.be/bupRePUOA18
Currently BNB 4bit models are ignoring FP16 Text Encoder and using embedded FP8 T5 text encoders
flux1-schnell_fp8_v2_unet
Model running at FP8 but Text Encoder is FP16
Turbo model : 23.32 GB VRAM usage - 8 steps - 7.85 seconds
flux1-schnell
Turbo model - DType set to FP16 manually so running at FP16
34.31 GB VRAM - 8 steps - 7.39 seconds
flux1-dev
Dev model - Best quality we have
DType set to FP16 manually so running at FP16
34.41 GB VRAM usage - 30 steps - 25.95 seconds
flux1-dev-fp8
Dev model - Best quality we have
Model running at FP8 but Text Encoder is FP16
23.38 GB - 30 steps - 27.92 seconds
My Suggestions and Conclusions
If you have a GPU that has 24 GB VRAM use flux1-dev-fp8 and 30 steps
If you have a GPU that has 16 GB VRAM use flux1-dev-bnb-nf4-v2 and 30 steps
If you have a 12 GB VRAM or below GPU use flux1-dev-bnb-nf4-v2 - 30 steps
If it becomes too long to generate images due to your low VRAM, use flux1-schnell-bnb-nf4 and 4 to 8 steps depending on speed and duration that you can wait
FP16 Text Encoder slightly increases quality so 24 GB GPU owners can also use FP16 Text Encoder + FP8 models
SwarmUI is currently able to run FLUX as low as 4 GB GPUs with all kind of optimizations (fully automatic). I even saw someone generated image with 3 GB GPU
I am looking for BNB NF4 version of FLUX.1-schnell-dev-merged model for low VRAM users but couldn't find yet
Hopefully I will update auto downloaders once I got 4bit version of merged model
Comments
Furkan Gözükara
true because main OS adds overhead. these are tested on a cloud thus it was already only 30 mb VRAM usage :)
Michael
Thanks for taking the time. I will say also for those using a 24gb card if that card is also running the main os's graphics you will sometimes get fast generation and sometimes you will hit speed bottlenecks.
