Furkan Gözükara
patreon
How To Convert 24 GB FP16 Fine Tuning / DreamBooth FLUX Checkpoints into 12 GB FP8 Checkpoints Tutorial
🕑 Added 2024-10-14 20:43:44 +0000 UTCAs you already know, FLUX Fine Tuning / DreamBooth is yielding the very best results compared to FLUX LoRA training. Here a detailed public post about it : https://www.patreon.com/posts/113970485
When you look the above post, you will see Fine Tuning / DreamBooth always yields way better results than any LoRA training.
Perhaps the biggest disadvantage of Fine Tuning is that its saved checkpoints are FP16 thus each one is taking 23.8 GB HDD space
But with using Kohya GUI, you can convert FP16 models into FP8 and there is no visible quality loss - the size gets down to half 11.9 GB
The first image above shows how to convert into FP8 and the second grid image shows there is no visible difference
Check all the images
And here how to do full proper Fine Tuning / DreamBooth as low as with 6 GB GPU : https://www.patreon.com/posts/112099700
Left one is FP16, middle is FP8 made with CUDA, right is FP made with CPU - no diff between CUDA and CPU
Comments
David Benollol
Is there a way to do all that programmatically using Kohya_ss?
Furkan Gözükara
It is your computer, your windows not allowing. Check your folder permissions python permissions
rockstaR05
Hi, when i try to merge model i got error: Traceback (most recent call last): File "E:\KOHYA_FLUXD\kohya_ss\sd-scripts\networks\flux_merge_lora.py", line 765, in merge(args) File "E:\KOHYA_FLUXD\kohya_ss\sd-scripts\networks\flux_merge_lora.py", line 626, in merge save_to_file(args.save_to, flux_state_dict, save_dtype, sai_metadata, args.mem_eff_load_save) File "E:\KOHYA_FLUXD\kohya_ss\sd-scripts\networks\flux_merge_lora.py", line 49, in save_to_file save_file(state_dict, file_name, metadata=metadata) File "E:\KOHYA_FLUX\kohya_ss\venv\lib\site-packages\safetensors\torch.py", line 286, in save_file serialize_file(_flatten(tensors), filename, metadata=metadata) safetensors_rust.SafetensorError: Error while serializing: IoError(Os { code: 5, kind: PermissionDenied, message: "Access is denied." }) 10:40:25-479873 INFO Done merging... Dreambooth training works fine with my kohya installation bot not merging, can you help with this?
Furkan Gözükara
not that i know any
Dmitrii
any converters to nf4?
Furkan Gözükara
yes you can merge into one via kohya gui
Tan
Thank you for this. I am not able to run the finetuned FP8 model in my 3090. However, I was trying to merge 4 Flux Lora together, but it was resulting in a full 23GB model. Is there a setting to use to merge multiple Flux Lora into one?
Furkan Gözükara
i tested this and it 4x vs 16x and it shouldnt make such diff. but you are training with higher resolution thus it will be slower
Furkan Gözükara
Robert if you are actively using PC other stuff you do may cause i think
محمد الذوادي
Sorry for the late reply, I used 24GB_GPU_23150MB_10.2_second_it_Tier_1 config on 1152x1536 images, and I found out that my GPU uses x8 PCIe lane not 16x because I have "4" 2TB m.2 on my motherboard and maybe this is why my gpu can't go full speed and why I can't use high batch on kohya, now I have to take off my custom water cooled gpu to remove all the m.2 and try the gpu on x16 speed :(
Robert Arsene
It varies. I usually get 6-7 seconds/it with RTX 4090, but someties jumps up to 14, even 20. Not sure what the factor is. Had 75 images at 300 epochs, 1 repetition, supposed to take 44 hours, but extended up to 60 due to variation in sec/it. I have a ryzen 3950 and 64Gb RAM.
Arcon Septim
Hello massed compute no a6000 available what is happening?
Furkan Gözükara
true i am looking for it :D
Furkan Gözükara
i have tested with 10800 images you can download checkpoints and see, i used our reg images from unsplash : https://huggingface.co/MonsterMMORPG/Big_FLUX_Fine_Tuning_Experiments/tree/main
st
we should be seeking for truth no matter whether it's in Kohya or not :)
st
I'm genuinely curious, how did that 11800 images finetune come out? Is it any good? I've never trained on more than 40 images.
Furkan Gözükara
Phoenix Fire yes your speeds are what we expect. great speeds
Phoenix Fire
Something's wrong with their configuration. I have the exact same CPU and GPU and I can lower blocks to swap down to 6 and get 5.7s/it. It increases to around 6.5s/it because I am doing 3x samples every 125 steps but their iteration shouldn't be almost 3x slow (5.7 vs 14).
Furkan Gözükara
yes cpu could be factor but 14 second / it is slow. people gets like 6 second / it with rtx 4090. which config you used?
محمد الذوادي
I get 20 sometimes 14 , everything sets right without any errors, maybe becuse i have 7950x3d cpu
Furkan Gözükara
this really helps. also your duration looks too much for 4090. you should get like 6 second / it
محمد الذوادي
Yesterday i finished finetune 11800 imges on 1 epoch , takes 42 hours on 4090 but the model is 22 gb and forgeui struggling with it , use 24gb vram and 50gb ram !! For 1 generation, I thought i waste my time, but i will try this method, Thank you
Furkan Gözükara
Because Kohya doesn't have it yet. I think ComfyUI might have but not sure
st
can you please elaborate on why fp8 and not Q8_0 gguf? Q8 seems to be closer to fp16 and is also 12GB, check this out: https://www.reddit.com/r/StableDiffusion/comments/1eso216/comparison_all_quants_we_have_so_far/?rdt=53989 From the comments to that post: https://imgsli.com/Mjg3ODgy/2/3
Furkan Gözükara
awesome
Jeroen Van Harten
This works great! Did a comparison in Photoshop to see if there's any difference. There is none.
Furkan Gözükara
you cant train but you can use trained model as fp8 there is absolutely 0 issues.
Daan Tilburg
So I can't train on fp8 dev model? Currently running dreambooth finetuning on fp8 model with Kohya, runs fine but result won't be any good then?
Furkan Gözükara
Maybe but I don't know how
Mime of Culture
is there a way to merge Vae + Clip into FP8 model too?
Furkan Gözükara
these are after training. fine tuning need fp16 base model and training is done in 16 bit precision
cool1
Do you mean you convert your checkpoint after it's saved/finished training from FP16 to FP8? There's a flux1-dev-fp8.safetensors and flux1-schnell-fp8.safetensors. Can a Flux Dreambooth (or Lora) be trained on either of those fp8 models while still being high enough quality, and will doing that use a lot less VRAM than training against the 22 GB flux1-dev.sft model (that I asume is fp16)?
