Furkan Gözükara
patreon
Full Fine Tuning / DreamBooth of FLUX yields way better results than LoRA training as expected, overfitting and bleeding reduced a lot, check oldest comment for more information, images LoRA vs Fine Tuned full checkpoint
🕑 Added 2024-09-16 10:47:40 +0000 UTCFull Fine Tuning / DreamBooth of FLUX yields way better results than LoRA training as expected, overfitting and bleeding reduced a lot, check oldest comment for more information, images LoRA vs Fine Tuned full checkpoint
Configs and Full Experiments
Full configs and grid files shared here : https://www.patreon.com/posts/kohya-flux-fine-112099700
Quality 1 Config better than Quality 2
Quality 2 is only for low VRAM GPUs
Always use higher VRAM config if you are on higher VRAM machine
Details
I am still rigorously testing different hyperparameters and comparing impact of each one to find the best workflow
So far done 16 different full trainings and completing 8 more at the moment
I am using my poor overfit 15 images dataset for experimentation (4th image)
I have already proven that when I use a better dataset it becomes many times betters and generate expressions perfectly
Here example case : https://www.reddit.com/r/FluxAI/comments/1ffz9uc/tried_expressions_with_flux_lora_training_with_my/
Conclusions
When the results are analyzed, Fine Tuning is way lesser overfit and more generalized and better quality
In first 2 images, it is able to change hair color and add beard much better, means lesser overfit
In the third image, you will notice that the armor is much better, thus lesser overfit
I noticed that the environment and clothings are much lesser overfit and better quality
Disadvantages
Kohya still doesn't have FP8 training, thus 24 GB GPUs gets a huge speed drop
Moreover, 48 GB GPUs has to use Fused Back Pass optimization, thus have some speed drop
16 GB GPUs gets way more aggressive speed drop due to lack of FP8
Clip-L and T5 trainings still not supported
Speeds
Rank 1 Fast Config - uses 27.5 GB VRAM, 6.28 second / it (LoRA is 4.85 second / it)
Rank 1 Slower Config - uses 23.1 GB VRAM, 14.12 second / it (LoRA is 4.85 second / it)
Rank 1 Slowest Config - uses 15.5 GB VRAM, 39 second / it (LoRA is 6.05 second / it)
Final Info
Saved checkpoints are FP16 and thus 23.8 GB (no Clip-L or T5 trained)
According to the Kohya, applied optimizations doesn't change quality so all configs are ranked as Rank 1 at the moment
I am still testing whether these optimizations make any impact on quality or not
I am still trying to find improved hyper parameters
All trainings are done at 1024x1024, thus reducing resolution would improve speed, reduce VRAM, but also reduce quality
Hopefully when FP8 training arrived I think even 12 GB will be able to fully fine tune very well with good speeds
Comments
Furkan Gözükara
hello. runpod is specialized version of ubuntu. perhaps look at the massed compute code.
Oleh Kopyl
I launched both RunPod_Kohya_FLUX_Installer_part1.sh and the second part, but when i run ./gui.sh, i get this error: /home/azureuser/Kohya_GUI_Flux_Installer_v37/venv/bin/activate: No such file or directory . How to fix it?
Joel Maynard
ok thanks
Furkan Gözükara
use Quality_1_27500MB_6_26_Second_IT,
Joel Maynard
Quality_1_27500MB_6_26_Second_IT, Quality_1_23100MB_14_12_Second_IT, Quality_1_15500MB_32_Second_IT which of these files can be used?
Furkan Gözükara
you are welcome
Furkan Gözükara
quality 2 is lower quality you shouldnt use it. it is only for low VRAM GPUs. how should i name these files so you wont get confused :D
Joel Maynard
i am testing your quality 2 config but it is very slow and that too on a100 it's taking 20 hours for 2600 steps
Oinksauce
I saw that post! Very exciting! The ability to pull different flavors of LORAS from a. Checkpoint makes it seem like this is probably the way to go for now. Thank you for all the hard work and analysis!!
Furkan Gözükara
sdxl works way better with bf16 dont do fp16 there
mypatreonemailacc
I had a lot of issues with full fp16 finetuning with SDXL. Is it better with Flux?
Albert
ok thanks
Furkan Gözükara
yes fine tuning only fp16 at the moment sadly :(
Albert
So if I downloaded a modified FP8 model I can't train it, but only FP16?
Furkan Gözükara
sure nice one adding to my list
Philip Heggie
Hi can you do an app for DeTextify please with batch support? https://github.com/guiyang/detextify
Furkan Gözükara
yes i plan to. it is exactly same as LoRA but this time load config into Dreambooth :)
Fedor
great, it would be great if there was a guide (video) on how to train DreamBooth with your configuration. Are you planning to record a video?)
Furkan Gözükara
fine tune then extract should yield better than lora training itself
Oinksauce
Is it possible that LORA's produced from this process may be better than those produced separately? Just curious. Really appreciate your work. This is really awesome stuff.
Furkan Gözükara
i will look for this so i can show in tutorial :D maybe kohya also can add options
Furkan Gözükara
nope they dont help in flux yet
LoRAMaker4435
How can we quantize a fine tune that comes out as 23 GB to fp8 or q8 gguf?
Sipriyani
Besides your classic 15 dataset images, Did you use, massive regularization images?
Furkan Gözükara
each config shows step speed so you can calculate based on your target step count. lets say 150 epoch 2250 steps = 240 minutes on RTX A6000
Manpreet Singh
What's the overall training times on 24GB and 48GB GPUs in your tests?
Furkan Gözükara
ye i plan that later. i think it will work great
Oinksauce
I remember in prior SD generations that some people would train a fine tuned model like this, then extract a LORA from that model achieving high quality results. Is this something you might test and compare to your other LORA training methods?
Furkan Gözükara
i asked fp8 he didnt tell time yet :/ but i am hopeful
Vicent Guardiola
Hi there! You're doing an amazing job!! It's incredible how you set everything up so that with just a few clicks, someone can have everything ready on their PC to replicate any face they want. Plus, you explain everything as you go—what you're testing, the results you get—thank you so much for all the effort! I wanted to ask you a couple of questions: Do you have an estimate of when FP8 training might be available? I'm surprised that with my 3090, I feel like I'm "falling behind" in image generation speed hahaha
Furkan Gözükara
this is not cherry pick so results are expected 1 : slower than sdxl 2 : 1024 best, you can train in higher res too 3 : yes just swap more blocks to reduce VRAM - will be slower 4 : config names has each step speed for rtx a6000 - almost same as rtx 3090
cool1
So it works with 24GB RAM, that's good, but it's slow at 14 secs per it. Is that slower than with SDXL? Does it train faster than SDXL (does it need less time to be accurate enough than SDXL at the same resolution?). What resolutions can it train up to with 1 24 GB GPU (and plenty of RAM) - I've trained at 1280x1280 with SDXL - can it do that too or higher (well I tried at 1536 square with SDXL but that took much too long per it so stopped it)? Rough how much training time does it need per training image? In the examples I assume the one on the left is the Lora and the one on the right is the full fine tune. The one on the right is doing better prompt following but not exactly (doesn't show a blue beard in paris). Also neither of the coffee shop ones exactly follow the prompt as the prompt was "in a luxury coffee in paris" and "in a luxury coffee in china" (not coffee shop/cafe) - maybe for prompt following exactly it should put the person in a cup of coffee :) I assume the 'yolo-face' bit is replacing the face. If it fine tunes it fully shouldn't' it not need to replace the face?
