Post "The Very Best Workflow For SDXL DreamBooth / Full Fine Tuning - Results Of 100+ Full Trainings" by "Furkan Gözükara" from patreon

Lucy

When lunching Runpod I get this: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. gradio 4.43.0 requires pydantic>=2.0, but you have pydantic 1.10.21 which is incompatible.

Furkan Gözükara

in that case use the last checkpoint step, and reduce repeating number if you want to lets say train half. yes hard to grasp at first but read here : https://github.com/kohya-ss/sd-scripts/issues/640

Denn Landd

Give last checkpoint as a pretrained base model? And what if I used 1 epoch like you suggested in this tutorial? Maybe I still didn'twrap my head around repeats/epoch/steps dependence and difference?

Furkan Gözükara

you can give last checkpoint - saved training model and continue from there. just subtract the epoch count. lets say checkpoint is 100 epochs and you want total 200 epochs then do 200-100 = 100 epochs

Denn Landd

Is it possible to resume training for DB training? I know you can do it in LORA by checking save state and then providing network weights path, but can't see such path option in Dreambooth.

Furkan Gözükara

50 repeat and train 2 epoch and compare steps based taken checkpoints

A.S.

how much repeats do you suggest for your config , for 94 images instead of 55?

Furkan Gözükara

since you have 96 images it could be overtrained with SDXL

A.S.

Hello, I redid the training with the same settings as in the tutorial with 55 repeats , now as you said with the realvisxl checkpoint as base and here are the results https://imgur.com/a/m35eST9 i think it looks overtrained what do you see what can be the problem? maybe that 55 repeats is too much?

Furkan Gözükara

yes kohya has extract : https://www.patreon.com/posts/extracted-lora-108634568

A.S.

This is the SDXL full fine tuning kohya via runpod, i did everything just like you said in the tutorial and here after regarding the change to 55 repeats for the dataset, am using all the files from the video shown, how do i extract the lora from the realvisxl fine tuned safetensors? Is there an option for it in the same stable diff. Web ui in runpod?

Furkan Gözükara

Are using kohya flux branch or regular main branch? Use main branch bot not flux. Actually one trainer is better alternative for sdxl

A.S.

It is Sdxl full fine tune with dreambooth, ok i will try that

Furkan Gözükara

OK here do this: train a model on real vis xl 4. Also are you training Lora or fine tuning? My sdxl lora config is outdated not good. Best do fine tuning, if you need lora you can extract

A.S.

It looks very overtrained, maybe you can tell what the issue is looking at the pictures,

Furkan Gözükara

Which base model you used?

A.S.

I ve redone everything with the 55 repeats carefuly looking at everything but the results are horrible, i have used the 96 images dataset with 55 repeats , added the 5200 reg images, did not change anything else about the json 24 gb config and carefully followed and watched your video 20times, these are the results my lora gave after replicating what you did : https://imgur.com/a/LChpT1H please help me with this, the dataset is 1024 x 1024

Furkan Gözükara

Volume Disk. Temporary is only important when doing initial install

A.S.

yes it has ended already but which disk should i expand? Container Disk (Temporary) or Volume Disk (Persistent) or both?

Furkan Gözükara

turn off pod, expand disk start again but your training will end

A.S.

The volume has been exceeded (107%) and the training ended halfway, is it the Container Disk (Temporary) that i have to increase when renting out the pod on Runpod or the Volume Disk (Persistent) or both? or what should i do so i can proceed?

Furkan Gözükara

the logic is using as many as possible reg images. if you started recently yes you can do 55.

A.S.

In your tutorial you kept the 40, should i redo it also if it is not as good and try it with 55 repeats on the 92 train images and 1 repeat on the 5200 reg images , while keeping all the same other settings loaded with the json? (SDXL)

Furkan Gözükara

i would make 55 repeating to use more reg images in every epoch but 40 is decent too

A.S.

7520 steps overall, left the same settings as your config 40 repeat for 90 training images and 1 repeat for 5200 reg images , on Runpod this is an SDXL full fine tuning

Furkan Gözükara

how many repeat?

A.S.

for 5200 reg images and about 92 training images , it says now checking cache validity 3854 and then checking latents is 1927, are these numbers correct you think?

A.S.

Actually i have managed to get it there from googledrive, i ve used this: https://github.com/wkentaro/gdown , got the reg images also with zipping and installing jupyter archive then could unzip it now i am doing the training lets see how it goes.

Furkan Gözükara

google drive not good use hugging face as shown in here : https://youtu.be/-uhL2nW7Ddw

A.S.

I did succeed with the model upload , Now how do i upload the 5000 REG images to RUNPOD ? for the SDXL finetune , i have them on googledrive in a folder and trying to do it with a notebook made for downloading large files but because it is a folder it does not recognize the share link from googledrive...

Furkan Gözükara

hugging face. we have a special notebook that uploads and downloads ultra fast. shown in this tutorial : https://youtu.be/-uhL2nW7Ddw

A.S.

When i want to train on a specific checkpoint on runpod, what is the fastest way to get that uploaded to the runpod after i have installed the kohyaa and want to give it the path of the specific checkpoint on the pretrained model panel? It does not let me upload it i think that the file size is too big. I am currently trying to upload it to the runpod, to the same folder as where the base sdxl model is after loading the 24gb config file but my checkpoint is 6 gb in size, how can i get it there so i can proceed? Please help me with this it is super important

A.S.

https://www.patreon.com/posts/for-runpod-kohya-84898806 so this is the installer for the latest Kohyaa right?

Furkan Gözükara

lets say you rented a RTX 4090. and you did maximum 10000 steps since you have 75 images. it would be like 10000 second so around 3 hours. putting 5$ should be sufficient

Furkan Gözükara

yes it works. it is optimized for latest kohya

A.S.

in 1024 x 1024 size all

A.S.

Aprox. how much time will it take, how much credit should i buy on runpod to fully train on SDXL a 75 image dataset with the 5200 regularization images if i do everything like in the tutorial video and do it on 24GB GPU just like in the video?

A.S.

Will this workflow (Tier1_24_GB_Slower_v2.json) work with this Kohyaa installer? https://www.patreon.com/posts/for-runpod-kohya-84898806 I should just follow your video and it should work right? cause someone said here that it is not the latest version of Kohyaa but in this workflow you say on the latest update that Tier1_24_GB_Slower_v2.json is optimized for the latest Kohyaa...

Luna Sato

Imagine you have 100 images. I guess what the question boils down to is this: Is there a difference between training 10 epochs with 1 step vs. 1 epoch with 10 steps (repeats)? In both cases the model is trained on the same number of images.

Furkan Gözükara

ye i like both.

Manpreet Singh

Thank you both for the reply. Good to know that OneTrainer is still the bleeding edge.

Furkan Gözükara

thanks for comment

Furkan Gözükara

both onetrainer and kohya uses new findings at the moment. actually i chosen min snr gama on onetrainer i think it works better there but not here

Pew

Funny, there's a reply on the OneTrainer post, asking your question in reverse; if they can use the updated OneTrainer configs for Kohya. In case you missed it, here are the updated configs for OneTrainer, just hours old. https://www.patreon.com/posts/onetrainer-xl-96028218

Pew

Holy wishful dream come true! You like me, Doc, you really like me!!

Manpreet Singh

Are these new changes only I use Kohya. Previous SOTA configs used to be on OneTrainer. Any way to use these new findings in OneTrainer?

Pew

::looks at OneTrainer post|looks at this Koyha post|genuine appreciation, patience, AND anticipation mounts:: lol

Furkan Gözükara

nice

mypatreonemailacc

I'm seeing in Kohya Dreambooth training that small batches without batch accumulation works much better than large batches with batch accumulation. It sounds like your findings agree with this.

Furkan Gözükara

you are welcome

Kraussian

Ah, I understand now. Thank you for your help!

Furkan Gözükara

it is using shared vram because you have 16 gb vram. kohya is not vram optimized. use onetrainer and you will get 1 second / it

Kraussian

Wow, I've just tried training on Dreambooth following the instructions with 150 repeats of 33 images, and Kohya is telling me it'll take ~22 hours on RTX 4090 Laptop GPU with 7.83s/it! Is this normal or am I doing something wrong? https://i.ibb.co/xskxBV5/kohya-dreambooth-20240722080737.jpg

Furkan Gözükara

16 GB is great. You should use OneTrainer config with it. It will work great. Extraction is explained here very well as well : https://www.patreon.com/posts/full-workflow-sd-98620163 and onetrainer config is here : https://www.patreon.com/posts/96028218 onetrainer tutorial is here : https://youtu.be/0t5l6CP9eBg onetrainer is more vram optimized so it will work better for you. use tier1_10.4GB_slow.json if tier1_15.4GB_fast.json works slow they are same quality

Kraussian

Ah I didn’t know “train a full model & then extract LoRA” was possible. Let me try that out! I’m using a RTX 4090 laptop GPU (which I understand is a misleading name because it’s totally different from a desktop RTX 4090 GPU) with 16GB VRAM.

Furkan Gözükara

hello welcome. for LoRA I don't have a great config atm. I plan to research but the reason I didn't yet is that the LoRA is inferior. best is train a full model and then extract LoRA if you need a LoRA. what is your GPU?

Kraussian

Hi, I’ve just found & joined this community after following way too many guides with conflicting info and failing to train even a single LoRA to an acceptable quality. Is there a similar guide as this for LoRA training?

Furkan Gözükara

hello don't install with pinokio. install with my tutorial : first install python properly - this will make you use all AI apps : https://youtu.be/-NjNy7afOQ0 then install kohya as shown here : https://youtu.be/sBFGitIvD2A

eduardo

Hello, getting an error whne starting training ths is waht i get load VAE: stabilityai/sdxl-vae Downloading config.json: 100%|███████████████████████████████████████████████| 607/607 [00:00 │ │ │ │ 743 │ args = parser.parse_args() │ │ 744 │ args = train_util.read_config_from_file(args, parser) │ │ 745 │ │ │ ❱ 746 │ train(args) │ │ 747 │ │ │ │ E:\PINOKIO\sd\api\kohya.pinokio.git\kohya_ss\sdxl_train.py:698 in train │ │ │ │ 695 │ │ │ save_stable_diffusion_format, │ │ 696 │ │ │ use_safetensors, │ │ 697 │ │ │ save_dtype, │ │ ❱ 698 │ │ │ epoch, │ │ 699 │ │ │ global_step, │ │ 700 │ │ │ text_encoder1, │ │ 701 │ │ │ text_encoder2, │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ UnboundLocalError: local variable 'epoch' referenced before assignment steps: 0it [00:01, ?it/s] ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ E:\PINOKIO\sd\bin\miniconda\lib\runpy.py:196 in _run_module_as_main │ │ │ │ 193 │ main_globals = sys.modules["__main__"].__dict__ │ │ 194 │ if alter_argv: │ │ 195 │ │ sys.argv[0] = mod_spec.origin │ │ ❱ 196 │ return _run_code(code, main_globals, None, │ │ 197 │ │ │ │ │ "__main__", mod_spec) │ │ 198 │ │ 199 def run_module(mod_name, init_globals=None, │ │ │ │ E:\PINOKIO\sd\bin\miniconda\lib\runpy.py:86 in _run_code │ │ │ │ 83 │ │ │ │ │ __loader__ = loader, │ │ 84 │ │ │ │ │ __package__ = pkg_name, │ │ 85 │ │ │ │ │ __spec__ = mod_spec) │ │ ❱ 86 │ exec(code, run_globals) │ │ 87 │ return run_globals │ │ 88 │ │ 89 def _run_module_code(code, init_globals=None, │ │ │ │ in :7 │ │ │ │ 4 from accelerate.commands.accelerate_cli import main │ │ 5 if __name__ == '__main__': │ │ 6 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │ │ ❱ 7 │ sys.exit(main()) │ │ 8 │ │ │ │ E:\PINOKIO\sd\api\kohya.pinokio.git\kohya_ss\venv\lib\site-packages\accelerate\commands\accelera │ │ te_cli.py:45 in main │ │ │ │ 42 │ │ exit(1) │ │ 43 │ │ │ 44 │ # Run │ │ ❱ 45 │ args.func(args) │ │ 46 │ │ 47 │ │ 48 if __name__ == "__main__": │ │ │ │ E:\PINOKIO\sd\api\kohya.pinokio.git\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.p │ │ y:918 in launch_command │ │ │ │ 915 │ elif defaults is not None and defaults.compute_environment == ComputeEnvironment.AMA │ │ 916 │ │ sagemaker_launcher(defaults, args) │ │ 917 │ else: │ │ ❱ 918 │ │ simple_launcher(args) │ │ 919 │ │ 920 │ │ 921 def main(): │ │ │ │ E:\PINOKIO\sd\api\kohya.pinokio.git\kohya_ss\venv\lib\site-packages\accelerate\commands\launch.p │ │ y:580 in simple_launcher │ │ │ │ 577 │ process.wait() │ │ 578 │ if process.returncode != 0: │ │ 579 │ │ if not args.quiet: │ │ ❱ 580 │ │ │ raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) │ │ 581 │ │ else: │ │ 582 │ │ │ sys.exit(1) │ │ 583 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ CalledProcessError: Command '['E:\\PINOKIO\\sd\\api\\kohya.pinokio.git\\kohya_ss\\venv\\Scripts\\python.exe', './sdxl_train.py', '--pretrained_model_name_or_path=E:/SSD2.0/SUPER SD 2.0 Dependencies/stable-diffusion-webui/models/Stable-diffusion/sdXL_v10VAEFix.safetensors', '--train_data_dir=E:/downloads/TRAINMODEL-CHCEKPOINT_SDXL\\img', '--reg_data_dir=E:/downloads/TRAINMODEL-CHCEKPOINT_SDXL\\reg', '--resolution=1024,1024', '--output_dir=E:/downloads/TRAINMODEL-CHCEKPOINT_SDXL\\model', '--logging_dir=E:/downloads/TRAINMODEL-CHCEKPOINT_SDXL\\log', '--save_model_as=safetensors', '--full_bf16', '--vae=stabilityai/sdxl-vae', '--output_name=Tier1_24_GB_Slower', '--lr_scheduler_num_cycles=1', '--lr_scheduler_power=1', '--max_train_epochs=0', '--max_data_loader_n_workers=0', '--learning_rate=8e-06', '--lr_scheduler=constant', '--train_batch_size=1', '--max_train_steps=2000', '--save_every_n_epochs=1', '--mixed_precision=bf16', '--save_precision=bf16', '--seed=0', '--cache_latents', '--cache_latents_to_disk', '--optimizer_type=Adafactor', '--optimizer_args', 'scale_parameter=False', 'relative_step=False', 'warmup_init=False', 'weight_decay=0.01', '--max_train_epochs=0', '--max_data_loader_n_workers=0', '--vae_batch_size=2', '--bucket_reso_steps=64', '--save_every_n_steps=451', '--gradient_checkpointing', '--bucket_no_upscale', '--noise_offset=0.0', '--max_grad_norm=0.0', '--no_half_vae', '--train_text_encoder']' returned non-zero exit status 1.

Furkan Gözükara

Sadly we also don't know. But the LR is more powerful now so we reduced the LR after new research

Pew

Dr. Gözükara, do you happen to know when, date or version, that Kohya_ss made change to the Learning Rate and how it impacts training? I have tried looking at release notes, however, I don't see it mentioned and knew about this only when you brought it forward to the community. Context: I would like to download a prior version to archive and I'm not sure when the change happened. Thank you.

Furkan Gözükara

you are welcome

Pew

Thank you for explaining further.

Furkan Gözükara

hello i cant make and it is not necessary either. i just trained a client and worked even better than my training because his dataset was better

Jannik

Hello, can you make your 15 training images available? I just want to reproduce your results :) Thanks!

Furkan Gözükara

thanks but as i said above. 1 step = 1 time GPU cycle. with batch size 1 : it process 1 image. 1 epoch = 1 time processing every image in the training dataset. so if you have total 100 images, 100 steps will get you 1 epoch . at every step it will modify model weights so that particularly trained image can be generated with more accuracy - but it will overwrite existing knowledge too. so you can look for terms of catastrophic forgetting in ai models with training

kurlu

I would also be interested in this. As I understand the training situation, every epoch learns from the previous. Please explain the logic behind the repeats and the epochs. Thanks Furkan. You brought me within 2 weeks to almost mastering stable diffusion. Best educater for AI out there!!

Furkan Gözükara

you need to understand logic of epoch and step. 1 step = 1 time GPU cycle. with batch size 1 : it process 1 image. 1 epoch = 1 time processing every image in the training dataset. so if you have total 100 images, 100 steps will get you 1 epoch i can give you private lecture if you need. so you can ask any questions

Pew

Dr. Gözükara, pardon my forwardness here; the fact is I don't know what I'm doing and therefore follow your guidance with much success. I'm a fan and supporter, though I would like to ask the following. I've read many times that training over several epochs produces higher quality outcomes versus same steps over the course of a single epoch. Can you please speak to this? I understand with your process, you still arrive at 10 checkpoints, though is that at a reduced benefit to say actually training 10 full epochs set to a defined number of steps each? I look forward to your insight and commentary!

Furkan Gözükara

for old preset load this latest one. change unet learning rate to 1e-05 and Text Encoder-1 learning rate to 3e-06. it should work same. we dont train text encoder 2 so it is set to 0

Daniel Alderson Smith

all the links to the old one redirect here. i am still on the old kohya and need the old presets. can you post a link to those ones?

Furkan Gözükara

i trained all 150 repeat 1 epoch. so i didnt compare checkpoints. i think best is 8e-06 unet and 3e-06 text encoder.

Furkan Gözükara

yes it tries to open in browser. right click and save as :D

Kadir Nar

The image file may not open because it is large. These links may be correct. https://huggingface.co/MonsterMMORPG/SECourses/blob/main/comparison1.jpg https://huggingface.co/MonsterMMORPG/SECourses/blob/main/comparison2.jpg

Kadir Nar

The links here are not working. https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/comparison1.jpg https://huggingface.co/MonsterMMORPG/SECourses/resolve/main/comparison2.jpg

Diego Sienra

what was the best checkpoint from the training in your opinion? how many repetitions have the sample pictures on this post?

Furkan Gözükara

if you want facial expressions and body angles you should include them in your dataset and also in prompts.

Christina Frazier

Can you explain about facial expressions? I notice your face is very similar in all of your images. Also, your body positioning is always very similar ( unless you are riding a dinosaur ) Are facial expressions and angles of the body things you train or things you prompt for?

Furkan Gözükara

around 1.2 second / it best for RTX 4090 and 1.5 second / it for RTX 3090 . but i may not be remembering accurately

dxqbYD

how much it/s do you have on which card? tests were done with community runpods. there could be some other factors except GPU with one of the machines, idk

Furkan Gözükara

these above numbers not looking accurate when doing fine tuning. they are different for me. when doing fine tuning RTX4090 vs RTX3090 difference is much lesser than that

dxqbYD

okay thanks, haven't looked into massed compute yet. batch size is probably a no-no with 16 GB. but the numbers I get don't surprise you? with LoRA I got decent results in 2000 steps (w/o reg though, reg made the quality worse with LoRA). Been running some tests because I want to buy me own machine, in case you are interested, for ~ 10000 steps: A4000: 5-6h | 0,5 it/s RTX3090: ~ 3h | 0,9 it / s RTX4080: ~ 3h | 1,0 it/ s RTX4090: ~ 1,7h | 1,7 it / s considering a 4070 Ti 16 GB, which is supposedly between the 3090 and 4080 in benchmarks, but not available on runpod for testing

Furkan Gözükara

same parameters. but how you use captions matter. you should try different captionings and see which one works best. like kosmos 2 llava etc

eduard joseph Ladia

whats the best parameters for training a style dreambooth checkpoint,

Furkan Gözükara

yes you can use Massed Compute and increase batch size 4. it will speed up significantly with only 31 cents per hour

dxqbYD

I need about 6000-8000 steps until the model gets really good, which can take about 4 hours on a cheap pod. Are these values to be expected, or am I doing something wrong? [meaning: 3000-4000 repetitions + reg] Have you experimented with faster settings that might be slightly lower quality? Thanks in advance

dxqbYD

people looking like clowns? I had that before in samples generated by Dreambooth, and it was related to the sampler, IIRC DDIM. Try another sampler?

Furkan Gözükara

the config dont show error

Profile Photo

could you check my config ?

Profile Photo

https://drive.google.com/file/d/1nxaXpZsWNJTHbA6UkUeU-Kr9x_0Bi2Fb/view?usp=drive_link This is link to my configuration, could you check it first?

Furkan Gözükara

hello. please make a video of how you are setting up training. otherwise impossible to know your error. i am also giving private lecture if you want

Profile Photo

Hello, I tried to train sdxl dreambooth, but all my models have pink-blue noise, could you say why did it happen ? Like this: https://drive.google.com/file/d/1rHOkp85bfNzTXuY_gsuqDC_SlIkb5W8T/view?usp=drive_link

Furkan Gözükara

i found my article here : https://medium.com/@furkangozukara/stable-diffusion-xl-sdxl-dreambooth-u-net-only-vs-text-encoder-enabled-training-comparison-455f94d20ad3

AmunRaw

I was hoping to see a comparison of the quality differences with text encoder on and off.

Furkan Gözükara

hello. 100 image 2 epoch and 52 repeat would mean = 100 * 2 * 52 * 2 = 20800 steps. so it will take a lot of time. you can do this. reduce repeat to 52 and 1 epoch and save every n steps to get multiple checkpoints

Furkan Gözükara

it is hard to find. you can skip

Furkan Gözükara

ye they are gone sadly. what info you need i can tell you better

AmunRaw

Will you please update the x/twitter links in your post? They are no longer valid in this section. With the current volatility of your x hosted material it may be beneficial to use some other hosting or patreon directly: The text encoder on and off difference is huge .e.g : https://twitter.com/GozukaraFurkan/status/1721845175478083958 The text encoder enabled training uses same VRAM but a little bit slower : https://twitter.com/GozukaraFurkan/status/1720942143143895357

Zhiwan Cheung

Should we always use 5200 for the Reg images? If we want to train something other than a man/woman, should we find 5200 class examples?

Furkan Gözükara

yes it will drop both. we don't train text encoder 2 so it should be fine to extract both

Dallin Mackay

But wouldn't I want to keep the first text encoder? Will raising minimum difference drop the accidentally trained second one without dropping the first one?

Furkan Gözükara

yes you can. also there is minimal difference. so if you set it bigger, it wont extract text encoder

Dallin Mackay

Could you in theory drop the second text encoder from an existing finetune when extracting lora from it by modifying extract_lora.py?

Furkan Gözükara

hello. it says that no way : https://www.gradio.app/guides/sharing-your-app

Furkan Gözükara

it could be another sampler in kohya. I prefer sampler as : DPM++ 2M SDE Karras

Alec Scott

Strangely enough, it seems like using `sd-scripts/sdxl_gen_img.py` to generate the images, with the exact same settings, generates differing results than Automatic1111. `sdxl_gen_img.py`'s results are definitely much closer to the sampled images, though

Vasiliy Bulanov

I understand that, however the link is only for 72 hours. I didn't quite understand how to host it permanently using gradio spaces

Alec Scott

So I've been able to consistently produce fairly high quality sample images during training, but then when I move the model over to Stable Diffusion in Automatic1111, the output quality is significantly lower, and looks nothing like what was sampled. The generation settings for the sample images and in Stable Diffusion are the exact same: W/H: 1024, 1024 Seed: 1 CFG Scale: 7.5 Sampling Method: Euler A (I understand that's not the best, but that's what the sampler uses) Sampling steps: 30 In theory, since the seed is the exact same, it should produce the same exact image between the sampler and Stable Diffusion, right? Any ideas what I'm doing wrong?

Furkan Gözükara

you need to add --share argument as start argument

Josh Tech

with the settings in my screenshot I need about 11 hour to finish two epoches with 24GB VRAM. Did I do something wrong xD?

Josh Tech

Newbie question here: Are those the correct settings (epoches, repeats, safe every N epoches) that you recommend for 100 training images and your reg. images? https://imgur.com/a/4SnXFtV Thanks a lot

Vasiliy Bulanov

Can you pleas give a hint on how to run permanent share link at gradio? don't quite get it:)

Furkan Gözükara

thank you so much. i think duplicate image wouldn't help so i would avoid it

Sugar Coat VFX Design

Thank you for this very best workflow ! Do you think it helps if I add additional duplicated image set but remove all the background of each image?

Furkan Gözükara

Also always do x/y/z checkpoint comparison to find best checkpoint

Furkan Gözükara

yes SDXL base 1.0 and also RealVisXL_V4.0 are good according to my test

Furkan Gözükara

you are welcome. hopefully updated new tutorials on the way

Franco Acosta Diaz

Thank you so much

Furkan Gözükara

ok it is easy first watch this tutorial : https://youtu.be/EEV8RPohsbw then this tutorial : https://youtu.be/16-b1AjvyBE for 29 photos i suggest use reg images + 150 repeat + 1 epoch and save every 29 * 150 * 2 / 10 + 1 = 871 steps. so you will get 10 checkpoints i am giving private education too if you need

Franco Acosta Diaz

Hello, I am quite lost. I don't understand the parameters and how should I configure it in the best way according to my needs. I have 29 photos. I have to take the .json and change the parameters according to the number of my photos. What are the parameters I should change in the .json and how. Thank you very much.

Sugar Coat VFX Design

Thanks~!

Furkan Gözükara

yes do x/y/z checkpoint comparison. that is the best way. the number of checkpoints you are going to get depends on you. my newest strategy of using higher repeat count and 1 epoch explained in this video : https://youtu.be/16-b1AjvyBE

Furkan Gözükara

if you have 100 images make repeating 52. it will use all reg images. so it will be total 5200 steps * 2 = 10400. if you need more than 1 epoch you can increase count. to save checkpoints use save every N steps. like for 10400 use 1041 to save 10 checkponts

Elie Hamouche

When the training is done, should the final output model be used? or do we need to look at intermediate checkpoints to try and find the best model (like the xyz analysis in this post)

Elie Hamouche

Looks like training other models isn't as good as the base, so will stick with the original method which is giving the best results.

Sugar Coat VFX Design

Hi, If I have 100 images for training, should I use all of your Reg images (5200 images) ? If I use less Reg images, should I use more Repeat for Reg images?

Elie Hamouche

Do you think it makes sense to train on top of juggernaut with the same settings as in this post? Happy to try it out and share my results with the community, any tips before I start the training would be appreciated!

Furkan Gözükara

it looks good but it gives the vibes of 3d render. currently i did training on RealVisXL V4 and testing the results in terms of realism

Elie Hamouche

Nice thank you, I've been using the juggernaut model for a while now and it's really high quality compared to these. Any ideas on how to get as good of quality in terms of realism. For example: https://civitai.com/images/7022338

Furkan Gözükara

for example these are SDXL : https://civitai.com/posts/1238778

Furkan Gözükara

hello . sure. you can download images here. they have png info data. so you can use them in png info of automatic1111 and see all parameters : https://civitai.com/user/SECourses/images

Elie Hamouche

Any tips on the parameters to set when generating images with the trained model? I've imported the model into Automatic1111 but can't seem to get the quality and sharpness shown in the SDXL examples.

Furkan Gözükara

it depends. if you are using our reg images, which are highest quality, more repeats better since it will use more variety of reg images. if you are not using reg images, then 1 repeat and more epochs better.

Davit Sharian

which one is better more epochs less repeats or more repeats 1 epoch, or they are the same, for the same steps count, thank you

Davit Sharian

train_imgs * 2(if class img used) * repeating_count * number_of_epochs according to your formula, is there any difference if I use 10 epochs with 15 repeats, and 1 epoch with 150 repeats? in two cases I'll have 9000 steps for 30 images

Furkan Gözükara

well for 30 images i suggest this. do 150 repeat, 1 epoch and save every 30 * 150 * 2 / 10 +1 = 901 steps. so you will have 10 check points and you can compare all

Davit Sharian

it's giving like 6 hours of training on RTX3090

Davit Sharian

Hi, if I have 30 images for training, do I need to change repeats count? or it will work for me with 40 repeats too?

Furkan Gözükara

Hi here tutorial that shows it : https://youtu.be/EEV8RPohsbw

Franco Acosta Diaz

Hi, I don't understand this, how do I implement it in Runpod?

Furkan Gözükara

still best. nothing new came so far. i mean new optimizer etc that made impact.

Arcon Septim

Is this getting outdated or still the best? ^^

Furkan Gözükara

that is accurate prompt. in after detailer use photo of ohwx man as prompt. also test just solo prompt ohwx man and see if you are getting your training images face

Tigger Saldy

I get an image for a prompt like "photo of ohwx man", but for prompts below I do not get the character (after trying to generate 20-25 times). Is the issue with that my training images need to show enough to match the prompt? (like a full-body image). Also in your YT video there is a ADetailer section - does the model need to be downloaded into a certain directory for it to show up? "High-resolution, full-body photograph of an (ohwx man:1.1) suitable for a popular Instagram post etc. etc." or "photo of ohwx man walking in new york city, shot on Fujifilm Superia 400 ..."

Tigger Saldy

Thanks

Furkan Gözükara

24GB_TextEncoder.json - it should work. it uses like 17GB

Atmateria

Hello, which configuration should I use for the NVIDIA RTX A4000 -20GB VRAM and 64GB RAM, 16GB or 24GB? With TextEncoder and without it?

Furkan Gözükara

it is advanced setting save every n steps

Tigger Saldy

clarifying that the change where the changes need to be made ... where would the 321 steps go to? Dreambooth/Parameters/Basic - Epoch=1 Dreambooth/DataSet Preperation/Dreambooth/LoRA Preperation/Training Images Repeats= 200

Furkan Gözükara

thank you for your support

Tigger Saldy

Thank you for your super quick response - I will try that. Thanks for all the videos and effort.

Furkan Gözükara

ye lora harder. before using mixed resolutions with bucketing, crop all to 1024x1024 and do that way training. after that try different resolutions with bucketing and compare. so you will have baselines for each case

OP-Cast

ok ill do that, my training images are a mix of 768,768 and 1200x1800 minus some weird ones where i cropped any oddities, was reading through your pdf and downloaded both 24 gb text and no text config files for kohya. Thank you for the help, lora training on sdxl i have found to be trickier than 1.5.

Furkan Gözükara

I also have 3090 it is a good gpu. i just replied your message

Furkan Gözükara

hello. you have too many images. i dont know their quality. if their quality is good and high then ok so here my suggestions. you should go step by step experiment and compare my first suggestion : pick 15 very good images like the dataset i have shown. you can even pick better one : use SDXL base 1.0 : train 150 repeat , 1 epoch, our regularization images : make all training images 1024x1024 so your training becomes 1024x1024 : use our 1024x1024 woman reg images dataset after you obtain some base results with this do this : use your 179 images dataset : make sure all cropped to 1024:1024 - we have auto cropper too : use repeating 29 : use our 1024x1024 reg images : train 2 epochs : save every : 179 * 29 * 2 / 10 + 1 = 1039 steps this will give you 10 checkpoints. compare them with x/y/z do not use captions. train with only ohwx woman after all these done let me know the results

OP-Cast

forgot to mention I have 3090 gpu

OP-Cast

Hi Sir, I would greatly appreciate guidance on fine tuning my steps epochs/repeats. Currently I've been doing my training on Kohya SS using prodigy following this guide here >> https://civitai.com/articles/3522/valstrixs-crash-course-guide-to-lora-and-lycoris-training I've been playing with a couple checkpoints EpicRealismXl V1 and NightVisionXL, (going for pure realism) do you recommend base sdxl? or go for a checkpoint already tailored for my use. Also I'm having trouble wrapping my head around steps ect and overbaking. My dataset consists of 179 photots of 1 subject (wife) I have a mixture of dynamic full body/half/closeupface. I was using WD14 captioning but as im typing this im using your personal tagger setup blip2 and the bg something lol from your yt video. Anyways do you recommend captions if i only have 1 subject, and also i downloaded your regurlization woman set both uncropped and 1536x1536. Appreciate any guidance! Thank you. "TLDR 179 images in dataset 1 subject, caption yes or no?, how many epoch and how repeat ex 20_woman ect what settings with your female dataset."

Furkan Gözükara

well that means you are out of storage. do this, 120 gb volume disk 200 repeat, 1 epoch, 8 images, save every 8 * 1 * 200 * 2 / 10 + 1 = 321 steps. it will save 10 checkpoints for you and it will work with our 24 GB text encoder config

Tigger Saldy

I am trying "How To Do Stable Diffusion XL (SDXL) DreamBooth Training (Full Fine Tuning) On Windows and RunPod" I am training with 8 (1024 x 1024) images, getting my regularisation images, with download_man_reg_imgs, and using 24GB_TextEncoder for my config file on a runpod RTX 3090 26 vCPU 93 GB RAM, with 150 GB Disk. But imaterial of the disk size I use (I have tried 10, 20, 50, 150), the training dies at 38% with this messsage, after running 6 epochs. shutil.Error: [('/workspace/train/DSC_0190.JPG', '/workspace/stable-diffusion-webui/models/Stable-diffusion/img/40_ohwx man/DSC_0190.JPG', '[Errno 122] Disk quota exceeded') Any feedback would be appreciated.

Furkan Gözükara

default are ok. you are using LoRA tab - sdxl_train_network.py this config is for DreamBooth

Nikoles Frances

This is what I get when trying both the no text encoder and text encoder 24gb (3090) The following values were not passed to `accelerate launch` and had defaults used instead: `--num_processes` was set to a value of `1` `--num_machines` was set to a value of `1` `--mixed_precision` was set to a value of `'no'` `--dynamo_backend` was set to a value of `'no'` To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`. usage: sdxl_train_network.py [-h] [--v2] [--v_parameterization] [--pretrained_model_name_or_path PRETRAINED_MODEL_NAME_OR_PATH] [--tokenizer_cache_dir TOKENIZER_CACHE_DIR] [--train_data_dir TRAIN_DATA_DIR] [--shuffle_caption] [--caption_separator CAPTION_SEPARATOR] [--caption_extension CAPTION_EXTENSION] [--caption_extention CAPTION_EXTENTION] [--keep_tokens KEEP_TOKENS] [--caption_prefix CAPTION_PREFIX] [--caption_suffix CAPTION_SUFFIX] [--color_aug] [--flip_aug] [--face_crop_aug_range FACE_CROP_AUG_RANGE] [--random_crop] [--debug_dataset] [--resolution RESOLUTION] [--cache_latents] [--vae_batch_size VAE_BATCH_SIZE] [--cache_latents_to_disk] [--enable_bucket] [--min_bucket_reso MIN_BUCKET_RESO] [--max_bucket_reso MAX_BUCKET_RESO] [--bucket_reso_steps BUCKET_RESO_STEPS] [--bucket_no_upscale] [--token_warmup_min TOKEN_WARMUP_MIN] [--token_warmup_step TOKEN_WARMUP_STEP] [--dataset_class DATASET_CLASS] [--caption_dropout_rate CAPTION_DROPOUT_RATE] [--caption_dropout_every_n_epochs CAPTION_DROPOUT_EVERY_N_EPOCHS] [--caption_tag_dropout_rate CAPTION_TAG_DROPOUT_RATE] [--reg_data_dir REG_DATA_DIR] [--in_json IN_JSON] [--dataset_repeats DATASET_REPEATS] [--output_dir OUTPUT_DIR] [--output_name OUTPUT_NAME] [--huggingface_repo_id HUGGINGFACE_REPO_ID] [--huggingface_repo_type HUGGINGFACE_REPO_TYPE] [--huggingface_path_in_repo HUGGINGFACE_PATH_IN_REPO] [--huggingface_token HUGGINGFACE_TOKEN] [--huggingface_repo_visibility HUGGINGFACE_REPO_VISIBILITY] [--save_state_to_huggingface] [--resume_from_huggingface] [--async_upload] [--save_precision {None,float,fp16,bf16}] [--save_every_n_epochs SAVE_EVERY_N_EPOCHS] [--save_every_n_steps SAVE_EVERY_N_STEPS] [--save_n_epoch_ratio SAVE_N_EPOCH_RATIO] [--save_last_n_epochs SAVE_LAST_N_EPOCHS] [--save_last_n_epochs_state SAVE_LAST_N_EPOCHS_STATE] [--save_last_n_steps SAVE_LAST_N_STEPS] [--save_last_n_steps_state SAVE_LAST_N_STEPS_STATE] [--save_state] [--resume RESUME] [--train_batch_size TRAIN_BATCH_SIZE] [--max_token_length {None,150,225}] [--mem_eff_attn] [--xformers] [--sdpa] [--vae VAE] [--max_train_steps MAX_TRAIN_STEPS] [--max_train_epochs MAX_TRAIN_EPOCHS] [--max_data_loader_n_workers MAX_DATA_LOADER_N_WORKERS] [--persistent_data_loader_workers] [--seed SEED] [--gradient_checkpointing] [--gradient_accumulation_steps GRADIENT_ACCUMULATION_STEPS] [--mixed_precision {no,fp16,bf16}] [--full_fp16] [--full_bf16] [--ddp_timeout DDP_TIMEOUT] [--clip_skip CLIP_SKIP] [--logging_dir LOGGING_DIR] [--log_with {tensorboard,wandb,all}] [--log_prefix LOG_PREFIX] [--log_tracker_name LOG_TRACKER_NAME] [--log_tracker_config LOG_TRACKER_CONFIG] [--wandb_api_key WANDB_API_KEY] [--noise_offset NOISE_OFFSET] [--multires_noise_iterations MULTIRES_NOISE_ITERATIONS] [--ip_noise_gamma IP_NOISE_GAMMA] [--multires_noise_discount MULTIRES_NOISE_DISCOUNT] [--adaptive_noise_scale ADAPTIVE_NOISE_SCALE] [--zero_terminal_snr] [--min_timestep MIN_TIMESTEP] [--max_timestep MAX_TIMESTEP] [--lowram] [--sample_every_n_steps SAMPLE_EVERY_N_STEPS] [--sample_every_n_epochs SAMPLE_EVERY_N_EPOCHS] [--sample_prompts SAMPLE_PROMPTS] [--sample_sampler {ddim,pndm,lms,euler,euler_a,heun,dpm_2,dpm_2_a,dpmsolver,dpmsolver++,dpmsingle,k_lms,k_euler,k_euler_a,k_dpm_2,k_dpm_2_a}] [--config_file CONFIG_FILE] [--output_config] [--metadata_title METADATA_TITLE] [--metadata_author METADATA_AUTHOR] [--metadata_description METADATA_DESCRIPTION] [--metadata_license METADATA_LICENSE] [--metadata_tags METADATA_TAGS] [--prior_loss_weight PRIOR_LOSS_WEIGHT] [--optimizer_type OPTIMIZER_TYPE] [--use_8bit_adam] [--use_lion_optimizer] [--learning_rate LEARNING_RATE] [--max_grad_norm MAX_GRAD_NORM] [--optimizer_args [OPTIMIZER_ARGS ...]] [--lr_scheduler_type LR_SCHEDULER_TYPE] [--lr_scheduler_args [LR_SCHEDULER_ARGS ...]] [--lr_scheduler LR_SCHEDULER] [--lr_warmup_steps LR_WARMUP_STEPS] [--lr_scheduler_num_cycles LR_SCHEDULER_NUM_CYCLES] [--lr_scheduler_power LR_SCHEDULER_POWER] [--dataset_config DATASET_CONFIG] [--min_snr_gamma MIN_SNR_GAMMA] [--scale_v_pred_loss_like_noise_pred] [--v_pred_like_loss V_PRED_LIKE_LOSS] [--debiased_estimation_loss] [--weighted_captions] [--no_metadata] [--save_model_as {None,ckpt,pt,safetensors}] [--unet_lr UNET_LR] [--text_encoder_lr TEXT_ENCODER_LR] [--network_weights NETWORK_WEIGHTS] [--network_module NETWORK_MODULE] [--network_dim NETWORK_DIM] [--network_alpha NETWORK_ALPHA] [--network_dropout NETWORK_DROPOUT] [--network_args [NETWORK_ARGS ...]] [--network_train_unet_only] [--network_train_text_encoder_only] [--training_comment TRAINING_COMMENT] [--dim_from_weights] [--scale_weight_norms SCALE_WEIGHT_NORMS] [--base_weights [BASE_WEIGHTS ...]] [--base_weights_multiplier [BASE_WEIGHTS_MULTIPLIER ...]] [--no_half_vae] [--cache_text_encoder_outputs] [--cache_text_encoder_outputs_to_disk] sdxl_train_network.py: error: unrecognized arguments: --train_text_encoder Traceback (most recent call last): File "C:\Users\thriv\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\thriv\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\thriv\AppData\Local\Programs\Python\Python310\Scripts\accelerate.exe\__main__.py", line 7, in File "C:\Users\thriv\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\accelerate_cli.py", line 47, in main args.func(args) File "C:\Users\thriv\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\launch.py", line 986, in launch_command simple_launcher(args) File "C:\Users\thriv\AppData\Local\Programs\Python\Python310\lib\site-packages\accelerate\commands\launch.py", line 628, in simple_launcher raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) subprocess.CalledProcessError: Command '['C:\\Users\\thriv\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', './sdxl_train_network.py', '--pretrained_model_name_or_path=E:/SD 1/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors', '--train_data_dir=E:/2.SD Tools/Fine Tuniing\\img', '--reg_data_dir=E:/2.SD Tools/Fine Tuniing\\reg', '--resolution=1024,1024', '--output_dir=E:/2.SD Tools/Fine Tuniing\\model', '--logging_dir=E:/2.SD Tools/Fine Tuniing\\log', '--network_alpha=1', '--save_model_as=safetensors', '--network_module=networks.lora', '--network_dim=8', '--output_name=Julie_Hadi_1.0XL', '--lr_scheduler_num_cycles=8', '--no_half_vae', '--full_bf16', '--learning_rate=1e-05', '--lr_scheduler=constant', '--train_batch_size=1', '--max_train_steps=3120', '--save_every_n_epochs=1', '--mixed_precision=bf16', '--save_precision=bf16', '--cache_latents', '--cache_latents_to_disk', '--optimizer_type=Adafactor', '--max_grad_norm=1', '--max_data_loader_n_workers=0', '--bucket_reso_steps=64', '--gradient_checkpointing', '--bucket_no_upscale', '--noise_offset=0.0', '--max_grad_norm=0.0', '--no_half_vae', '--train_text_encoder', '--vae=stabilityai/sdxl-vae']' returned non-zero exit status 2.

Furkan Gözükara

you are welcome

Teamin

Thank you. These are great insights.

Furkan Gözükara

I tested it thoroughly and it doesn't increase likeliness a lot but causes model to overfit. So model generalization degrades.

Furkan Gözükara

Sadly I never done inpainting training so don't know.

Teamin

I see that you are not training the second text encoder. Can you let me know why?

Anduvo

Hi Furkan, since kohya ss doesn't have an inpainting training, which other tool would you recomend to train an inpaint model ?

Anduvo

You are right! I was training a lora, sorry my bad for the false alarm. Thank you!

Furkan Gözükara

hello. you are trying to train LoRA. This config is for DreamBooth. I just tested and verified and working

Furkan Gözükara

i am going to install latest version now and update. thanks for heads up

Anduvo

Hi Furkan, with the latest khoya I am getting this error with your 24GB text encoder parameters: sdxl_train_network.py: error: unrecognized arguments: --train_text_encoder It seems they removed or changed something ? could you please take a look ?

Furkan Gözükara

sadly i dont know. if you test let me know please. currently i am testing SD 1.5

mike oxmaul

Are their particular models that are suitable for dreambooth training compared to others? What makes them suitable? I'm going to do some 48gb test encoder trainings today. I could test a specific model for you compared to base if you'd like.

Furkan Gözükara

yes i tried and they didn't perform better for realism for me so far. but for stylization i got better results

mike oxmaul

Have you done any trainings or other models? Instaed of the sdxl 1.0 or 0.9 base. I know we used to train on realistic vision when doing SD1.5. Looking at realism in particular.

Furkan Gözükara

Yes we have excellent config for one trainer. sadly I couldn't record video yet but will do hopefully.

jason santana

Thank you. I actually have 16GB in my GPU, i'll try this method.

Furkan Gözükara

Hello. You have 12 GB GPU. Therefore SDXL DreamBooth will run very slow. I suggest you to try OneTrainer. It has better VRAM usage. We have config here : https://www.patreon.com/posts/96028218

jason santana

I tried every single step of your video from signing it up to your Patron and trying the json on here, to using only 14 images. My computer still says it's going to take 27 hours. I have a 4070ti. Do you have a possible resolution to this issue?

Furkan Gözükara

hard to decide i agree. you can do more training and more frequent checkpoint saving to see where it starts overfitting

RtBx

mm im having trouble to decide how many should i choose as i have other dataset of 41 images and another of 50 images..how could i calculate the number of steps, at least approximately

Furkan Gözükara

those comes with experience. you can also set other numbers. just do more experiments. i gave rough numbers with comparing my 15 images 150 repeat 1 epoch

RtBx

why 801? how do you calculate the num of steps? and why 2 epoch?

Furkan Gözükara

for 100 images make repeat 52, 2 epoch, and save every 801 steps

RtBx

thank you!, and for 100 images?

Furkan Gözükara

sadly there is no formula. but as the image count increases you need to reduce number of epochs. lets say 319 images and we have 5200 reg images. so make repeat 16, make 2 epoch training and save every 639 steps.

RtBx

how many repats should i use if i have a dataset of 319 images. And how many is they are 100 images.? There is a formula or how i calculate how many repeats should i use, im using your reg images.

Furkan Gözükara

There is no parameter. we are using very best ones. so to make it better you need to improve your training dataset. lets say you have collected 50 images then do this. 5200 / 50 = 104. so do 104 repeating and 2 epochs and save 10 checkpoints . to save 10 check points make 102 * 2 * 50 * 2 / 10 + 1 = 2041 . so save once every 2041 steps usually 200 repeating and 1 epoch is good. but you can train longer and compare more checkpoints

Anonyme pas trop anonyme

Is there ANY parameter to make a way longer training and get better results ? if yes, what dataset size and how many repeats ?

Furkan Gözükara

you are right i said inaccurately. the lora folder is stable-diffusion-webui\models\Lora

Felix Rockwell

In this video https://www.youtube.com/watch?v=EEV8RPohsbw you say: save the lora file into /workspace/stable-diffusion-webui/models/Stable-diffusion/model But the lora files are here: workspace/stable-diffusion-webui/models/Lora So I don't understand why are you not saving it directly to Lora folder?

Furkan Gözükara

thank you so much for keep supporting me

Ksottam

Much appreciated. Keep it up! :)

Furkan Gözükara

Hello. You should use text encoder it really does improve. I will remove them from the post and add notice. thank you for support

Ksottam

I appreciate all the work you've put into this! I'm trying to understand whether or not to use text encoders and unfortunately the links you have on your page are dead. Should I use them? Happy New Year! https://twitter.com/GozukaraFurkan/status/1710995764162216205 https://twitter.com/GozukaraFurkan/status/1720942143143895357 https://twitter.com/GozukaraFurkan/status/1721845175478083958

Furkan Gözükara

Hello. Hopefully I will research that one too very soon. Not ready yet sadly.

RtBx

Hello, could you share settings, I mean a .json file but for stable diffusion 1.5 models too?

Furkan Gözükara

do like this. 200 repeat. 1 epoch. so it will make 3000*2 = 6000 steps. save checkpoints every 1201 steps. so you will get 5 checkpoints to compare

Khoa Vo

Can you elaborate on that a bit more? If I have 15 images are you saying I should repeat 150 times and just do 1 epoch? Right now I am doing 15 images and 40 repeats and 8 epochs. Which takes quite a long time to train.

Furkan Gözükara

yes it is high. you don't have to train that much. usually 150 epoch is good. so if you make repeating 3 epochs. if you make 150 repeating 1 epoch.

Khoa Vo

I see that the number of epochs for the 24GB config is set to 8. Isn't that really high? I'm using around 15 images.

Furkan Gözükara

sadly i don't have. I have only woman and man

Jan PR

Do you have regularization images of dogs?

Furkan Gözükara

lower the min diff threshold like 0.00001

Furkan Gözükara

reduce the difference it requires. like 0.0001

Dallin Mackay

from "Extract Lora" tab on kohya, with 192 dim and alpha. And using a lower dim and alpha made no difference. I also get the same message Meito posted ^^ before it OOMs

Meito

even with lora extraction i get - Text encoder is same. Extract U-Net only.

Furkan Gözükara

Sorry for delay. Kaggle is just updated and a new video released : https://youtu.be/16-b1AjvyBE

Furkan Gözükara

interesting. lora extraction should work on RAM. how did you try it?

Dallin Mackay

will do. also I tried extracting a Lora with your recent settings but I run out of memory. not sure if its ram or vram but i have 48gb and 24gb respectively. any idea?

Furkan Gözükara

i haven't tested on custom base models but should work very well. give it a try

Dallin Mackay

great params. have you tried training on finetuned base models? with loras I didn't find any advantage training on finetunes but I used them for inference for a guaranteed improvement in quality. but I figured DB might be different

DAVID PEREZ

Great news, also if you could make a github update or and update on the Kohya kaggle patreon page would be awesome

Furkan Gözükara

hello. it is in production hopefully soon. full SDXL DreamBooth training

Arcon Septim

Can you please make a full video tutorial with all the steps and in detail? Like from choosing best training images and then fine tuning afterwards, thaank you!

Furkan Gözükara

you just need to do basic git branch switch. i explained in this video thoroughly : https://youtu.be/kvxX6NrPtEk 18:08 How to switch to dev branch of Automatic1111 SD Web UI for SDXL TensorRT usage

Furkan Gözükara

I prefer to use adetailer for faces, 40 steps, cfg 7 and DPM++ 2M SDE Karras sampler. no other settings are changed as for faces photo of ohwx man or photo of ohwx woman sorry for late reply

So Sha

Furkan, I wanted to setup your recommended text encoder for kohya on ubuntu, but I couldn't understand what should I do : Currently text encoder training of SDXL is only supported in sd-scripts-dev branch So open a cmd and do git pull in your Kohya GUI folder Then do git checkout sd-scripts-dev Then do another git pull Would you please explain this part more?

Daniel Seravalli

Hi Furkan, thanks for your work on his. What settings do you use when you are prompting your trained models? sampling method, step count, cfg scale. Are you changing any other settings in auto1111?

Joel Maynard

how can i do this

Furkan Gözükara

sure message me from discord lets setup a consultation

Joel Maynard

i will be glad if you give consultation

Furkan Gözükara

it can be due to many factors. i am giving private consultation as well if you are interested in

Joel Maynard

hello sir, i appreciate all your work, but when I am training lora or dreambooth following your instructions, i am facing the problem of likeness, i am using training 12+ images and 2000+ reg images and i am using sdxl 1.0 model, do you have any tips to improve likeness ?

Furkan Gözükara

you need to upload relauncher.py restart pod then before start training kill web ui with fuser -k 3000/tcp

Chris

I get CUDA out of memory using the A5000 on runpod if I use the 24gig, I have to remove the VAE and the additional parameters to get it to run. Any help to use properly?

Russ Rehm

I tried this yesterday but the rare tokens all mixed in to the class token (man) and one of the rare tokens ended up dominant. It was possible to repair this with negative prompts but not to a high enough level. Very interesting outcome but obviously not the desired one!

Furkan Gözükara

don't use AI generated images unless you have to. because the model was not trained with AI images but it was trained with real images

M R

Thanks. What about the regularization set? Have you seen better results with AI generated images, as many seem to recommend, or real photos such as your "Massive 4K Resolution Woman & Man" set?

Furkan Gözükara

captions still testing. i haven't made a new video about this yet but my newest suggestion will be make 1 epoch 200 repeat and get checkpoints based on number of steps

M R

What's your recommendation for regularization images – how many, is there a particular set you've found best, captions or not?

Furkan Gözükara

thank you

So Sha

Good Job Dr, I’ll try it and let you know the results. 👍

Furkan Gözükara

you are welcome. thanks for keep supporting me

Ec Jep

Fantastic collection of parameters for everyone. Thank you very much for the research and sharing.

Furkan Gözükara

good question. same class tends to bleed but never tried with this new powerful technique. worthy to try

Russ Rehm

Do you think this would work for multiple rare tokens in the same class? I have used your config from last month and also compared with LoRA, Dreambooth is still the best. Thanks for this update I will try it on my pod!

Furkan Gözükara

i used 40 repeat count and trained up to 8 checkpoints. then compared checkpoints and found that 4th checkpoint was best. used 13 training images.

mahendra naik

Yeah, it was an issue with me setting up the data folders. It works fine now. What is the repeat count ?

Furkan Gözükara

Hello. you must have a config error. i used them so many times. can you show me your executed command and how did you setup your data folders and their paths? please watch this to understand : https://www.youtube.com/watch?v=EEV8RPohsbw

mahendra naik

The accelerate command for 48 GB setup at the end doesn't work. It always says 0 train images and 0 reg images even when the directory has images. Please update the documentation. It is misleading.

Furkan Gözükara

yep you can use that template . sorry for late reply

Rafael Villalba

I will ask the most innocent question of your entire channel, once I finish the training in runpod, how can we initialize the SD UI? In the future when we want to use our trained models, is it okay for us to use the webUI that runpod gives us by default in its SDxl template? Thank you so much !!

Furkan Gözükara

unfortunately not yet :/ but i have this quick one https://www.youtube.com/watch?v=EEV8RPohsbw

JS

SECourses: Tutorials, Guides, Resources, Training, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözükara 1w Tutorial very soon hopefully Has this video been created yet?

Furkan Gözükara

you can increase. like 32 64 128. as you increase it will learn subject more but the model general knowledge will get reduced. 8 is pretty low. try 32 first . by the way DreamBooth don't have rank. only LoRA has rank. Don't get confused. DreamBooth trains entire model and it is better .

Hassan Alhassan

should we keep these the same ? Network Rank (Dimension) 8 Network Alpha 1

Furkan Gözükara

yes here : https://www.youtube.com/watch?v=EEV8RPohsbw

Furkan Gözükara

2 is added if you use regularization images

Furkan Gözükara

I am still searching for text encoder effect. I think you don't have to use yet.

LO Last

Is there a tutorial for using this to train dreambooth model using Kohya?

Đạt Nguyễn

The formula = train_imgs * 2(if class img used) * repeating_count * number_of_epochs In my case : 13 * 2 * 40 * 4 = 4160 Sorry for not understanding, can you explain more clearly why it is multiplied by 2?

Anduvo

when we should use the text encoder one and when not ?

Zsédely Ruben

error: unrecognized arguments: --train_text_encoder Is there a newer kohya script that can handle this command?

Furkan Gözükara

sure let me know

Doc Snyder

I'll try 6 epochs :-)

Furkan Gözükara

almost same yes. just install as shown in video, do not install intel accelerate if it asks.

Furkan Gözükara

it depends on training dataset. i got good results at 4 th epoch but you can train up to 8 epochs

Doc Snyder

The config shows 4 epochs and your .txt tells 8 epochs? What is useful?

Nathan L.

Cool - nothing has changed since then?

Furkan Gözükara

it is directly inside kohya folder : https://github.com/bmaltais/kohya_ss - please also watch this tutorial : https://youtu.be/sBFGitIvD2A

Nathan L.

"./sdxl_train.py" - where can I find this script?? Also, how many input images did you upload?

Furkan Gözükara

i will make a quick tutorial and show into the first post hopefully today

Furkan Gözükara

that means your system is using shared RAM due to VRAM insufficiency. how much vram your computer is using when you don't do training? 4160 steps is normal count that you should train

Alexandre Ruffer

If I do 160 repeating count which I believe is equal to Repeats on Dataset preparation field, it just increases me the training to +100 hours. I have a Nvidia 4090 and I'm using best_settings_24_gb_VRAM_config_no_xformers, what suggestions would you give me? accelerate launch --num_cpu_threads_per_process=4 "./sdxl_train.py" --pretrained_model_name_or_path="C:/stable-diffusion-webui/models/Stable-diffusion/sd_xl_base_1 .0.safetensors" --train_data_dir="C:/Users/alexr/Documents/Stable Diffusion/Models/Alexandre Belorio/Output-160\img" --reg_data_dir="C:/Users/alexr/Documents/Stable Diffusion/Models/Alexandre Belorio/Output-160\reg" --resolution="1024,1024" --output_dir="C:/Users/alexr/Documents/Stable Diffusion/Models/Alexandre Belorio/Output-160\model" --logging_dir="C:/Users/alexr/Documents/Stable Diffusion/Models/Alexandre Belorio/Output-160\log" --save_model_as=safetensors --full_bf16 --output_name="negao-160" --lr_scheduler_num_cycles="1" --max_data_loader_n_workers="0" --learning_rate="1e-05" --lr_scheduler="constant" --train_batch_size="1" --max_train_steps="4160" --save_every_n_epochs="1" --mixed_precision="bf16" --save_precision="bf16" --cache_latents --cache_latents_to_disk --optimizer_type="Adafactor" --optimizer_args scale_parameter=False relative_step=False warmup_init=False weight_decay=0.01 --max_data_loader_n_workers="0" --bucket_reso_steps=64 --gradient_checkpointing --bucket_no_upscale --noise_offset=0.0

Furkan Gözükara

Tutorial very soon hopefully

Furkan Gözükara

Thank you. If you make one epoch make your repeating count like 160. So it will be actually 160 epochs

Alexandre Ruffer

So I am running best_settings_24_gb_VRAM_config_no_xformers.json on dreambooth the only change I did was I edited 8 epochs to 1, this means I will only have one .safetensor model to train, do I lose anything lowering down this value? I did this because the training time was to dramatic, now with 1 epoch It is one hour. Second question would be I was monitoring GPU stats for this training and it rarely used it, always taking power of CPU and Memory, is this right? Thanks for the grind, excited about this upcoming dreambooth for sdxl tutorial. Cheers.

JS

I guess I will wait for the video you mention in this response on your YouTube channel: @SECourses 4 days ago thank you so much. the biggest speed up would come from higher batch size like 2 or 3 and lesser number of steps. both way things require a precisely found new learning rate. moreover I suggest you to try my dreambooth workflow which is 10x better literally in terms of quality and speed almost same : https://www.patreon.com/posts/very-best-for-of-89213064 - a video for this workflow coming soon hopefully

JS

Is there a video or text tutorial on how to get this all to work with SDXL?

Furkan Gözükara

I am also working on text encoder trained version. currently we don't train it. I will update this post once I have even better config

Furkan Gözükara

use this version. it is same as 48 gb a little bit slower best_settings_24_gb_VRAM_config_no_xformers.json

JS

Very best one : 48_gb_VRAM_config_best.json What would be used to achieve 48gb Vram? I am using a 4090 what is the best config file for it? Also is this the latest method for training SDXL or is there another post that you are working on?

Furkan Gözükara

yes but quality could be a little bit lower and flexibility of the model could be a little bit lesser

noone

how about not using reg images? will it give realistic results?

Furkan Gözükara

1 : between 10 to 20 2 : it depends on gpu and number of images. on windows i get like 1.5 second / it with rtx 3090 3 : quality will be same. it saves vram but slows down training

noone

thank you for this valuable information just have few questions How many images (minimum) I should provide? How long it takes to finsh the training? What if i wnat to enable the gradient checkpointing will it reduce the quality?

Furkan Gözükara

i see. ye sadly 8 GB very limited :/

Zsédely Ruben

Thanks, I used these settings on lora with default LR and got flexible yet fairly accurate results. Right now I have 8GB VRAM so I cannot do dreambooth training yet.

Furkan Gözükara

you pc must be using shared vram. so you must have a config error somewhere. are you on pc or runpod? if on runpod did you kill auto1111 web ui instance?

Samuel

160 hours for 13K steps in a 3090... what am I doing wrong?

Furkan Gözükara

Use best_settings_24_gb_VRAM_config_no_xformers.json . if you get out of vram error or if it uses shared vram enable xformers too

Furkan Gözükara

because your config is wrong. you have --network_dim=8 which means you are training LoRA however you are using learning rate of DreamBooth :)

Miguel Jeronimo

I've been trying to run this configs during the last week on py pc (Windows) and I had no success at all. Using the generated Loras has no effect at all. Using the same training images with but creating the configuration from zero using the kaggle training tutorial works perfectly (local or in kaggle). Can you spot the error in my configuration? accelerate launch --num_cpu_threads_per_process=4 "./sdxl_train_network.py" --pretrained_model_name_or_path="stabilityai/stable-diffusion-xl-base-1.0" --train_data_dir="E:/training2\img" --reg_data_dir="E:/training2\reg" --resolution="1024,1024" --output_dir="E:/training2\model" --logging_dir="E:/training2\log" --network_alpha="1" --save_model_as=safetensors --network_module=networks.lora --network_dim=8 --output_name="best_DreamBooth" --lr_scheduler_num_cycles="8" --no_half_vae --full_bf16 --learning_rate="1e-05" --lr_scheduler="constant" --train_batch_size="1" --max_train_steps="12160" --save_every_n_epochs="1" --mixed_precision="bf16" --save_precision="bf16" --cache_latents --cache_latents_to_disk --optimizer_type="Adafactor" --optimizer_args scale_parameter=False relative_step=False warmup_init=False weight_decay=0.01 --max_data_loader_n_workers="0" --bucket_reso_steps=64 --gradient_checkpointing --bucket_no_upscale --noise_offset=0.0 I'm running this in a Windows pc with a 4060 ti 16gb. Have tried to change bf16 to f16 with no success at all

Furkan Gözükara

you are welcome thank you so much for supporting me

Keith F

Thanks man, these jsons save so much time, appreciate your work

Furkan Gözükara

A6000 is good

Keith F

Oh on what GPU A6000?

Furkan Gözükara

use this one faster and best quality : 48_gb_VRAM_config_best.json

Keith F

Thanks, how about the 48gb one?

Furkan Gözükara

yes use all. it will use necessary number of them among all

StableMelter

Thank you for answering. So I should always use all the class images you have provided on this page, regardless of how many training images I have for the subject? What I mean is like if I have 15 training images or 30 training images, I should always use all the 3000+ training images for dreambooth training?

Furkan Gözükara

use this one on RTX 3090 TI (most cost effective): best_settings_24_gb_VRAM_config_no_xformers.json

Keith F

if fine tuning on runpod, which gpu is best for the best configuration?

Furkan Gözükara

I suggest this. make your all training images 1024x1024. use the 1024x1024 class images. if man , ohwx man, if woman ohwx woman. so ohwx is rare token man or woman are class tokens

StableMelter

what ratio do you recommend for training images : class images when doing lora or dreambooth training of a character with unique token?

Furkan Gözükara

All shared here : https://www.patreon.com/posts/massive-4k-woman-87700469 - i also sent a PM to you

Always Creative

Where can I find regularisation images?

Furkan Gözükara

ye we use T4x2 and fp16

Ec Jep

The kaggle options available on my free account are T4x2, P100 & VM v3-8. I was using T4x2

Furkan Gözükara

yep 100% could be related to the bucketing system bug. even in 1 case different aspect ratios were causing training to be completely broken and error

Guillaume Bieler

Thanks, that is interesting. I've been using dreambooth quite a lot over the last few months and I usually use Bucketing. That said, I've noticed that some trainings just don't work as well as others and I can't always tell why. Is it possible that bucketing has something to do with it? I remember reading somewhere that if your buckets are unbalanced, some have just one image for example, then it can bias the training. Is that the kind of problem you are trying to avoid here?

Furkan Gözükara

since all of my images are 1024x1024. also bucketing system had errors in past so i dont enable. moreover bucketing causes more VRAM usage. but if you want to train different aspect ratios you have to enable bucketing which i dont suggest until you become experienced

Furkan Gözükara

yes on kaggle you need fp16 due to given hardware. which GPU they give you?

Guillaume Bieler

I've noticed that you don't use bucketing in those configs, is there a reason?

Ec Jep

I tried doing kaggle 24gb DB training using your suggested parameters (bf16) but I get this error. Maybe I need runpod instead? "ValueError: bf16 mixed precision requires PyTorch >= 1.10 and a supported device."

Furkan Gözükara

I suggest RunPod. they have the cheapest prices. A6000 has 48 GB only 69 cent per hour

Ec Jep

What platform do you recommend for performing 48gb training. Kaggle? I don't mind paying for a few trainings to get the quality.

Furkan Gözükara

you can ignore it. use the gradio interface training button. the gradio interface will execute same command as that for you. just make sure you have prepared your dataset folders properly via dataset preparation tab of kohya

Furkan Gözükara

1st just download the links i shared. they should be sufficient. number is calculated as number of repeat x your training images count. make all of your training images 1024x1024 then you can use 1024x1024 class images

Jason Sutis

P.s. - I don't understan 'The very best found command is as below' - do you use that in addition to the .json config file? or if you use the config file it is baked in??? (sorry i'm a dumb video editor)

Jason Sutis

For the regularization images, how many should you strive for (with the 40 training images you mentioned). Also should the regularization images be 1024x1024 or can they be multi sizes.

Furkan Gözükara

Yep hopefully coming soon.

Jorge Reverte Sevillano

Is there gonna be a step by step video on this? Because i feel absolutely lost 🙄

Furkan Gözükara

I just saw will reply there. your training is messed up not correct

So Sha

For /it I sent you an screenshot through discord. For me it showed 15 hr at start but it took 2:30 hours finally. But didn't get good result.

Furkan Gözükara

unfortunately 4090 is having major problems. for example 3090 is getting 1 it / second when doing training with xformers enabled fast config. how much it / s you are getting? by the way 24 gb GPU has to use xformers. otherwise it will bottleneck the GPU VRAM. so with xformers enabled gradient checkpoint disabled 7200 steps taking 2 hours for RTX 3090

So Sha

Great result! So, how long did it take for you to train, and what was your hardware configuration? I tested your setup on 4090 24GB and 64GB of RAM. Here are the results: 135 hours without Transformers and gradient checkpoint. 19 hours with gradient checkpoint and Transformers. Screenshot: https://ibb.co/kKZW0nh It means I wasn't able to continue the training process. Therefore, we need to take hardware configurations into account when selecting our parameters.

Furkan Gözükara

I don't suggest any other changes. Make sure your training set quality is good

Furkan Gözükara

Sdxl much superior

Furkan Gözükara

Thanks

San Milano

Amazing work! I've been waiting for this, collecting images and I want to test it very soon

Rafał Bednarczuk

How is realism of photos and consistency of person between photos of SDXL compared to dreambooth 1.5 stable diffusion training?

Eric Born

Hi Furkan, this is great information as I'd been struggling to get Dreambooth to produce anything decent on SDXL. Would you suggest any major changes for training on a larger imageset besides the formula = train_imgs * 2(if class img used) * repeating_count * number_of_epochs? Keep up the great work!