Post "OneTrainer Stable Diffusion XL (SDXL) Fine Tuning Best Presets" by "Furkan Gözükara" from patreon

Furkan Gözükara

there isnt definitive formula but my flux tests shown this . square root of batch size x batch size 1 learning rate. so if learning rate is 0.0001 batch size 4 makes it 0.0002

yui

When I increase the batch_size, how should I set the learning_rate? For example, when batch_size=4

Furkan Gözükara

flux dev is best to train products. you can also use flux kontext to put products into different photos

Pablo Montero

Hi Furkan, so far your research, what's the best tool and configuration for a Product with logos (a bottle of parfume for example)? I have 15 images at 4K, and will use slices to enlarge the dataset and provide more information. Did you research this technique?

Furkan Gözükara

yes we never use big files upload to massed compute vi thinlinc client. you need to use 3rd party like hugging face or one drive or google drive. i recommend hugging face notebook : https://youtu.be/X5WVZ0NMaTg

Lee

Hi...I successfully trained via OneTrainer last year using your video tutorial, thanks for such useful information. At that time, with a 60mb internet connection the upload of files to Massed Compute was unbearably slow! Since that time I have upgraded to 900mb full fibre and was looking forward to doing some further training. However, upload speeds to MC are similarly terrible. Is this normal, are MC upload speeds from local machines via Thinclient typically very slow...or have I missed a setting somewhere that might improve the situation. Frustrating to lose an hour or two to uploading files before any training can be started! Many thanks.

Furkan Gözükara

i really dont trust samples. could be broken. after training generate grid to test

D Dunn

any idea why my samples are of men when Im training a girl? both concept txt files say girl

Furkan Gözükara

i would ask this to here if i were you because i dont know either :D https://github.com/Nerogar/OneTrainer/issues

bao tran quoc

Hi Dr Furkan, thanks a lot for your works. but i have an error. I have server connect by ssh. but i cannot run UI , so I train by cmd: ./run-cmd.sh train --config-path=configs/tier1_15.4GB_fast_v4.json. OneTrainer/modules/util/config/TrainConfig.py", line 697, in to_pack_dict with open(config.sample_definition_file_name, 'r') as f: FileNotFoundError: [Errno 2] No such file or directory: 'training_samples/samples.json' where can i find samples.json? thanks a lot

Furkan Gözükara

i don't have any atm but it truly reduces quality

Film Maker Official

Thanks for your reply. You mention the quality may decrease when using xformers. Do you have some example images on the quality degradation?

Furkan Gözükara

sadly i dont have a best config for LoRA , i didn't research recently. you can also increase dreambooth speed with xformers = true

Film Maker Official

Hi Dr Furkan, thanks for your reply. I use 4090 gpu to train. For sdxl finetuning, the speed of "tier1_10.4GB_slow_v4.json" or "tier1_15.4GB_fast_v4.json" are similar, about 1.10it/s, gpu memory are 10.4GB and 15.4GB respectively. For lora training, I use kohya to train, speed is about 4.22it/s, gpu memory is about 18GB. Some of the lora config items are: network_alpha = 1 network_dim = 48 network_module = "networks.lora" network_train_unet_only = true train_batch_size = 1 gradient_accumulation_steps = 4 lr_scheduler = "cosine" unet_lr = 0.001 mixed_precision = "fp16" loss_type = "l2" cache_latents = true no_half_vae = true xformers = true The speed difference between finetuning and lora may be cause by the network size. In lora config, I only use network_dim=48, which is much smaller than your suggesion of network_dim=256 Could you share the best SDXL lora training config for OneTrainer? I like OneTrainer than kohya. OneTrainer saves everything during training, and can resume training easily. But kohya seems only can resume the weight, but the scheduler's state are lost when resuming training.

Furkan Gözükara

actually it is faster than best LoRA config. which configs have you tested and compared?

Film Maker Official

Hi Dr Furkan, thanks a lot for your contribution. The training speed of finetuning seems much slower than the lora training. Are there any configs that speed up the finetune traing? The gpu memory is not an issue, I may rent 40GB or 80GB GPUs for training. Looking forward to hearing for you.

Furkan Gözükara

i am glad you solved sorry for late reply

abaj006

Never mind, watched your video again, the part about the Bucketing with OneTrainer.

abaj006

I have been getting great results with your Preset "tier1_15.4GB_fast_v4.json" and following your tutorial video. One question, do the Regularization images and the Training Images need to have the same aspect ratio when using OneTrainer?

Furkan Gözükara

you can have both. put both under reg folder. name them as 1_man and 1_woman - it will use both - but training time will increase

Nikola Popovic

Thanks for answer, what about regularization? Use man 5200 images for example and woman images at same time, tag them both as man and woman

Furkan Gözükara

in that case you can caption them like ohwx man and bbuk girl (i assume woman). with flux this doesnt work atm but with SDXL it should work. also if you can get both in the same picture, it works great for flux as well

Nikola Popovic

Regards, I have a relatively simple question, lets say we want to do a training for 2 persons. One is man other is female (kids for example). How can we approach it ? tag one person with token A, other with Token B? . If different gender, what would be reference set?

Furkan Gözükara

The explanation of this that either OneTrainer some new coding slowed it down or the latest libraries they use for SD3 and FLUX . 2.4 second without gradient checkpointing is too slow. you can enable SDPA / xformers attention but it slightly reduces quality

Maxiking

Thanks for all the detailed informations, you are doing really good work! I have tried to go through the massed compute route with 48 GB VRAM and used the newest 15.4 GB json file. Also I have disabled the gradient checkpointing as you have shown in the video. But nevertheless I get only 2.4s/it and not 1.4s/it in training, as you have mentioned in your post. As I have seen in nvitop, also only 34 GB from the 48 GB are used. Do you know, why there could be such a difference in the iteration duration, even if I have done the configuration the same as you have done?

Furkan Gözükara

Ye it takes some time not make mistakes

PropaGandalf

I realised I had put save evert 10 and I'm just making 10. So it had not saved any. But lucky there is the save now button. So I got one saved now on case something happens it's training yhe 5th now

PropaGandalf

Yes. It is working. Just very slow. After 12 hours it has only made 4/10

Furkan Gözükara

200 may cause overtraining for 82 images. maybe go 100 and see. always save checkpoints. currently config file works right?

PropaGandalf

I have 82 images in my set. Should it really be making 200 epochs? One seems to take about 2 hours. In any previous training tests I did, I only had about 6 to max 10 epochs. I am using the 15GB json ,and otherwise seems to work well. Before I joined the got your file, It kept crashing even though I tried to follow your youtube video. ( I am using 4080 16GB)

Furkan Gözükara

sure you are welcome

Christopher Ritter

Thanks for pointing that out! I was using 'Fine Tune' instead of 'LoRA' in OneTrainer. 🫢

Furkan Gözükara

this is for fine tuning not for lora. you sure you trained lora or fine tuned model? and this can be used for any task with right dataset

Christopher Ritter

Can these configs be used to train a style as well as a person? I downloaded 50 images of people wearing costumes and trained an SXDL LoRA model along with 5200 regularization images. Everything cropped to 1024x1024. After 8 hours of training, I couldn't spot any difference when I tested the LoRA model.

juggzz143

Training pony is the exact same as training xl.

Furkan Gözükara

you are welcome

Neto Leutwiler

Thank you so much! :)

PM

Pony trained fine with his previous settings, I havent tried the new ones. A trick for more photorealistic images, use a realistic pony merge AND train again on a regular SDXL model of choice (so two separate trained models in the end). Then in only A1111 or Forge (only works on these as far as I know, because of how they handle refiners), use your trained pony as base. Set steps to 50 (yes, that's a lot). Check the Refiner checkbox, set your SDXL trained model as the model in the refiner, and set the switchover to 0.6 (or you can play around with values but 0.6 is a good start). For best results, enable Adetailer and use the faceyolos model, and under adetailer settings drop down "Inpaint" and select "use separate checkpoint" and select the SAME SDXL trained model that you're using as a refiner. Write your prompt, and boom, flexibility of pony with the realism of SDXL.

Furkan Gözükara

i just did :D

Neto Leutwiler

Hi Dr Furkan, thanks a lot for your updates. Can we apply this best configs on kohya as well? If its possible we use this file "tier1_15.4GB_fast_v4" or the other one of kohya "Tier1_48_GB_Faster" with some modifications?

Furkan Gözükara

good question. I didn't test on Pony sadly and as far as i know it is different than other models significantly. you can try. Also I am testing 2 more new settings right now so config may get updated and become better

Pew

Hey Doc, do you know if these new settings would apply to finetuning a Pony model or does that require a different approach than what your scripts and training entails?

Furkan Gözükara

evet bugün bir test planlıyorum kohya vs onetrainer ve yeni bir video şart

Cemil Hacimahmutoglu

Hocam bu train işi artık iyice karıştı valla. Sil baş yaparak bulduğunuz en sağlam ayarlarla ve yönlendirmelerle yeni bir video ile bu işi sonlandırsanız çoook iyi olur.

Furkan Gözükara

today i am going to make training on both to compare hopefully. so we can see which one. i will use best settings for both

shen oracle

onetrainer or kohya_ss,which one is better?

Furkan Gözükara

you don't have to re-cache unless you change their captions or some other certain parameters that can affect the cache

Furkan Gözükara

at the bottom of above post tier1_15.4GB_fast_v2.json tier1_10.4GB_slow_v2.json

Furkan Gözükara

hello. i dont suggest lora training unless you have 8 gb vram. fine tuning certainly yields better results and you can extract lora as shown here : https://www.patreon.com/posts/extracted-lora-108634568

shen oracle

could u test lora?

Daniel Limia Aspas

Where are the new attachements? I can't seem to find them

Ezequiel Casas

Caching the 5200 reg images before each new training is too slow, it's normal? I have a 3090 and an HDD 7200 rpm, also a SSD maybe I will have to pass the dataset or one trainer itself to an SSD?

Furkan Gözükara

yes there is a way with kohya gui i explained here : https://www.patreon.com/posts/full-workflow-sd-98620163 follow images instructions to extract lora

Zach Shukan

I used your settings to do an SDXL finetune with OneTrainer and they worked great, but is there a way to get a LoRa from that checkpoint that gets generated?

Furkan Gözükara

thank you so much. i appreciate your all comments

reaper557

Finally going to try for a SDXL model -> LoRA after getting a successful SD1.5 last night. It is a relief to have access to your research, configs, testing, and examples when starting something new. Every project now begins with a quick check of your Patreon Exclusive Index list for related topics. Thank you very much, your hard work is making my life so much easier! (>'.')>[<3]

Furkan Gözükara

nice let me know results

Hendrik Luehrsen

I've just found, that for my training set even 0.000008 seems too high. I've had good results on 5e-6 with around 75 epochs. After that it was starting to artefact. I'm now down to 1e-6. Now I just have to wait a few hours to see what happens. :D

Furkan Gözükara

i recently reduced learning rate on kohya since it was broken. it is reduced to 0.000008 from 0.00001 for u-net. masked training didn't change learning rate for me. what masked training does that it generalize model better but then anatomy problems occur. i made a comparison here : https://medium.com/@furkangozukara/onetrainer-fine-tuning-vs-kohya-ss-dreambooth-huge-research-of-onetrainers-masked-training-d2694126ef4e

Hendrik Luehrsen

First off, thank you for doing all that research! The google colab times feel like a lifetime ago. To my question/finding: I am currently training base SDXL with dreambooth using tier1_slow. My set has 20 masked images. 200 epochs take me about 7 hours on my 3090TI. I've found that after around 120 images the samples begin artefacting, meaning the model is somewhat fried. But the likeness at 120 is still far away. Have you made any experiments with a drastically lower training rate? How about masked training, does the mask lead to higher learning and faster overfitting?

Furkan Gözükara

as a base model you can give your last saved checkpoint and continue from that

Der Sandmann

If it turns out that there were not enough epochs, can I “extend” the training with additional epochs or do I have to start all over again?

Furkan Gözükara

No formula but to be safe do 150 epoch save every 15 epoch a checkpoint.

Der Sandmann

How many epochs should I use with 40 training images of a person? Is there a formular?

Furkan Gözükara

yes it randomly picks among 5200 at every epoch. it will pick 20. real images yields better

Artie Medvedev

Hello. Thanks for presets. I have a question. Let's say I have 20 images of a person to use for model finetuning in OneTrainer. Also I have 5200 regularization images of a person from your dataset. To use "dreambooth" method I have to create another concept with those 5200 regularization images and ratio of 0,0769 in OneTrainer. The question is: will the OneTrainer choose random regularization images set for every epoch (or will OneTrainer somehow shuffle class images iteratively during training)? Also have you checked if class images generated by the base model give better results with OneTrainer then real images from your dataset?

Furkan Gözükara

i tried it but wasnt as good as adafactor config i have.

Visalyar

I know you claim finetuning and extracting is the best, but I would very much still prefer to tinker with LoRA's tbh, Had it down pretty perfect for SD1.5 and like to do the same for SDXL. Always worked with prodigy, could train a pretty good lora in half an hour or so and then perfect it further by merging with other LoRA's. Any suggested settings and have you messed around with prodigy in onetrainer?

Furkan Gözükara

ye sometimes patreon is having issues. you are welcome

Kay Tichelmann

yes, 9it seemed to have been a hccup on patreons side :) Thanks alot

Furkan Gözükara

they are. you can download from here : https://www.patreon.com/posts/massive-4k-woman-87700469

Kay Tichelmann

hi mr. furkan, the woman-regulation zip is not downloadable anymore

Furkan Gözükara

as you try more you will get more experience

George Gostyshev

kk will try it

Furkan Gözükara

sadly there is not a 1 for all solution. you can do training on Massed Compute cheaper, and make batch size 4 5 to train faster. 31 cents per hour : https://youtu.be/0t5l6CP9eBg

George Gostyshev

Yeah I want to create a full fine-tuned model and then extract lora from there. I've already did some and wanted to compare my results with your latest configs. It's not for commercial use - just for my own needs - trying to create nice stylized lora with images I like) My main question is about steps, epochs and such. Because with 15 pics I understand how it will go and in what time and with such amount of pics - I can't. All images are captioned. Previous model I did with same dataset in Kohya made in 14 hours on runpod a6000...

Furkan Gözükara

you just need to caption them properly according to your desired results. what you want to achieve in the end determines how you should caption. you want to fine tune model? what you want to do? I can also give you a private lecture if you are trying to make a commercial model

George Gostyshev

I do have 390 images to train. It's all separated by type: animals, characters, creatures, icons, illustrations. What's setting I should care most with provided configs? Could you please write some tips on what settings will you change to what? Apprciated

Tom Bloomingdale

That seems to add detail but to a face that looks nothing like mine - the face is consistent, but it is not my face.

Tom Bloomingdale

replied to the wrong comment

Tom Bloomingdale

I wasn't using that and will try - but the faces weren't bad they just don't look like me, they have a consistent look but not of me

Furkan Gözükara

i think all custom models have baked VAE of the accurate one. if model is popular i don't think they wouldnt have. Also all safetensors have baked VAE nevertheless. Either it is SDXL 1.0 or fixed SDXL 1.0 VAE

mike oxmaul

What if the model doesn't say its got it baked like these ones?

Furkan Gözükara

you dont need to baked is good. i said in new tutorial that i am editing

mike oxmaul

when training on custom models like realistic stock photo 2 or realvis4, do i need to specify the VAE still? Some models say it's baked in, others say nothing. what difference would i get from having no vae to having the vae? is there a best one for sdxl? i see there are lots of different revisions.

Furkan Gözükara

you are welcome

Furkan Gözükara

are you using adetailer to fix the face after generation? that is super important. please let me know after trying

Tom Bloomingdale

Hello - this is working in that it is training and finishing. The results are fairly consistent and look like the same person (kind of) but nothing like the me (the subject). I have tried a few different models as a base, and gone through the settings as you describe in your video/writings several times. Any idea why I am not getting results as expected? This is SDXL, the tier1_10.4GB_slow settings, 200 epochs.

mike oxmaul

Thank you

Furkan Gözükara

in SDXL we don't use EMA. it doesn't improve. I tested thoroughly

mike oxmaul

Why not use EMA? Previous post said using EMA made this better than Kohya.

Furkan Gözükara

the best would be train a model for each one and extract lora for each one

Fruit

Hello Mr. Furkan! Thank you. It works. I ran into another problem: the saved model had learned the style of the dataset perfectly, but it didn't remember the details of the character (hairstyle and little hair ornaments). If my goal is to train multiple characters on an animation model, what would be a good approach to use?

Furkan Gözükara

if you don't select VAE file it is ok. it will use embedded VAE of the model

Fruit

Yes, i have seen your video many times. I don't know why but if i don't select vae file, it is okay.

Furkan Gözükara

well it appears your setup is wrong for some reason. have you watched this? https://youtu.be/yPOadldf6bI

Fruit

Hello Mr. Furkan, thank you for your great work. it looks like OneTrainer cannot load the selected model, what's wrong? I have already read comments. but it doesn't work. Traceback (most recent call last): File "C:\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 432, in load_config config_dict = cls._dict_from_json_file(config_file) File "C:\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 554, in _dict_from_json_file text = reader.read() File "C:\Users\asd\AppData\Local\Programs\Python\Python310\lib\codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbf in position 25705: invalid start byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 296, in load model = self.__load_ckpt(model_type, weight_dtypes, model_names.base_model, model_names.vae_model) File "C:\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 186, in __load_ckpt pipeline.vae = AutoencoderKL.from_pretrained( File "C:\OneTrainer\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\OneTrainer\venv\src\diffusers\src\diffusers\models\modeling_utils.py", line 567, in from_pretrained config, unused_kwargs, commit_hash = cls.load_config( File "C:\OneTrainer\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 436, in load_config raise EnvironmentError(f"It looks like the config file at '{config_file}' is not a valid JSON file.") OSError: It looks like the config file at 'C:/3webui/webui/models/VAE/sdxl-fp16-.vae.safetensors' is not a valid JSON file. Traceback (most recent call last): File "C:\OneTrainer\modules\ui\TrainUI.py", line 477, in __training_thread_function trainer.start() File "C:\OneTrainer\modules\trainer\GenericTrainer.py", line 113, in start self.model = self.model_loader.load( File "C:\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 304, in load raise Exception("could not load model: " + model_names.base_model) Exception: could not load model: C:/3webui/webui/models/Stable-diffusion/animagine-xl-3.1.safetensors If i use SDXL 1.0 base model, File "C:\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 436, in load_config raise EnvironmentError(f"It looks like the config file at '{config_file}' is not a valid JSON file.") OSError: It looks like the config file at 'C:/3webui/webui/models/VAE/sdxl-fp16-.vae.safetensors' is not a valid JSON file. Traceback (most recent call last): File "C:\OneTrainer\modules\ui\TrainUI.py", line 477, in __training_thread_function trainer.start() File "C:\OneTrainer\modules\trainer\GenericTrainer.py", line 113, in start self.model = self.model_loader.load( File "C:\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 304, in load raise Exception("could not load model: " + model_names.base_model) Exception: could not load model: C:/3webui/webui/models/Stable-diffusion/sd_xl_base_1.0.safetensors

Furkan Gözükara

hi that is lora. i dont suggest lora . these are fine tuning : https://www.linkedin.com/posts/furkangozukara_why-i-dont-research-lora-training-because-activity-7164700874097856512--Ecj/?utm_source=share&utm_medium=member_desktop

Marian Ban

Hi, what are the correct settings for network rank and dimension? These are not included in the json files. I'm asking about the Kohya settings. Thanks

Furkan Gözükara

it is a research thing for compaines.

Furkan Gözükara

thanks a lot. after big one sure i can add.

George Gostyshev

That's really great work done! Appreciated) OneTrainer big tutorials could be nice. Also, can I ask\suggest to sometimes add not so big videos? Like videos about some small but useful technics. Anyway thanks again)

Ганга Монгол

how to train clothes?

Furkan Gözükara

you are welcome thanks for support

Kenji Klimaszewski

I see, thank you

Furkan Gözükara

we only have now Tier 1 for OneTrainer. most settings are same with Kohya but OneTrainer has extra VRAM reduction compared to Kohya

Kenji Klimaszewski

Btw, are the configs interchangeable between Kohya and OneTrainer? And what's the difference between tier 1 and 2?

Kenji Klimaszewski

I'll try both, thank you!

Furkan Gözükara

just added 10.3 GB config check it out

Furkan Gözükara

just added 10.3 GB config check it out

Furkan Gözükara

ok i tested more and you can train only text encoder but not U-NET. this way it reduces vram usage to way below but quality will be degraded. give it a try. on 12 gb your best option is SD 1.5 training : https://www.patreon.com/posts/very-best-config-97381002 it works amazing

Furkan Gözükara

ye this is newly added feature. i wonder if that reduced to 12gb i havent had chance to test yet. i think it is enabled by default

John Dopamine

(Not OP, but maybe look for this setting)? Option to disable the autocast cache for reduced VRAM usage https://github.com/Nerogar/OneTrainer/commit/3242026e77b29ee356ea8115f69f54b90fd9c7ce

Furkan Gözükara

it is already tweaked to the least. did you try it? i wonder if after recent updates working on 12gb or not

Kenji Klimaszewski

Hi, how can "tier2_SDXL_slow_13_GB" be tweaked to work on a 12GB card?

Furkan Gözükara

here i have for kohya which is better : https://www.patreon.com/posts/full-workflow-sd-98620163

Furkan Gözükara

hello i have for Kohya . i prefer it to extract lora : https://www.patreon.com/posts/full-workflow-sd-98620163

Kenji Klimaszewski

This! Any tutorial?

Furkan Gözükara

i don't use image variations. i think it is like cropping them or flipping them

Karl B

I have a question, on the Onetrainer wiki it says if "latent caching" is on as it is in your file, then you need to use more than 1 on the "image variants" setting under concepts. I have about 50 photos ready to train, Onetrainer doesn't really explain what do set that number to. Any ideas?

Furkan Gözükara

I plan to test new parameters just didnt have chance yet.

John Dopamine

There's a new branch (just about merged) w a new "fused base" feature that allows for a significant decrease in the amount of VRam needed to train. If you haven't seen this yet, keep an eye out. With 24gb you can now train w float32, or large batch sizes like 14. It also should allow people who have cards under 16gb train now also. Mentioning this after seeing you updated after reviewing stochastic rounding. This discovery kinda came from that (which came due to Cascade not training at all). Perfect timing w SD3 right around the corner (please let them not backtrack).

Furkan Gözükara

you can set it to use single txt file as caption or read txt files same name as training images file names

Furkan Gözükara

in SDXL EMA doesnt work good so i dont use. works good on SD 1.5

Javi dltr

in sdxl config 1 ema is off but decay is 0.999?

Javi dltr

how do we use captions here for reg and for data?

Furkan Gözükara

Thank you John. I noticed this like 2 days ago and I am testing and comparing the effect as you said. Super important. Thank you also bringing to my attention.

John Dopamine

The OneTrainer developer has added support for "Stochastic Rounding" in the optimizer settings for Adafactor. This new rounding method was found to be essential for any training of Stable Cascade using bfloat16 (only way on consumer gpus). Without this rounding method no training would be learned until the LR was put very high so the little values that were getting rounded down to zero would instead round up to a value. Anyway, this new toggle/feature definitely has an effect on SDXL training (and probably less so SD1.5) as well. Perhaps it even fixes the issue where one of the SDXL text encoders doesn't seem to learn much at all during training while the other does. 1e-05 is suuuuper quick to converge w/ stochastic rounding enabled which could be good or bad. Basically it's just different and quicker. Stability or any company training a base model would use float32 or mixed precision to train so they don't have this issue. (Actually: Can you say if you trained using fp16, bfloat16, or even float32/mixed when using Runpod to get these best case settings? They didn't really match w/ the LR values I'd come to be comfortable with. However maybe w/ the rounding fix I will have to try again w/ the 1e-5 unet / 3e-6 txt-encoder LRs) Curious if you would ever possibly revisit your trials on this and see if "Stochastic Rounding" can better dial in SDXL training. If so a new learning rate for UNet and TXT encoders will definitely be required. I'm personally trying between 1e-06 and 1e-05 - but that's w/o touching the txt encoder rates (just keeping them all the same). If you ever get a moment to rerun a few ballpark trials to give a thought on this new "fix" that came about due to Cascade failing it would be much appreciated. "John Dopamine"

Furkan Gözükara

it is for all. but for art styles and concepts, the dataset that you need to have and your captioning strategy changes. and according to your number of training images, your epoch count changes

Mario Santiago

Are these settings intended for characters, art styles or concepts?

Furkan Gözükara

you can use depending on your training task. if style of course use captions. but if only a person i still prefer only rare token + class token.

Pablo Montero

and why don't you use elaborated captions with OneTrainer?

Furkan Gözükara

so that fixed? yes i think it requires diffusers model in input

Furkan Gözükara

no that is not mine. i use ohwx man or woman for training persons. for training style i compare kosmos 2 , llava, blip 2 and such

Pablo Montero

By the way Furkan, is this your article? Why are you using in this training such short captioning method? https://www.reddit.com/r/StableDiffusion/comments/118spz6/captioning_datasets_for_training_purposes/

Pablo Montero

apparently the VAE must go as Diffuser Model. You can use in the VAE box: madebyollin/sdxl-vae-fp16-fix

Furkan Gözükara

can you try SDXL base 1.0 and let me know the results?

Pablo Montero

Hello Mr. Furkan, it looks like OneTrainer cannot load the selected model, what's wrong? Paste long console message: activating venv C:\AI\OneTrainer\venv A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\models\lora.py:300: FutureWarning: `LoRACompatibleConv` is deprecated and will be removed in version 1.0.0. Use of `LoRACompatibleConv` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`. deprecate("LoRACompatibleConv", "1.0.0", deprecation_message) C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\models\lora.py:384: FutureWarning: `LoRACompatibleLinear` is deprecated and will be removed in version 1.0.0. Use of `LoRACompatibleLinear` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`. deprecate("LoRACompatibleLinear", "1.0.0", deprecation_message) Traceback (most recent call last): File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 275, in load model = self.__load_internal(model_type, weight_dtypes, model_names.base_model, model_names.vae_model) File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 56, in __load_internal with open(os.path.join(base_model_name, "meta.json"), "r") as meta_file: FileNotFoundError: [Errno 2] No such file or directory: 'C:/AI/SD_Models/Stable-diffusion/SDXL-based/SDXL/formulaxlXLComfyui_v20Pruned.safetensors\\meta.json' Traceback (most recent call last): File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 282, in load model = self.__load_diffusers(model_type, weight_dtypes, model_names.base_model, model_names.vae_model) File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 97, in __load_diffusers tokenizer_1 = CLIPTokenizer.from_pretrained( File "C:\AI\OneTrainer\venv\lib\site-packages\transformers\tokenization_utils_base.py", line 1925, in from_pretrained raise ValueError( ValueError: Calling CLIPTokenizer.from_pretrained() with the path to a single file or url is not supported for this tokenizer. Use a model identifier or the path to a directory instead. Traceback (most recent call last): File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 428, in load_config config_dict = cls._dict_from_json_file(config_file) File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 550, in _dict_from_json_file text = reader.read() File "C:\Python3106\lib\codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbf in position 25705: invalid start byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 289, in load model = self.__load_safetensors(model_type, weight_dtypes, model_names.base_model, model_names.vae_model) File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 237, in __load_safetensors pipeline.vae = AutoencoderKL.from_pretrained( File "C:\AI\OneTrainer\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\models\modeling_utils.py", line 569, in from_pretrained config, unused_kwargs, commit_hash = cls.load_config( File "C:\AI\OneTrainer\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 432, in load_config raise EnvironmentError(f"It looks like the config file at '{config_file}' is not a valid JSON file.") OSError: It looks like the config file at 'C:/AI/SD_Models/VAE/sdxl_vae.safetensors' is not a valid JSON file. Traceback (most recent call last): File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 428, in load_config config_dict = cls._dict_from_json_file(config_file) File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 550, in _dict_from_json_file text = reader.read() File "C:\Python3106\lib\codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbf in position 25705: invalid start byte During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 296, in load model = self.__load_ckpt(model_type, weight_dtypes, model_names.base_model, model_names.vae_model) File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 186, in __load_ckpt pipeline.vae = AutoencoderKL.from_pretrained( File "C:\AI\OneTrainer\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\models\modeling_utils.py", line 569, in from_pretrained config, unused_kwargs, commit_hash = cls.load_config( File "C:\AI\OneTrainer\venv\lib\site-packages\huggingface_hub\utils\_validators.py", line 118, in _inner_fn return fn(*args, **kwargs) File "C:\AI\OneTrainer\venv\src\diffusers\src\diffusers\configuration_utils.py", line 432, in load_config raise EnvironmentError(f"It looks like the config file at '{config_file}' is not a valid JSON file.") OSError: It looks like the config file at 'C:/AI/SD_Models/VAE/sdxl_vae.safetensors' is not a valid JSON file. Traceback (most recent call last): File "C:\AI\OneTrainer\modules\ui\TrainUI.py", line 466, in __training_thread_function trainer.start() File "C:\AI\OneTrainer\modules\trainer\GenericTrainer.py", line 116, in start self.model = self.model_loader.load( File "C:\AI\OneTrainer\modules\modelLoader\StableDiffusionXLModelLoader.py", line 304, in load raise Exception("could not load model: " + model_names.base_model) Exception: could not load model: C:/AI/SD_Models/Stable-diffusion/SDXL-based/SDXL/formulaxlXLComfyui_v20Pruned.safetensors

Pablo Montero

Where can I find more info about this extraction process?

Furkan Gözükara

open a cmd and type nvidia-smi. i will reply you there

John Tomlinson

How do I verify my VRAM usage? As far as I know it's the latest but I'll check. I'm on the discord as OhTheHueManatee.

Furkan Gözükara

Can you verify your VRAM usage before trying preset? Also are you using latest version of the OneTrainer? If you message me from discord i can connect via anydesk and check it out

John Tomlinson

Regardless of which Tier presets I use I get the CUDA out of memory error. I have a 16GB card and have ran other trainings with different settings just fine before. What setting in the presets may be doing this? I'm mainly trying the 13GB one.

Furkan Gözükara

hello. sadly i cant tell details without seeing exactly what you are doing. but text encoder 1 is trained and text encoder 2 is not since i did extensive testing and training text encoder 2 only made the model lose its generalization meanwhile didn't improve similarity

Felipe Cortes

Hello, using onetrainer, I tried to create a checkpoint, but for some reason it says it can't find the basemodel (onetrainer error I guess), so I created an SDXL lora instead, which worked quite well for me, creating different situations and different clothes but using lora it does not apply styles, 2d, art etc, it only looks photographic, what do you think is the reason for that? Why don't I use regularization images? Why is the text encoder turned off? Or why does the description of my images of a man say "a man" and not "a ohwx man"? Or something else? thank you

Furkan Gözükara

great

Anduvo

It works, thank you

Furkan Gözükara

your lora extraction not accurate. extract regular lora 128x128 rank and use 0.00001 difference and float precision

Anduvo

Hi Furkan, do you know why when I train a dreambooth model with OneTrainer and it makes good images but when I extract the lora with kohya, the lora makes undertrained images?. lora 64 alpha 1

Furkan Gözükara

200 epochs probably too much for 400 images. what are you training?

Wktra

Thank you. I started training with 400 images this morning. I think this will take more than 24 hours. 😭 200 epochs...

Furkan Gözükara

yes for art styles i suggest "ohwx style, captioning" , so captions are helpful. reg images are not necessary. as many as images training better. at least 100

Wktra

Do you have any advice on training an art style? Would I use the same settings as your presets? Also, what is the minimum number of training images to use for a style? Any tips on captioning for styles? Are regularization images necessary? Again, thank you for your hard work.

Furkan Gözükara

that is weird i think developer had fixed it. i can try later hopefully

Furkan Gözükara

true

Furkan Gözükara

yep possibly

Wktra

Also, I get errors when I choose the SDXL VAE in Onetrainer. So I trained without it. The results are still good, but does anyone else have this problem?

Wktra

I trained my own original anime character using your methods, but the difference is that I used 100 training images and no regularization images. The results are good. It's very difficult to decide which epoch is best. It takes 12 hours to train and test results so I have to be careful about the details I change so I can improve it.

Anduvo

I see, I must had an older version of your parameters, I will re-download

Furkan Gözükara

kohya SDXL preset also trains only 1 text encoder. the second text encoder learning rate is set to 0

Anduvo

Hi Furkan, your kohya sdxl presets train both text encoders and the oneTrainer presets only text encoder 1 , is there are reason or a benefit for excluding the second text encoder in oneTrainer ?

Furkan Gözükara

yep. you can also compare different values but i used that works decent

Wktra

I think you're right. Do you use the same settings as SD1.5 on kohya to extract? Minimum difference: 0.00001 and Rank 128 and select SDXL?

Furkan Gözükara

I use kohya. he also fixes errors which I report

Anduvo

do you use kohya or other tool to extract the lora ?

Furkan Gözükara

sure

Furkan Gözükara

sadly i dont have. I prefer to train DreamBooth and then extract lora. this will give you a much better LoRa model

Wktra

I'm curious, thanks to your setup files, I am FINALLY able to train SDXL with my 15GB 3080. Do you have any guides for SDXL Lora training with Onetrainer?

Wktra

Oh!!! Then it's time for me to experiment with this too!

Furkan Gözükara

single text file contains single text that is used by all images in that concept

Furkan Gözükara

you should use all 1024x1024 until you understand and get good results. after you become more experienced you can play with bucketing and test results

Furkan Gözükara

it is true all 15 images uses single txt file as ohwx man. i find this is working better than captioning for training a person

Wktra

Also, why don't you turn on bucketing for different ratios of training images?

Stefan Dragisic

that solved it, thank you!

Wktra

I just want to be clear: so ALL of your 15 images only point to one text file. And the one text file only has "OHWX man"? No other words, no other descriptions?

Wktra

You wrote "Prompt path examples are a txt file that contains ohwx man (training images rare token + class) as text and a txt file that contains man (reg images class) as text" ... This is unclear. What this tells me is that you don't caption each training image? You just use one text file that ONLY has the words "ohwx man" for each training image? This is confusing because in these comments, you mention the we should use the LLava captioner??? Does one text file contain all of your training captions? Could you please give an example of what the training text file looks like?

Furkan Gözükara

you are welcome

Wktra

Thank you for the clarification!

Furkan Gözükara

hello. did you update your one trainer to latest version? please do git pull

Furkan Gözükara

hello. i just added the information you asked. please refresh page and check bottom

Wktra

Can you please add to the post how many training images you used and how many regularization images you used? It's very important.

Stefan Dragisic

Seams that latest tier1_sdxl present has an error: ~~~~~~~~~~~~~~~~^^^^^^ KeyError: "{'__version': 0, 'optimizer': 'ADAFACTOR', 'adam_w_mode': False, 'alpha': None, 'amsgrad': False, 'beta1': None, 'beta2': None, 'beta3': None, 'bias_correction': False, 'block_wise': False, 'capturable': False, 'centered': False, 'clip_threshold': 1.0, 'd0': None, 'd_coef': None, 'dampening': None, 'decay_rate': -0.8, 'decouple': False, 'differentiable': False, 'eps': 1e-30, 'eps2': 0.001, 'foreach': False, 'fsdp_in_use': False, 'fused': False, 'growth_rate': None, 'initial_accumulator_value': None, 'is_paged': False, 'log_every': None, 'lr_decay': None, 'max_unorm': None, 'maximize': False, 'min_8bit_size': None, 'momentum': None, 'nesterov': False, 'no_prox': False, 'optim_bits': None, 'percentile_clipping': None, 'relative_step': False, 'safeguard_warmup': False, 'scale_parameter': False, 'use_bias_correction': False, 'use_triton': False, 'warmup_init': False, 'weight_decay': 0.0}" when I load it

Nenad Kuzmanovic

It is not opening template, but GPU Cloud page... I've tried with search one trainer but no results...

Furkan Gözükara

i replied you. your mistake is not setting prompts source

Anduvo

Couldn't make a video but sent you some screenshots via email

Furkan Gözükara

maybe you didnt set the images directories and their prompting accurately? if you make a quick video of how did you setup your concepts i can tell your possible mistake

Anduvo

Hi Furkan, first time trying OneTrainer instead of kohya to train SDXL dreambooth (fine tuninig). So I replaced the optimizer json, put the other presets in the presets folder. Now I took an old dataset with captions which gives good results in kohya (80 images), but using OneTrainer with your 14.5gb preset doesn't produce a correct model. It just produces kind of abstract images that doesn't resemble anything. Do you have any idea what I might be doing wrong ?

Furkan Gözükara

yes i am planning. there is a template but i didnt have chance yet : https://runpod.io/gsc?template=1fqh6661pb&ref=m0vk9g4f

Nenad Kuzmanovic

Are you planning to make a Runpod installer? I know you are bussy, but i can't resist asking :-)

Furkan Gözükara

you can try to collect older people from unsplash but that would take long. so do this. 1st train without reg images. second train with our existing shared reg images. and compare which one perform better

gao yu

could you make a quick tutorial on how to selecting the reg images for training a LORAFACE model to create a character of my 90-year-old grandmother.

Furkan Gözükara

i cant tell without seeing everything. perhaps try kohya?

likewisemarien

I dont know what I'm doing wrong, the face is only slightly similar when I trained it last night. It doesnt work

Furkan Gözükara

yes hopefully i will do still doing research for best parameters

Meito

could you make a quick tutorial on how to add the concepts and class images, its not working for me

Furkan Gözükara

yes LLaVA slow but really good

Hassan Alhassan

i captioned them using open AI xxl if i remember the name well, i am redoing it just finished captioning for 9000 images, its taking a lot of time to caption with LLaVA but it seams worth it

Furkan Gözükara

how did you caption them? did you try our LLaVA captioner? the caption would matter for such task

Hassan Alhassan

Any Idea on how to train the model for putting it in Civit, (not my photos) i tried training it with 3000 photos, but when i reach 40k steps it because like a bad-cartoonish look, once i pass 40k it doesn't look like good at all

Furkan Gözükara

thanks.

Hassan Alhassan

based on my findings, this is the list of rarest unbroken tokens with 4 letters, sorted by rarest to less rare. i use it because i usually train more than one person in the same model. though am not 100% sure if my findings are accurate. olis 148210 dits 148187 httr 148185 sown 148182 asar 148112 dori 148083 sohn 148080 suru 148053 veli 147979 sura 147938 toid 147937 onex 147929 apro 147864 wray 147756 maar 147740 peth 147733 enei 147719 ront 147694 inem 147689 gera 147680 miki 147676 rith 147663 fler 147602 tich 147582 minh 147547 arum 147545 enig 147543 hiko 147521 teer 147461 nery 147410 ohwx 146806

Furkan Gözükara

not that i know. it requires GUI so a template with GUI desktop mandatory. i plan to do tutorial hopefully soon

mike oxmaul

any runpod template where one trainer is already configured?

Furkan Gözükara

i see. wow if even 4090 not enough it is so bad :/

Hassan Alhassan

i tired the PixArt using all the methods they had on their web page non of them worked with me, i think the main one needs a GPU thats better than 4090, and the other trainings didnt work, specially with the limited information they provided for training

Furkan Gözükara

sadly i don't know it yet :( i plan to also train pixart and Wurstchen as well

Hassan Alhassan

this question might sound silly, i trained the Wurstchen model on using one trainer, and the model was in Diffusers format (folders with files inside them) when i dragged it to the models folder in SD.Next it didnt recognize it as a model, i tried to add it to the diffuser as well didint work, i never worked with diffusers before so not sure how to let the app know its there. i tried googling it but didnt find an answer that works

Furkan Gözükara

yes i generate images with adetailer usually. still doing search

Furkan Gözükara

actually when training a person i dont find it is better to use captions. but from sota captions try llava and blip2-flan-t5-xxl. dont forget to add prefix ohwx man to captions

Hassan Alhassan

i have 25 images + 25 images of 2 different people (i tried them seperate and together all same result), 140 epouch, with regulators 4050 images, repeats 0.0045, i used your Json files 18gb and the optimiser. i used wd14 captioning, and now i am rerunning it with Blip 2, which captioning do you recommend from SOTA ?

chriseppe

the results seem promising. Would love to see if it looks even more realistic without a surreal background. Did you simply generate the images without using Adetailer on top?

Furkan Gözükara

probably very related to your training set. how many images you have and how many epochs you did?

Hassan Alhassan

my model is getting over trained quickly, i used the 18gb and 14.5 gb both are getting over trained

Furkan Gözükara

something must be wrong.

RtBx

mnm i dont know what didnt worked, but after trained the model it only generates this kind of images. https://imgur.com/a/sTHqJCK Using your 14,5 gb settings file.

Furkan Gözükara

yes reg images made huge improvement : https://medium.com/@furkangozukara/experimenting-with-onetrainer-onetrainer-vs-kohya-realism-vs-stylization-reg-images-vs-0438950e9515

guangyu niu

is the reg dataset still needed for using the onetrainer process?

Furkan Gözükara

i would try up to 20 30 epochs. you know all about experimenting each dataset is different

Furkan Gözükara

almost equal but for realism i think onetrainer a little bit better. still testing though

guangyu niu

comparing with kohya-ss, which one gets a better result?

RtBx

one last question, and for 100 images? thank u

Furkan Gözükara

yes of course too many. for 319 images try 10 epoch and analyze results

RtBx

thank you, using your settings, it seems that you set 500 epoch, they are not too many for 319 images ?, or is allright ?

Furkan Gözükara

yes. 1 repeat for training dataset and based on training dataset and reg images dataset i set repeat that too. for example 5200 reg images 0.003 repeat means 15 image used at each epoch

RtBx

are you using repeats for the concepts?

Furkan Gözükara

sorry for late reply : photo of ohwx man wearing an expensive suit in a news studio, 2k, 4k, hdr, uhd, canon

Jhb

Whats is the prompt for Comaprison.Jpg

Furkan Gözükara

you need to caption them good then it will train for all those captions

seng sha

I have a batch of pictures that I want to fine-tune ckpt, but not a certain concept. What should I do?

Furkan Gözükara

not sure. i have done more comparison and added to attachments

Joel Maynard

oh nice, one more thing your hands and fingers also look better in this model, what is the reason for this?

Furkan Gözükara

yes it is more fast since uses lesser vram thus we can disable gradient checkpoint. also if you dont use reg images it halves the training duration but i am testing reg images effect right now

Joel Maynard

is it very fast compared kohya?

Furkan Gözükara

yes this one : https://runpod.io/gsc?template=zue4ub5xy4&ref=vfker49t

Furkan Gözükara

yes it is sufficient. you are welcome. fixed comparison.jpg

John Dopamine

Link for comparison.jpg above gives an error. Loaded your 18gb settings/optimizer earlier and first training test looked great. Thanks for researching one trainer - I always had issues getting it to train properly. Now I just need to figure out if I can train a concept (say likeness), then after add another and another etc, or if I need to create a large dataset to train all at once. Currently just trying the captions as one txt file and only putting the trigger name in the .txt. I assume that's sufficient for basic test.

diffusers

Is there a template link? I've tried to find it but can't 🙃 haha thank youi

Furkan Gözükara

I will tell him sure. thank you

Marco van der Merwe

Thank you for this! Testing these settings now. Kohya has been great but it's long overdue that we move to a better GUI. I'm loving Onetrainer but I havent managed to find the perfect settings like the ones you shared for Kohya. The only feature OneTrainer is missing now is a 'total time' calculator like how Kohya shows estimated time. If you have a direct line to the developer please request this small feature

Furkan Gözükara

I think OneTrainer has more advanced features but I need more experiment. It lacks still few things I told to the developer

Furkan Gözükara

I will rent a pod with multiple GPUs to test it with Kohya for you guys. you are right i delayed too much

Furkan Gözükara

I will rent a pod with multiple GPUs to test it with Kohya for you guys. you are right i delayed too much

Furkan Gözükara

RunPod has GUI template. but i am yet to test it there. on there you should be. i plan to test and cover that too

RtBx

same

Azamat Galimzhanov

How would you compare one trainer and kohya?

Pete Stueve

Listen I hate to be that guy so stoked for whatever you put up but I was just just looking for a 1.5 guide if you find the time would be amazing as I am using 1.5 / animatediff but I deleted all my old lora/models thinking I was on to sdxl and now I find myself back