FLUX LORA TRAINING + LORA TESTING WORKFLOW

Otniel Ocampo

Hi I Recently Upgraded to a 3090ti Card and all seems to work well but qhile doing the image to prompt I get this error message! Prompt outputs failed validation: Florence2Run: - Failed to convert an input value to a INT value: seed, randomize, invalid literal for int() with base 10: 'randomize' - Value not in list: task: 'True' not in ['region_caption', 'dense_region_caption', 'region_proposal', 'caption', 'detailed_caption', 'more_detailed_caption', 'caption_to_phrase_grounding', 'referring_expression_segmentation', 'ocr', 'ocr_with_region', 'docvqa', 'prompt_gen_tags', 'prompt_gen_mixed_caption', 'prompt_gen_analyze', 'prompt_gen_mixed_caption_plus']

Kyle Lauro

has anlyone else been able to get the workflow to test the Lora working? When I load the json, I set the advPlot Combo node to lora_name and "select" however when I try to select it just says "select widget for options"

Matteo De Mutiis

Solved: I don't know how, I had already tried closing and opening it several times, but I reopened it now and the workflow works

Matteo De Mutiis

Lora Testing... In the advPlot string node and in the advPlot combo node, in the node field, it shows me "Connect to xyPlot for options" and in the fields widget, start_from, end_with, and select, it shows "Select node for options." These nodes end up in advanced xyPlot, where the x_plot and y_plot fields are turned off/disabled. How can I fix this? Thanks!

James Stewart

Oh and likewise from your earlier video, removing backgrounds etc to make them transparent. Do you still do this for training clothing or an object if you don't want the background as part of the training data? Thanks @Aitrepreneur and the community (if they know!)

James Stewart

Hi K, back in one of your older videos on LORA training an item of clothing on SDXL you had to create a black censored square over faces. Should you do the same with Flux Gym?

Valentino Canale

in the FLUX_XY-PLOT_LORA_TESTING.json workflow, on the first Load LoRa (the purple one, under DualCLIPLoader), what should be put?

Erosi

the XY plot comfyui workflow just hangs on "Requested to load flux". the Gpu is running on 100% but it seems stuck? or just slow? (4060ti)

Simon

hey, i have a 3090 and use your settings but get only steps: 5.77s/it in your video you have around 2. why is yours faster? accelerate launch ^ --mixed_precision bf16 ^ --num_cpu_threads_per_process 1 ^ sd-scripts/flux_train_network.py ^ --pretrained_model_name_or_path "C:\Users\mail\Downloads\gym\fluxgym\models\unet\flux1-dev.sft" ^ --clip_l "C:\Users\mail\Downloads\gym\fluxgym\models\clip\clip_l.safetensors" ^ --t5xxl "C:\Users\mail\Downloads\gym\fluxgym\models\clip\t5xxl_fp16.safetensors" ^ --ae "C:\Users\mail\Downloads\gym\fluxgym\models\vae\ae.sft" ^ --cache_latents_to_disk ^ --save_model_as safetensors ^ --sdpa --persistent_data_loader_workers ^ --max_data_loader_n_workers 2 ^ --seed 42 ^ --gradient_checkpointing ^ --mixed_precision bf16 ^ --save_precision bf16 ^ --network_module networks.lora_flux ^ --network_dim 16 ^ --optimizer_type adamw8bit ^ --learning_rate 5e-4 ^ --cache_text_encoder_outputs ^ --cache_text_encoder_outputs_to_disk ^ --fp8_base ^ --highvram ^ --max_train_epochs 10 ^ --save_every_n_epochs 1 ^ --dataset_config "C:\Users\mail\Downloads\gym\fluxgym\outputs\volvoem90\dataset.toml" ^ --output_dir "C:\Users\mail\Downloads\gym\fluxgym\outputs\volvoem90" ^ --output_name volvoem90 ^ --timestep_sampling shift ^ --discrete_flow_shift 3.1582 ^ --model_prediction_type raw ^ --guidance_scale 1 ^ --loss_type l2 ^ --enable_bucket

roaddog4

That worked. Thanks.

Dave

The maximum image count is 150. More than 150 images causes the "Start" button to disappear.

Digital art

tell me how to download WORKFLOW FLUX_SIMPLE_REDUX.json I did not find the link

roaddog4

I installed this following your Pastebin procedure. Everything went well until I tried to start the training and have no "Start" button showing. How do I start the training by command line?

Mathilde Leroux

The trained Loras don't work well with GGUF models. Only works with the safetensor versions. T_T

Nathan Blake

My inpainting is working fine, but my outpainting, expands the picture but only fills that expanded space with an empty gray

Uncc234

Attempted another install after a fresh restart of computer. Seems to be working for now.

Uncc234

The process seems to hang at 'caching latents...' I've significantly reduced the batch files as a test, and it has hung at this point using a sizeable 24 gigs of RAM, neither the CPU or GPU are being utilized in resource monitor. I've let it sit for 8 hours. 4070 Super, 64 gigs of Ram, AMD CPU. [2025-02-22 14:08:41] [INFO] 2025-02-22 14:08:41 INFO Loaded AE: [2025-02-22 14:08:41] [INFO] import network module: networks.lora_flux [2025-02-22 14:08:41] [INFO] INFO [Dataset 0] train_util.py:2598 [2025-02-22 14:08:41] [INFO] INFO caching latents with caching train_util.py:1104 [2025-02-22 14:08:41] [INFO] strategy. [2025-02-22 14:08:41] [INFO] INFO caching latents... train_util.py:1153 I have attempted both the automatic and manual install and still get stuck at the same place. Any assistance would be appreciated. I've also attempted the 'Skip_cach_check' in advanced options.

Aitrepreneur

Bro what's wrong with you? You have posted everywhere yelling but you haven't even send me a DM explaining the issue...as I say in my videos i can't see every single messages hence why I ask people to DM me when they need help...

ahazy

The install doesn't work either! You need to make a new post so people don't waste their time and MONEY!!!!!!!!!!!

Clinton

I feel like you made the Flux XY workflow just for me because I was asking you for it. Yes, I know that others must have been, too, but still, it's great to have this tool. Thank you!

Kerfelox

https://huggingface.co/black-forest-labs/FLUX.1-dev/discussions/362 It seems i am not the only one with the same error problems

Kerfelox

I have accepted the license and it says "Gated model You have been granted access to this model" in the website. What should I do?

Mathilde Leroux

It's been fixed. It seems like a comfyui update earlier yesterday broke it. They fixed it with another update.

Aitrepreneur

Which part? Everything seems to work for me

Mathilde Leroux

Aitrepreneur

it's only from a model present on huggigface, not civitai. You can however dl the model from civitai and upload it onto your own free huggingface model space and use those data for the model download

Aitrepreneur

go here: https://huggingface.co/black-forest-labs/FLUX.1-dev and accept the license

Kerfelox

After using every setup steps you suggested and starting the train i get this message and does not continoue the process, i waited 4 hours as you mentioned it may seem it doesnt doing anything, however i doubt that with my RTX 3060 12gb i am too impatient :D, what should i do? license_str = license: other license_name: flux-1-dev-non-commercial-license license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md no samples

vik56

Anyone had success using flux models from civitai for lora training? Aitrepreneur mentioned editing the yaml file but I haven't had any luck. Let's say for example if I wanted to use Copax TimeLess rather than dev or schnell, how would I go about getting the copax checkpoint to appear in the dropdown and work for training? Would appreciate any advice.

Hash Browns

joined your patreon for the SDXL training, but then I jumped on Flux and was surprised at first to see you didn't have a video out for Flux Lora training yet. Anyway, used some of the same arguments from the SDXL Low-VRAM training while training in Flux, but I'm not sure if they have any actual effect and maybe you can answer after some more testing (like no-half-vae, cache-latents, fused-backward-pass). I might retrain some of my LORAs with the new parameters you mention in this video (mainly the multi aspect ratio), but so far Flux Loras look soooo much better even at 512x512 than 1024x1024 SDXL loras. also if anyone out there is like me running these on a 10GB VRAM and running FluxGym through Pinokio, you can simply download the fp8 version of Flux Dev and place that in FluxGym's checkpoints folder with the same name (flux1-dev) and same format (.sft)

AI Enthusiast

Official free one click installer right on the fluxgym github page. https://github.com/cocktailpeanut/fluxgym?tab=readme-ov-file#install

Virtamouse

Very cool thanks! Hunyuan lora training when???

Colja Carls

I tried Fluxgym and... well... it's easy, yes. It's for when you just want to try something out quickly. And you come pretty far with it. BUT at the time writing this, it does not have the option to save or load the configuration for your training. You want to try some small changes on what you did a day earlier? Have fun loading up the pictures again (and maybe caption it if you wanted) and make all the small changes to the configuration again and again. Hit F5 in the browser? Then your current training will continue in the background but you also have to prepare everything again for the next run. So Fluxgym has it's drawbacks but when you try heavy lifting on some cloud providers, Fluxgym is simply not an option. When going into the cloud, you strive for efficiency (preparing everything around the dataset and the configuration of the training) and you want to upload everything and start right away. That's what Fluxgym can't do, at least not at the moment.

sn1ze

if i want to train a character for NSFW, would you recommend to use a NSFW flux model instead of the flux1.dev model? if yes, which flux model is currently the best for uncensored NSFW?

VinPre

Finally, can't wait to try this out! Sadly you got me with that on my last day before I will be away for a few weeks XD. But it's ok when I am back I will get a 5090 and try this.