Create CONSISTENT CHARACTERS from an INPUT IMAGE with FLUX [ADVANCED WORKFLOWS]

Shaz Hossain

I keep getting toon style sheets... Why??

Tim Closs

Mick are the links above still active? I've just subscribed but can't get any of them to work. Thanks

true warrior

True Trash shove those 10$ up your white ass. This don't even look good

yang

When I run the workflow, Report an error：ApplyPulidFlux 'ModelPatcher' object has no attribute 'get_additional_models_with_key' How to solve this problem? I am troubled

JoJo

wow we gave this mf 10 bucks and he cant even reply with some basic fixes

Yeris

if you want to run on cloud, there is docker image can be used on discord, im using and work fine.

Emmanuel ramos

You will have to change. Remove the InstantX Flux Union ControlNet Loader. Add Load ControlNet Model. Add SetUnionControlNetType. Turn on both and then turn on SetUnionControlNetType in Apply Controlnet with VAE.

Chai Biscuits

This workflow eventually worked for me on Runpod with Pytorch 2.2 template, setting up comfyui and manager from scratch on a python venv. Still, it's not perfect. The output is sometimes cartoonish and the headshots look wonky most of the time.. it requires very high quality input. Don't try using a ComfyUi premade template, the ones available have dependencies that conflict with this workflow dependencies. I had to find out the hard way. Also the latest consistent character flux + pulid workflow torch version and other dependencies doesn t work with any runpod template. If the environment of the cloud instance doesn't match the workflow requirements and you have to upgrade or downgrade install torch cuda to different version it will break the environment and you will be stuck in a loop trying to solve dependency conflicts. Last thing I need to try is using docker on runpod. Caution with this, runpod instances already run on docker in backend and you can't run docker inside docker. There's a way around that... you can use bazel to create a docker template with the correct setup for this workflow... I have yet to try that... if I succeed and publish a template I will share it. If anyone knows any easier ways to achieve realistic consistent characters, please let me know.. I been using chatgpt. Been methodically checking the requirements.txt file of each custom node on the github repo and feeding it to help me figure any dependencies that going to conflict so I can install non breaking requirements in correct order and figure out most problems some people here stuck. I recommend it but be careful to not end up in dependency sorting loop, at which point your instance is compromised and it means your template pytorch cuda version is not compatible with this workflow, upgrading or downgrading will not help you and using a virtual environment won t protect you unfortunately

Cerdinak

I'd love to know why the Em Constants (Em1-Em5) for Emotions are not applying to the section 6. Background(s) of the workflow any longer. No error messages, just don't apply.

Roman Bradáč

im stuck on 'ModelPatcher' object has no attribute 'get_additional_models_with_key'

Антон Крайчинский

Is anyone having ApplyPulidFlux 'ModelPatcher' object has no attribute 'get_wrappers' . How to fix it?

Christian Lösch

Yes, please read the comments in the video. Someone explained the solution there. 😃

Kateryna Knyazevska

Hi dylan, did you solve it? Struggling with the same issue. :(

Kateryna Knyazevska

Hi, did you solve it? I'm trying for a really long time already :(

N Dilleesh

anyone using fluxgym ...?

randolph preisinger-kleine

I am working with a local installation, the portable version of ComfyUI. I proceeded as follows: first I looked at the general structure. Then I tried to find out what each individual node does. I had problems with some nodes. For example, the set and get nodes from the KJNodes package, cg use everywhere, and the nodes required for Pulid. I was able to fix the KJNodes problem by installing the latest version of KJ Nodes and then manually updating each SET/GET node in the workflow with "Fix Node". For Pulid I used the pulidflux.py patch. In addition, I replaced the existing sampler with a newer version and finally executed the requirements.txt. Other error messages such as "Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!" result, in my experience, from using an incompatible model or an incorrect combination of models. The workflow documentation is not always clear on this, so you have to try it out yourself. It is also very important to add the path to "python_embeded" to the environment variables in Windows. Otherwise, some nodes complain that something cannot be found. After doing all this, I still get Javascript errors. But this does not affect the functionality.

David

In my case, I was able to solve it modifying PulidFluxHook.py line 175. The key was to paste the end of the log in ChatGPT and ask for a solution.

Chai Biscuits

same issue I had the other day, remained unresolved -.- no one saying anything.

Chai Biscuits

how??? it's an absolute mess running this on cloud.

McLovin

I got everything running, the only thing that is not working are the faces. I choose for example "Face-5" and the face looks the same

Andrea Řečinská

What should I do if Im missing ViT-L-14 ? in DualCLIPLoader

Petr Šalša

i dont know why but face expresion node is skip and nothing is created. and nodes are in red circle. but dont know why, in consol i dont have any issue

dylan

Have you solved the issue? I think I ran into a same problem.

dylan

I am having the same issue. How you solved the issue yet?

randolph preisinger-kleine

Excellent work! After some minor fixes, error messages have gone, and everything runs smoothly.

Rodrigo Gallardo

It is unfortunate that I am experiencing difficulties. I encountered missing nodes during the graph loading process: Missing Node Types The following node types were not located: PulidFluxEvaClipLoader, PulidFluxInsightFaceLoader, ApplyPulidFlux, and PulidFluxModelLoader. Could you provide guidance on how to install these nodes? The process seems rather intricate. A tutorial video on the installation of Pulid Flux in ComfyUI would be greatly appreciated. Best regards from Miami.

Khjunxx

https://www.patreon.com/posts/create-from-with-115147229

Phillip Gimmi

where is the controlnet pose image?

Kevin Sozanski

I have issues with loading the ControlNet model on both the InstantX and Shakkerlabs Flux version mentioned in the installation gdoc: Error occurred when executing ControlNetLoader: 'pos_embed.proj.weight' File "/workspace/ComfyUI/execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ComfyUI/execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ComfyUI/execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ComfyUI/nodes.py", line 720, in load_controlnet controlnet = comfy.controlnet.load_controlnet(controlnet_path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ComfyUI/comfy/controlnet.py", line 433, in load_controlnet return load_controlnet_mmdit(controlnet_data) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ComfyUI/comfy/controlnet.py", line 342, in load_controlnet_mmdit new_sd = comfy.model_detection.convert_diffusers_mmdit(sd, "") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/ComfyUI/comfy/model_detection.py", line 500, in convert_diffusers_mmdit depth = state_dict["pos_embed.proj.weight"].shape[0] // 64 ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^

Kelly Michelle

Anyone else having issues with loading the PuLID Enhanced model?

Max Salamonowicz

I'm constantly getting head being too large during first step (composition of positions). Any idea how to overcome this issue? It (problem) wasn't present in non-pulid version (without input image).

Jim

Similarly, I updated PuLID from within ComfyUI (In Pinokio) and then replaced the pulidflux.py file from the XmYx fork, which basically adds the lines 79 and 80: "transformer_options={}, attn_mask=None," https://github.com/XmYx/ComfyUI-PuLID-Flux-Enhanced/blob/main/pulidflux.py I don't know if it slows it down or something else, but it works well enough for now.

Wei Choong Lee

I keep getting stopped out between Stage 3 and 4:got prompt Failed to validate prompt for output 369: * ImageBatchMulti 364: - Required input is missing: image_2 - Required input is missing: image_1 Output will be ignored Prompt executed in 1.92 seconds got prompt Failed to validate prompt for output 87: * (prompt): - Required input is missing: images * SaveImage 87: - Required input is missing: images Output will be ignored

Thomas Wahl

Found this discussions on reddit, possibly a bug in ComfyUI memory handling while swapping modules between GPU and CPU when free VRAM gets low. https://github.com/comfyanonymous/ComfyUI/issues/3179

Michael D Toal

ChatGPT fixed it for me. Edit the model.py file located at "...\ComfyUI_windows_portable\ComfyUI\comfy\ldm\flux\model.py". At the bottom of the document, find the line that says "out = self.forward_orig(img, img_ids, context, txt_ids, timestep, y, guidance, control, transformer_options, attn_mask=kwargs.get("attention_mask", None)) ", then delete the attn_mask parameter so it becomes "out = self.forward_orig(img, img_ids, context, txt_ids, timestep, y, guidance, control, transformer_options) ". Save a copy of the document first though because it is very touchy about extra space characters if you mistakenly make any. This is only a temporary fix though as it will revert on update. The node developer for PuLID needs to update their software and it will continue happening until they do.

Marceli Kosinski

I've followed the guide and yt video again and looks like all works fine :) thanks! cheers Marceli

MrNapz

I made some research, in my case i updated comfy, it can be related to update, so im rolling back.

KI Film

Definetly VRAM normally you would define where the models should run and if they are larger than for example your gpu it can get a bit tricky, sadly dont know a fix right now

KI Film

Go to the manager and click on install missing nodes, then install the wanted models and select them in the corresponding nodes. Hope that helps :)

MrNapz

same

Marceli Kosinski

SamplerCustomAdvanced - forward_orig() got an unexpected keyword argument 'attn_mask' :(

Ferdinand

Running ComfyUI on google colab. I am getting this error after drag & dropping the Flux Dev workflow. Anyone got an idea how to fix? Missing Node Types When loading the graph, the following node types were not found String Literal GetNode JoinStrings SetNode Anything Everywhere ImageBatchMulti ImpactImageBatchToImageList PulidFluxEvaClipLoader Fast Bypasser (rgthree) ImageResize+ PulidFluxInsightFaceLoader ToDetailerPipe Load Lora ExpressionEditor AnyLineArtPreprocessor_aux ModelPassThrough ApplyPulidFlux UltralyticsDetectorProvider PulidFluxModelLoader Int Literal FaceDetailerPipe UltimateSDUpscale Fast Groups Muter (rgthree) ToBasicPipe FromBasicPipe_v2

wwew

anyone having problems with load diffusion model node not identifying models ?

Michael

Hi there, is anyone running this on mac? I got all installed expect this on "DF_Get_latent_size". Any idea how I can install it on mac? How I can fix that? "Fix it" function is not working here.

Halil İbrahim

Are there any easy methods to set openpose body proportions to make character a bit taller?

Niklas Wagner

Solution found and working... :) Thank you!

Niklas Wagner

Thanks! Will try

Penny Taylor

恢复上一个版本即可

Penny Taylor

Error while deserializing header: MetadataIncompleteBuffer

Zom Boy

As soon as I ran this command, it started working no errors: git reset --hard 0f954f3 This is what they said in that github forum: "As a temporary solution, it's easier to open teminal inside ComfyUI folder and run this command to revert to the commit before: git reset --hard 0f954f3"

Zom Boy

I'm getting this exact issue. It seems we aren't the only ones: https://github.com/comfyanonymous/ComfyUI/issues/6077

Niklas Wagner

Trying to run it on Mac M3 ,... keep getting error "orward_orig() got an unexpected keyword argument 'attn_mask',.... tried many things to solve, any ideas?

Thomas Wahl

All workflows running, except Flux_ADV with PuLID ON brings up the error "Expected all tensors on same device, but found at least two devices, cuda:0 and cpu!" (SamplerCustomAdvanced) It's a common error with Pytorch, maybe an VRAM issue (4060 TI 16 GB)

Olivier C

I have this error message: "Warning: Missing Node Types When loading the graph, the following node types were not found: ControlNetLoaderAdvanced"

Olivier C

I have downolad here: https://huggingface.co/hanamizuki-ai/insightface-releases/tree/main . After hours of research, it works. But I have an another problem...

Christian Lösch

https://youtu.be/Uls_jXy9RuU?si=aZclaYHwSiqhAPbG Please check the comments here! I could solve it with bringing the antelopev2 folder directly under pulid/models

Prashanth G

everyone is waiting there only

Jordan Lamarche

Did you find any way to solve this issue?

Christian Lösch

Somebody has a idea to solve this? Thanx!

Christian Lösch

I get PulidInsightFaceLoader PulidEvaClipLoader ApplyPulid PulidModelLoader

Prashanth G

I’m also I’m trying many types and tutorials but unable to solve this

FuzzySilk

I have the same issue. I get:Missing Node Types PulidFluxEvaClipLoader PulidFluxInsightFaceLoader ApplyPulidFlux PulidFluxModelLoader And then when i try to add the nodes in the manager I get: (IMPORT FAILED) ComfyUI-PuLID-Flux-Enhanced and I choose fix or uninstall and then reinstall and it doesn't work Im stuck in a loop.

Prashanth G

Is it solved for you if yes can you please help me resolve

Prashanth G

Even im facing the same problem Let me know if find the solution

Prashanth G

Getting errors with PulID Unable to load these custom nodes tried multiple ways can any one help me please

Reuben Boxley

Trying to install the .whl file and getting the message below in CMD prompt ERROR: insightface-0.7.3-cp311-cp311-win_amd64.whl is not a supported wheel on this platform.

Reuben Boxley

I keep receiving the error "PulidFluxInsightFaceLoader"

Marco Belletz

I don't get why I keep obtaining some kinda unrealistic images, like rendering or illustrations. Tried both FLUX and SDXL wf and different samplers/checkpoints

Marco Belletz

SOLVED: I had just to update the KJNodes

Marco Belletz

I get a series of errors on all GetName, SetCN nodes that looks like this: Cannot execute because a node is missing the class_type property.: Node ID '#472'

Miftahudin

hey, can you do this on macos? i use mac mini m1 macos sequoia

WillieLonkya

Prob run this on 20/24gb. If you dont want to invest in biggies then prob have to use Runpod

Rigel Armanda Z.

same

INSECURE BOYZ

Is there a way I can run this on mac? I was able to get comfyui installed and running but am having trouble loading the rest?

Z

Unfortunately it crashed on my RTX 3080 with 16GB vram.. what is HW requirements to run those workflows ? Thx

Brandon J

I'm having this same issue :/ Is there a workaround for this? I have a Ti3070 with 8Gb vram.

Frederic Chauveau

I tried everything and I get all the time this error: SamplerCustomAdvanced Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! (when checking argument for argument mat2 in method wrapper_CUDA_mm) I have a Ti3070 super 16Gb vram

Ron Harris-White II

Run the following command in from the command prompt in your ComfyUI folder: git reset --hard cc9cf6d1bd957d764ad418258b61d7e08187573b

4n0nym0us.us3r

Can you explain how you did this? Downgrading I mean.

Marcin Onbike

I downgraded ComfyUI to v0.2.6 and workflow is working . Solution form GitHub doesn't work for me either.

Duke

I was able to eliminate all of the other bugs and errors that I see in these comments, but I keep getting another error: SamplerCustomAdvanced 'out_channels' Is that my system running out of VRam or something else?

marco mazzarella

It seems to have been resolved here although it doesn't work in my case https://github.com/zhangp365/ComfyUI-PuLID-Flux/blob/master/pulidflux.py

Mick Mahler

Unfortunately I can't fix this because I did not create the custom nodes. Someone created a fix that you can get on github https://github.com/balazik/ComfyUI-PuLID-Flux/issues/53

Frederic Chauveau

I am trying to connect to the Discord server using https://discord.com/channels/1128337217132171284 invite link but Discord answer: this invite is invald or expired

Frederic Chauveau

I have exactly the same problem. I tested everything on run_nvidia_gpu.bat but nothing is working for me! File "I:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\functional.py", line 2576, in layer_norm return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: expected scalar type Half but found BFloat16

Frederic Chauveau

It would be great to have a worflow for clothing. An idea for a future video ?

Wasay ali

After update this needs to be fixed and I'm still waiting.... "SamplerCustomAdvanced forward_orig() takes from 7 to 9 positional arguments but 10 were given"

Juha Pönkänen

same

Juha Pönkänen

same here

Cluster4000

Great workflow but does not work with latest ComfyUI update: SamplerCustomAdvanced forward_orig() takes from 7 to 9 positional arguments but 10 were given https://github.com/comfyanonymous/ComfyUI/issues/5608

M

Hey dude, Thanks so much for the workflow! I'm getting an error: SamplerCustomAdvanced forward_orig() takes from 7 to 9 positional arguments but 10 were given It seems to be happening in block 6 for backgrounds. And idea what is wrong here? I've not modified any of the nodes

Ekka

PulidFluxEvaClipLoader PulidFluxInsightFaceLoader ApplyPulidFlux PulidFluxModelLoader inside comfui and cannot install the 91 comfyanonymous/clip_l its giving me this on cmd ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager\glob\manager_server.py", line 267, in get_model_dir base_model = folder_paths.folder_names_and_paths["clip"][0][0] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'clip' can anyone help me?

UAknight

I'm getting this error when on the first step: LoraLoader.load_lora() missing 1 required positional argument: 'lora_name' - PLEASE HELP. nothing is mentioned about it in the video.

Vertical Heroes Kenya

Does this also work for products and/or clothing?

Pedro Anjos

discord server not working? I can't access the link

Egor Lukin-Diachkov

Just found, ComfyUI has additional arguments for running: --gpu-only Store and run everything (text encoders/CLIP models, etc... on the GPU). --highvram By default models will be unloaded to CPU memory after being used. This option keeps them in GPU memory. --normalvram Used to force normal vram use if lowvram gets automatically enabled. --lowvram Split the unet in parts to use less vram. --novram When lowvram isn't enough. --cpu To use the CPU for everything (slow).

Egor Lukin-Diachkov

Yep, my first person looks like Sims 2 character 😁

Egor Lukin-Diachkov

Yes

Egor Lukin-Diachkov

Unfortunately I could not run it with 8gb video card. But it works with Colab Pro L4 card.

Egor Lukin-Diachkov

Find in the previous post here

Levi

Where do I get the character pose sheet?

Coco

If i make this available on colab would anyone like it?

Gerhard Bultema

does anyone know what to do with this error? In samplercustom advanced: 'expected scalar type Half but found BFloat16'

Prairie Fawkes

It took a while to set up as all ComfyUI workflows seem to do, but I actually got it working! Put in a fairly low quality picture of a character from an indie book a friend wrote and managed to get decent character outputs. Once the LORA is trained they can make their own book graphics! Great work Mickmumpitz and thank you!

Ken McHardie

I'm a complete beginner, intrigued by what can be done with visual novels. Does this method require a powerful pc? I'm a little confused what hardware is needed. I have a Lenovo P1 Laptop-Workstation with 4gb video card and 32GB ram. Is this going to be enough, or should I get a larger desktop?

Tom Baron

When I run this I keep getting this error as it goes through the steps, does anyone know is this a floating point mismatch or maybe my GPU memory can't handle it? "RuntimeError: expected scalar type Half but found BFloat16"

Ebba Vei

Specs for AI and ML: Total VRAM 12282 MB, total RAM 32509 MB and NVIDIA GeForce RTX 4070

Ebba Vei

Yep, I trainned ~15 photos for a Lora, it takes ~8/9 hours to complete the 16 steps with 12gb, I am working with the PylarAI/Workstation https://github.com/PylarAI/workstation

red baron

Is it THAT slow for everybody??? It takes around 30 minutes to generate a first step, character sheet. Even with Flux Schnell version the same problem

Elena Pavlenko

Same here

Elena Pavlenko

Where i can find discord server

Coco

Lee Chars

Don't bother trying, guys. I pulled out a character sheet and trained them, but I can't get a consistent character. ^^ There is no technique to make a consistent character at this point. A reasonable compromise is needed. Just train the model face and use it, and dress it in the simplest clothes possible. For example, jeans and a white T-shirt. Look at the video, the character is wearing only simple clothes, right? That's the truth. Nothing magical happens.

Massimo Moro

Use CPU version of CumfyUI. (use comand: python3 main.py comand inside ComfyUI directory). But it's very very slow with Flux and LoRA!

Vitalii Mykhailyshyn

Hey, thanks for workflow! How to install PuLID when I’m using ComfyUI in ThinkDiffusion ?

The AI Surfer

Hey bro, been a fan for a long time.. I have a Macbook PRO m1 max... I notice you do NVIDIA setups.. any guides for MAC? Do I need to get a PC?

TheAI BriD

How can i sue it on MIMICPC

Ahmed Bilal

I am facing this issue : DualCLIPLoader module 'torch' has no attribute 'float8_e4m3fn'

Ahmed Bilal

I followed the steps in the tutorial and when asked to restart after installing missing nodded, it stuck and not even opening the comfy UI

TAJASEVEN

I installed and everything works besides character consistency, meaning every time I upload an image and queue, the generated image does not look like my uploaded image. How can I fix this? I even tried different seeds. Thanks!

AI Aniverse

Same

v33ts

For this error, I fixed it by: pip uninstall apex

Fred Leicht

Yes same here all I get is a thermal window that pops up and asks me to hit enter then the window closes and Poof nothing.

v33ts

Installed with nothing missing, but when executed I get PulidFluxEvaClipLoader - No module named 'fused_layer_norm_cuda' Anyone else? Linux, python 3.11, 4090ti

ReidloGames

doesn't work for me.

Mark K

I had this problem as well, you have to click on the .whl file on Github page - this brings you to the next page where you can download the raw version of the .whl file which is 851kb. This worked for me. Hope that helps.

Len

YES!

FCG

is the constant seed on the upscaler across all json on purpose? ending 3828 ty

Meta Intelligence

Try to click Queue Prompt again, it solved the problem on my side.

Luis Oliveira

Hi! Can anyone point me in the right direction? When I try to install insightface, I get this error: "ERROR: Wheel 'insightface' located at C:\Users\Utilizador\Downloads\insightface-0.7.3-cp311-cp311-win_amd64.whl is invalid." Thks in advance!

Lee Chars

I think there's something wrong with the pose sheet. The character's head is too big and the body is too short.

red baron

https://www.runcomfy.com/ update pls

Blueboy X

Here is the fix for the InsightFace assertion error: https://github.com/comfyanonymous/ComfyUI/issues/5133#issuecomment-2413422455

Kamil Banc

Any idea why i cant get this error sorted out? SetUnionControlNetType 'NoneType' object has no attribute 'copy'

Saucy Feet

Great post! Just one thing im on the latest version of ultimate SD upscale and it fails to import the nodes. Can you give a screenshots of all the ultimate SD nodes so I can copy the settings? I can re-add the node myself but having to guess the settings.