NerdyRodent
patreon
Flux Regional Prompting, RF-Edit & PAG
🕑 Added 2024-12-01 15:53:26 +0000 UTCVideo - https://youtu.be/6AHL0XZKbqg
Loads of new things in the Fluxtapoz repo now including
InstantX based Regional Prompting
RF-Edit
Perturbed Attention Guidance (PAG) and Smoothed Energy Guidance (SEG)
(Example RF-Edit before and after)
Links:
https://github.com/logtd/ComfyUI-Fluxtapoz
Overview
RF-Edit is another "unsampler" which allows you to prompt for edits - https://github.com/wangjiangshan0725/RF-Solver-Edit - turn photos to cartoons, give people different hair, change the background... so many options!
The Regional Prompting allows you to specify areas for your prompts, such as left and right. Based on the InstantX implementation - https://github.com/instantX-research/Regional-Prompting-FLUX - you can get some really fun things going :)
(Example head attached to a body using regional prompting & Redux)
PAG & SEG can be used in any workflow, and should help to bring out details.
As for the downsides, RF-Edit, PAG & SEG will take a whole lot longer to render, but the regional prompting isn't too bad and is loads of fun :)
Workflow Information
I've been testing lots of things, so a whole bunch of workflows are attached this time!
Flux RF-Edit Regular & RF-Edit Hyper are modified versions of their examples with the usual offloading to CPU and lovely, colourful nodes. Also sorted nicely into an order that makes sense (to me, at least). Hyper is simply an 8-step config test.
Flux Regional, redux text mask and redux PAG are once again coloured, sorted, offloaded versions.
Redux is using the nodes from https://github.com/kaibioinfo/ComfyUI_AdvancedRefluxControl as that version also supports masks.
For the basic masking I used the old https://github.com/BadCafeCode/masquerade-nodes-comfyui as it's simple, but there are loads of other automask options out there.
Flux PAG / SEG examples I tested with. Still need to find decent settings for SEG, I think!
Comments
Nerdy Rodent
Yes indeed it is :)
Danny
Hello I would like to do these half/half faces like in this video. Is it possible with another ratio than 1:1?
Nerdy Rodent
Which node is it that gives the error? That may give some tips. Someone on the github may also have a similar issue?
Asyouare
Thank you for the reply. No, it's a 4090. I always get the weird errors. Story of my life.
Nerdy Rodent
Putting CLIP into RAM just saves VRAM is all. "Not implemented for half" sounds like maybe a very old card, but just to make sure it's worth updating to the latest version of Comfy & fluxtapoz.
Asyouare
Uff..."upsample_nearest2d_channels_last" not implemented for 'Half'" Aaaargh. No idea what to make of this. Tried a few things but this keeps happening. Any ideas anyone? Also tried set clip device/force node to "cuda". Did not help.
Nerdy Rodent
All the latest updates should be working now, so I think that's one for the GitHub issues!
EruoAuri
Yeah I've updated to the newer version and still the same problem :/
Runebinder
That Configure Modified Flux node works for me, on Windows so didn't need to disable Xformers as I don't think it uses them.
Nerdy Rodent
You can ignore NF4 & GGUF as they're just cut-down versions. I stick with fp8 myself as it's the fastest with almost fp16 quality :)
Damian Konczak
Indeed. It works now. Thanks!
Jo Hannsen
Is it a self trained model? there are already too many flux models on my pc :D A lot different ones. Some are working in some workflows, then they don't in others. Could you do a table which explains the different models? Fp8/FP16/NF4/GGUF/hyper/turbo/arrgh when to use what with which.. what is the fastest, which is supported on which system.. basicly i am going through testing and waste hours on combinations.. Maybe you know a good conclusion on it already?
Runebinder
That explains it, thanks :)
Nerdy Rodent
Both Comfy & Fluxtapoz have had a few updates in recent hours. As a result, PAG/SEG now also needs the Configure Modified Flux node & xformers disabled also, like in the v1 workflow. The very latest updates in Comfy (2hrs ago) also seem to have broken a variety of things, so I expect we'll see a bunch of patches soon!
Runebinder
I've run a couple of gens on the PAG_SEG_Example workflow and can't see any difference between the first 3 outputs. I do on the 2 with the SamplerCustomNoiseDuo nodes though. Very odd.
Nerdy Rodent
They just updated the node a few hours ago for an issue related to the very latest release of Comfy, so worth updating again and seeing if it's fixed. If not, then over to those GitHub issues!
Damian Konczak
I did "Update ALL" today and the current version is ComfyUI: v0.3.6-4-g57e8bf6 (2024-12-02) Manager: V2.55 Sadly, I'm still getting the same error. Is 12GB of VRAM the issue?
Nerdy Rodent
Could be an older version of ComfyUI maybe? Should be at least ComfyUI: 2875[3fc6eb](2024-11-30)
Nerdy Rodent
Could be an older version of ComfyUI maybe? Should be at least ComfyUI: 2875[3fc6eb](2024-11-30)
Nerdy Rodent
You can click on the node to select your local flux model :)
Michael Weiss
I am not finding the fp8-hyper-turbo-detail-sameface-merge-v2_00001_.safetensors model file anywhere. Can you post a link, or use a different model in your workflow?
Damian Konczak
I'm getting this error with rfedit_regular when the process goes to the second sampler SamplerCustomAdvanced Expected key.size(1) == value.size(1) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.) Any ideas what's happening here?
EruoAuri
Couldnt make the Regional Prompting to work, even using the xformers flag. Here's the error: ModelSamplingFlux The default implementation of __deepcopy__() for non-wrapper subclasses only works for subclass types that implement new_empty() and for which that function returns another instance of the same subclass. You should either properly implement new_empty() for your subclass or override __deepcopy__() if it is intended behavior for new_empty() to return an instance of a different type.
