NerdyRodent
patreon
Flux.1 - Slower Upscaling - aka "Look Ma, no bands!"
🕑 Added 2024-09-20 11:09:22 +0000 UTCvideo - https://youtu.be/9zdcB7A_TJA
Well, this took a long time to test! During testing found a few things, which I split into 3 sections.
1. IMG2IMG "Trick"
Don't have the VRAM for a controlnet, or simply want a new way to do IMG2IMG? Then try ddim_uniform along with Hyper and a denoise of 1.0! Yes, I know it shouldn't work but it does, so now it's a thing :)
2. Non-Fast Upscaling
If you don't care how long upscaling takes, then there are a bunch of options... and I tested a lot of them. Long story short, try a hyper with a double dual sampler config and then an ultimate upscale at around 0.25 denoise (or just a plain model upscale if you don't have all day). It's my personal favourite right now, but that pyramid noise is tempting me too!
3. Nice Vision LLM
I've been "chatting" a lot with MiniCPM-V recently and found it's pretty good! Better than Llava and Florence for me, so maybe worth a try!
https://ollama.com/library/minicpm-v
ollama pull minicpm-v
Attachment info
Workflows used in the video are attached to this post. Here is a quick summary of what they do!
Flux_Dev_IMG2IMG_Hyper_v0.json = Simple Flux IMG2IMG + Hyper. A quick way to test the IMG2IMG trick!
Flux_Dev_IMG2IMG_Hyper_2SamplerHRF_LLM_v0.json = Flux IMG2IMG + Hyper + double dual sampler HRF upscale + optional LLM. This is the one for "everyday use".
Flux_Dev_IMG2IMG_Hyper_HugeCompare_v0.json = Flux IMG2IMG + Hyper + Loads of HRF options + optional LLM. Not for "everyday use", but for comparing. Nerds only! ;)
ComfyUI Manager Info
If "install missing nodes" fails somehow, these are the custom nodes packs used:
Impact Pack = ID 2
WAS Node Suite = ID 25
Extra Models for ComfyUI = ID 53
ComfyUI-GGUF = ID 54
UltimateSDUpscale = ID 71
Comfyroll Studio = ID 81
rgthree's ComfyUI Nodes = 175
ComfyUI Extra Samplers = ID 241
Use Everywhere (UE Nodes) = ID 247
🐰 MaraScott Nodes = 625
comfyui_LLM_party = ID 905
AuraSR-ComfyUI = ID 1104
Extra Model Info
Optional CLIP = https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14
Shakker Labs = https://huggingface.co/Shakker-Labs
Hyper-SD = https://huggingface.co/ByteDance/Hyper-SD
AuraSR = https://huggingface.co/fal/AuraSR-v2
Update - Custom Sampler Supreme Edition (testing)
rk4 seems to smooth things out, which may help with boxes / banding. The attached workflow gives above 2k, for example - Flux_Dev_IMG2IMG_Hyper_2SamplerSupremeHRF_LLM_v0alpha.json
Still testing this one, but it takes a while at 3K+ ;)
Update - "Look, ma! No bands!" - Sampler Supreme Anti-Band 2k+
Now out of alpha, this is version 1 of the "anti-band" samplers. rk4 looks to get rid of banding (please do check, my eyes are old and so is my monitor.
Example of a typical low-res (960x1280) image with banding:
High-res (2480x2480) Anti-band example:
The downside of rk4 was it made things too smooth :/ Never mind! I figured why not just add that detail afterwards :)
Example showing the workflow in action:
Now attached:
Flux_Dev_IMG2IMG_Hyper_LLM_AntiBandUpscale_v1.json
Comments
Nerdy Rodent
Flux does indeed tend towards the more "realistic" / "plastic" style! It's kinda why I like they way minicpm-v re-describes images as an image-to-text-to-image :)
Nerdy Rodent
My first guess would be out-of-date software, so I'd start with updating both ComfyUI and all custom nodes. Personally, I've added updating to my launch script so it gets the latest stuff each launch :)
GeoMage
I seem to have a weird issue...whenever I try to load any of these workflows I always get an error that says "Loop (272,99,340) with broadcast (397) - not submitting workflow". If I turn of the loop check setting in Anything Everywhere (which some have recommended) then it crashes Comfy completely and I need to restart the process. Any ideas what could be causing that? TIA
Stanislav Yurev
Thanks for the answer! Is it possible to convert 3d base or a fast 2d sketch into a realistic image with Flux? I did this with SDXL quite simply using promt, but I can't get similar result with Flux. Usually, Flux, on the contrary, enhances the 3D or 2D style, instead of making the image realistic. Am I use it wrong?
Nerdy Rodent
A lot depends on how much you want the image to change. Unlike previous SD models, flux has a more narrow range it seems to prefer when using a typical workflow. For the most part, the denoise value will need to be between 0.65 and 0.85 for any meaningful changes. Controlnets can help to keep things like specific edges, if they are important. Fully changing the image using an LLM is best at denoise 1, like in the "fast upscaling" post examples of changing images into Lego model versions - https://www.patreon.com/posts/flux-1-faster-111499834
Timothy Monaghan
I like that upscale that this seems to be capable of. Is this a LoRA or is this an upscale model. How do I download the model and where do I put it if it is not an upscale model? AuraSR is at https://github.com/fal-ai/aura-sr (https://huggingface.co/fal/AuraSR-v2)
Stanislav Yurev
Hi. I'm trying to use img2img with Flux but always get very simplified images that look like 3d or cartoons. Am I doing something wrong or is flux not that good at this? Example (I want 3D to be more realistic): https://imgur.com/a/BTWZIX2
Nerdy Rodent
AuraSR is at https://github.com/fal-ai/aura-sr (https://huggingface.co/fal/AuraSR-v2) and my test beardsley lora is just an example showing you can use whatever loras you have on top of HyperSD as well :) Simply switch that to none or use any of your own!
gloorbit *
I wonder where the model.safetensor comes from as well as the beardsley??? lora.
