NerdyRodent
patreon
Hunyuan Video in ComfyUI
🕑 Added 2024-12-06 17:04:40 +0000 UTCNOTE
The licence only allows certain territories to use the model, so I've removed the video and there are no outputs shown here. Next time I'll need to make sure I read the licence properly!
Mistake 1:
The license for the Tencent Hunyuan model only allows usage in certain territories, and I’ve now realized that the license excludes the United Kingdom and South Korea, which I initially missed. As a result, I’ve removed the video and the model files to comply with these restrictions. Going forward, I will ensure I carefully review licensing terms to avoid such oversights.
You can see the official Tencent license here:
https://huggingface.co/tencent/HunyuanVideo/blob/main/LICENSE
Take care as this is different to the licence file in the kijai repo, which has only the European Union excluded:
https://huggingface.co/Kijai/HunyuanVideo_comfy/blob/main/LICENSE
The Tencent license change commit was made later, so it’s easy to miss. This may also be present in other places.
Mistake 2:
Geo-restricted licensing is new to me, but as per the Tencent license terms under "5. Rules of Use", it appears that anyone sharing videos made with Hunyuan Video on platforms like Twitter, TikTok, Reddit, GitHub, or Hugging Face could also be violating the licensing agreement (unless they are properly restricting access by territory). I now realize I should have been more cautious in understanding the broader implications of this.
Please note, I am not a lawyer and this is not legal advice, but after reviewing the licensing terms, it’s clear I should have been more diligent in reading the fine print. This mistake is entirely my own, and I sincerely apologise for any confusion caused.
For those who are still able to comply with the terms, the information below is hopefully still okay. Please ensure that you are adhering to the correct territorial restrictions before proceeding.
ComfyUI
As seems to be the way nowadays, ComfyUI is the place to go if you want to run it at home on your own computer.
For a standard ComfyUI install on Linux, you can install everything as usual:
conda activate comfyui
cd custom_nodes/
git clone https://github.com/kijai/ComfyUI-HunyuanVideoWrapper
cd ComfyUI-HunyuanVideoWrapper
pip install -r requirements.txt
pip install -U flash_attn sageattention triton
For Microsoft Windows, it's all identical apart from the last line, as you may not be set up to compile flash & sage attention on that OS.
These attention modules are optional. You can create videos without them, but if you don't use sage attention then you'll be taking quite a chunky performance hit.
To get the last line to work on MS Windows, I'd suggest checking out this post by Purz - https://purz.notion.site/Get-Windows-Triton-working-for-Mochi-6a0c055e21c84cfba7f1dd628e624e97
It happens to be an old one for Mochi, but shows how he got sageattention installed on his Windows system.
For Mac OS I haven't seen much, but some GitHub comments appear to suggest it can be made to work.
For the "Comfy" attention, once again you'll need to start with xformers disabled if you have it installed. E.g.
python main.py --disable-xformers
Links
Workflows
Workflows attached are:
HunYuanVideo_t2v_example_v0 = Text to video example (rodent eating pasta)
HunYuanVideo_t2v_lowvram_example_v0 = Includes settings for low VRAM cards
HunYuanVideo_v2v_example_v0 = Example with both t2v & v2v together
HunYuanVideo_v2v_example_v1 = Example with just v2v
Note that the sampler does odd numbers of frames - 1,5,9,13,17,21, etc.
Comments
pazuzu
To anyone facing similar issues: not sure what I did, mostly reinstalling cuda and its buddies, and it seems to work now :)
pazuzu
Cool. Then I get to the next problem...the fp16 option is not (no longer?) supported by the model loader - indeed, when I look at the Python code, the only options are fp32 and bf16 ) in the HyVideoModelLoader class)...so I try putting everything on bf16 and get an error about "no execution plan to support the graph," and with fp32 I get an OOM (reasonable, though I do have 24GB) - any ideas?
Nerdy Rodent
Kijai's uses the ones linked - https://huggingface.co/Kijai/HunyuanVideo_comfy/tree/main - can't say about any others!
pazuzu
Where are the checkpoints from? on their hugging-face I'm only seeing .pt files with different names - did you just download/rename them?
Tribaloide
Looking forward to it soon
Nerdy Rodent
They haven't released image to video that I'm aware of?
Tribaloide
Hi. Please, image to video workflows?
Asyouare
For example, maybe you heard, ALL publishers of entertainment software have had to conform to(in this case) Germanies code of conduct when it comes to age classification of the product. Failure to do so means that it is illegal to sell the product there. Meaning any issued "EULA" from that point on and any of its terms are then null and void.
Asyouare
I am certain, changes to a license, after the fact, have limitations. If a license is issued, in China, for(for example) the EU area, and then, post-haste changed(or after some time), after the user/customer has started to use the product specified in the license, it does NOT mean that this change / these changes actually have legal merrit. The term "we reserve the right to change..." cannot change certain fundamentals to the product that was the sole base for the customer initial decision on the use of the product. I am 99% certain that in a court of law the customer would have far more levrage than companies would like to admit. But since this is a free product....don't know. Certainly, it is important that these license can be wholy unappplicable in a users area. Such is the price for making buisness with a global space. Many larger companies know this by now(and have, every now and then tried to push things through/ ignore things either way). But I would not fret too much about smaller enterprises laying down "the law". Should always check with a lawyer who specialises in this sort of thing. It is not THAT expensive. If it is important. In the end, these actors can only withdraw from regions or conform to THEIR legal space. Which is also why it is important to be/get involved, personally(or as a group) for user/customer rights. We are our very own lobbyists.
