NerdyRodent
patreon
Image-2-Video with Cosmos in ComfyUI!
🕑 Added 2025-01-19 15:51:01 +0000 UTCVideo - https://youtu.be/fjUdzKEIypk
ComfyUI now supports some of the Nvidia Cosmos models, which means image to video fun at home, for free, on your own computer :)
Ok, so it's not as good as the pay-for websites, but it is still a lot of fun with plenty of potential in this first release.
It is fairly slow and needs 12+ GB VRAM to run, but it does seem to have a better license than other models.
Both Image-2-Video and Text-2-Video and possible, with each of the 7B models weighing in at around 14GB each.
Links
The blog has lots of useful and interesting info, with Comfy examples having the example workflows. Nvidia's video on Cosmos is well worth a look, and if you need ollama then that's there too!
Blog: https://blog.comfy.org/p/comfyui-now-supports-nvidia-cosmos
Example: https://comfyanonymous.github.io/ComfyUI_examples/cosmos/
Omniverse training data: https://www.youtube.com/watch?v=9Uch931cDx8
Ollama: https://ollama.com/
Workflow Information
The attached workflows are numbered in order of appearance in the video. There are 3 different versions -
Standard Image/Text to video - set the switch to pick which latent to use and select a model. Video2World model for Image-2-Video, and Text2World for Text-2-Video.
LLM option - As standard, but with an llm option. You'll need ollama running with your choice of vision LLM. You don't have to use the image though, and could disconnect that to use a smaller language model, for example.
Multi - As standard, but used to create a looping video. Takes 3 input images, and saves outputs individually (A, B and C) to allowing independent scene modifications. Stitch the pieces together in your favourite video editor, add some sound and enjoy!
Comments
Nerdy Rodent
Weird. Not something I've heard of! My first guess would be a node conflict with something. It takes me 7 mins for 704x704 so I image their 4090 time is accurate :)
Tribaloide
In my custom_nodes there is ComfyUI-Universal-Styler but not ComfyUI_NAI-mod
Tribaloide
Thanks for the video tutorial and workflows, you are always at the forefront of ComfyUI and creating super creative and interesting workflows!, you are amazing! Please two questions: 1} Error. No naistyles.csv found. Put your naistyles.csv in the custom_nodes/ComfyUI_NAI-mod/CSV directory of ComfyUI. Then press "Refresh". Your current root directory is: F:\ComfyUI_windows_portable_nvidia\ComfyUI_windows_portable\ComfyUI. Where can i download naistyles.csv? 2} Is it true that with an RTX 4090 graphics card it takes ten minutes per queve to generate a 1280x704 121 frame video (how many seconds are 121 frames?),we will have to wait for future updates to make it last less time.?
Ducks and Owls
Just had to update Comfy
Ducks and Owls
Where do we get these nodes, sorry? :) ``` EmptyCosmosLatentVideo CosmosImageToVideoLatent ```
