Furkan Gözükara
patreon
Magic Animate Automatic Installer and Video to DensePose Auto Converter For Windows And RunPod
🕑 Added 2023-12-05 01:39:58 +0000 UTCJoin discord and tell me your discord username to get a special rank : SECourses Discord
Fully automatic installers for Windows using pip and VENV - not even Conda necessary,
This tutorial video made for this video. Watch it to learn how to install Python, Visual C++ tools properly > https://youtu.be/-NjNy7afOQ0
1 January 2024 Update
- Full tutorial video published both for Windows & RunPod
- https://youtu.be/HeXknItbMM8
- CodeFormer Gradio APP improved
- Now auto deletes generated frames after video process completed
- Just uncheck the option to keep them
- Also now can handle space etc character having input video files
- Download newest magic_animate_v6.zip
9 December 2023 Update
- CodeFormer auto installer and a Gradio web app added to the scripts
- Read instructions
- Significantly improves Magic Animate made videos face quality
- Can also upscale video into 4 times with Real-ESRGAN upscaler
8 December 2023 Update
- RunPod installer and instructions added both for DensePose maker and Magic Animate
- Now supports custom .safetensors file loading
- Default model changed to Realistic_Vision_V6.0_NV_B1.safetensors
- You just need to edit magic-animate\configs\prompts\animation.yaml and give model file path
- I didn't notice any difference between SD 1.5 and Realistic Vision V6
Pre Requirements
- Make sure that you have installed Python 3.10.11 (3.10.x) : https://www.python.org/downloads/release/python-31011/
- Installed CUDA Toolkit 11.8 : https://developer.nvidia.com/cuda-11-8-0-download-archive
- Installed Git : https://git-scm.com/
- Installed FFMPEG and added to the environment path (2:56) : https://youtu.be/OI1LEN-SgLM
- https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl.zip
- Installed Microsoft Visual Studio C++ Tools 2022 : https://visualstudio.microsoft.com/downloads/
- https://aka.ms/vs/17/release/vs_BuildTools.exe
- Make sure that your install directories DO NOT HAVE ANY SPACE letters
- All above are necessary unfortunately
Current Status
- Currently only 512x512 resolution is working. I am trying different resolution solution
- You can generate more than 20 seconds videos with 24 GB. If you get out of VRAM error reduce the duration of the input video.
- You can try maximum duration according to your GPU
- Uncheck --debug
5 December 2023 Huge Update
- Please download latest installer zip file from attachments
- I have managed to make a very optimized DensePose video generator
- When you download and extract files you will get 3 folders
- compose_densepose
- pre_shared_densepose
- compose_magic_animate
compose_dense_pose
- Inside compose_densepose first run install_detectron. bat
- This file will install all libraries etc into a new VENV for to be able to generate DensePose video from given any raw video
- Making this work took me like 15 hours. Make sure that you have followed all Pre Requirements
- Then to generate a DensePose of any given raw video first edit make_dense_video. py file
- Edit video_path according to your video path
- Edit final_output_path according to your desire
- You can also edit frames_folder and densepose_output_folder (optional)
- Then to generate DensePose video run generate_dense_video. bat file
- final_output_path will be DensePose video of given raw video fully automatically with best possible DensePose model
- I have written a very special script and edited apply_net. py for batch processing. So this process is blazing fast
- Before generating another DensePose video I suggest you to manually delete dense_frames and
compose_magic_animate
- Inside compose_magic_animate first run install magic animate. bat
- This will generate a new VENV and install everything inside there
- It will also download all necessary models automatically for you (huge task)
- Making this work took me like 12 hours. Make sure that you have followed all Pre Requirements
- After that start Gradio Web APP with run gradio app. bat
- It will locally start web APP and open in browser
- Then upload your reference image and DensePose video and click generate
- You can watch the status from the CMD
- The longer the DensePose video it will take longer and it will use more VRAM
- The generated videos will be saved into magic-animate\demo\outputs folder
pre_shared_densepose
- Inside pre_shared_densepose you will find already prepared DensePose videos that you can directly use with reference images
- I also added 2 reference image for my DensePose videos
code_former_improve_face
- Inside code_former_improve_face you will find
- RunPod / Linux Install Instructions For Magic Animate Gradio install code former. bat and run code former app. bat
- To install CodeFormer into its unique venv run install code former. bat file
- After that start the myself coded CodeFormer Gradio Web UI with run code former app. bat
- With this gradio you can improve faces in given video file significantly and can also upscale video resolution with Real-ESRGAN full automatically
- The result videos will be automatically saved inside CodeFormer/results folder
RunPod / Linux Install Instructions For Magic Animate Gradio
https://runpod.io?ref=1aka98lq
- If you are a Linux user just change paths from /workspace to your local computer or cloud service path
- Select runpod/pytorch (e.g. 2.0.1) template
- Edit pod, set volume disk 50 GB
- The more VRAM machine you select the longer duration video you can generate
- Upload install. sh and downloader. py file from compose_magic_animate folder (inside magic_animate. zip) into your workspace folder
- Run below command for install
- apt update --yes
- cd /workspace
- export HF_HOME="/workspace"
- chmod +x install.sh
- ./install.sh
After installation has been complete run with below command
- cd /workspace/magic-animate
- export HF_HOME="/workspace"
- source venv/bin/activate
- python -m demo.gradio_animate --share
- After that you will get gradio live share
- Sadly proxy connect still not working due to Gradio version
- The generated video files will be saved inside workspace/magic-animate/demo/outputs by default
RunPod / Linux Install Instructions DensePose Maker
Upload all files inside compose_dense_pose (inside magic_animate. zip) folder except densepose folder into workspace/dense_pose folder. You need to compose dense_pose folder in workspace manually. Enter inside it and upload there as shown in video.
- Run below command for install
- apt update --yes
- apt-get install wget unzip --yes
- cd /workspace/dense_pose
- export HF_HOME="/workspace"
- chmod +x install_detectron.sh
- ./install_detectron.sh
After installation
Upload your video file into anywhere
Edit make_dense_video. py and set video file and folder paths
Then run make_dense_video. py file as below
apt update line 1 time necessary after each restart
ffmpeg line 1 time necessary after each restart
- apt update --yes
- apt install ffmpeg --yes
- cd /workspace/detectron2
- export HF_HOME="/workspace"
- source venv/bin/activate
- cd ..
- cd dense_pose
- python make_dense_video.py
Then download generated DensePose video and use from your magic-animate gradio
RunPod / Linux Install Instructions CodeFormer Improve Face
Upload all files inside code_former_improve_face (inside magic_animate. zip) folder into workspace/code_former folder. You need to generate code_former folder manually.
- Run below command for install
- cd /workspace/code_former
- export HF_HOME="/workspace"
- chmod +x install_code_former_Runpod.sh
- ./install_code_former_Runpod.sh
After install run CodeFormer Gradio Web APP as below
update and ffmpeg lines 1 time necessary each time after Pod restart
- apt update --yes
- apt install ffmpeg --yes
- cd /workspace/code_former/CodeFormer
- export HF_HOME="/workspace"
- source venv/bin/activate
- python code_former_app.py --share
To make it work on Linux system just change folder paths
The result videos will be automatically saved inside workspace/CodeFormer/results folder
Extras
Example video : https://www.pexels.com/video/a-man-walking-inside-a-production-plant-3209176/
To auto crop with center
- ffmpeg -i input.mp4 -vf "crop=min(iw\,ih):min(iw\,ih), scale=512:512, fps=25" -c:a copy output.mp4
Extract only first second of video to test quicker
- ffmpeg -i input.mp4 -t 1 -c copy input_1sec.mp4
Hopefully a full tutorial coming soon
Install inside e.g. G:\magic_animate - do not have any spaces
Comments
Furkan Gözükara
hi give more info. what is your python version? did you install c++ tools? your folder path and video file name?
hypily123
Hi, I have the following error when trying to import apply_net Aborted (core dumped)
Furkan Gözükara
Hello. please download magic_animate_v6.zip file which is in the attachments of the post
kack bob
where is history magic-animate-x.zip
Furkan Gözükara
hi do you see frames in the folders? looks like it wasnt able to determine your input video dimensions. i would reencode input video with ffmpeg and provide that way
Huey
Hi, I am receiving this error at the end of the codeformer completing the processing -- Video Saving... Traceback (most recent call last): File "C:\Users\clien\Documents\MagicVideo\code_former_improve_face\CodeFormer\inference_codeformer.py", line 264, in height, width = video_frames[0].shape[:2] IndexError: list index out of range
Furkan Gözükara
I agree with that. I will keep my eye on hopefully future stuff. Thank you for support
PTMarks
Thanks for the really great video and explanation! All of the installs worked flawlessly! Unfortunately Magic Animate is so inferior in quality, no matter how much up-scaling you throw at it, it is a throw-away app. Maybe when they develop an SDXL version it might be worth looking at again. It is good to have the Code Former setup for up-scaling other videos, so thank you for that!
Furkan Gözükara
ye i didn't bother with multi gpu scripts. sadly don't know even if they are working or not. but since libraries are installed they should work if original authors made them work
Mark Burazin
Question, seems like it's only using one GPU (magic-animate), and since you modified the start scripts that they are not the standard ones I don't see a _dist variation to run multi gpu? Any quick pointers?
Furkan Gözükara
it could be due to missing c++ tools. i am editing a video right now. follow that video and install Python and c++ tools like that. hopefully will be published tomorrow on channel
Mark Burazin
Hi, good job, it works I can generate videos from samples and pictures I need, except I can't get compose_dense_pose to install correctly. It's failing on 'No module named torch'. Any suggestions? I installed the build tools and in them Desktop development and Universal Windows Platform build tools is checked. but when running install_detectron.bat it fails.
Furkan Gözükara
i doubt so. now supporting SD 1.5 custom models but i didnt see any difference yet
Arcon Septim
Can we use a SDXL checkpoint?
Furkan Gözükara
wow nice i definitely must make an auto installer and a video
楠 陈
Can you develop a one-click installation of DEMOFUSION? https://github.com/PRIS-CV/DemoFusion
Furkan Gözükara
probably detectron weren't installed properly. did you install c++ tools? did you see any error while installing?
Furkan Gözükara
yes i ignore all working great
Furkan Gözükara
can you show me screenshot of your folder?
San Milano
Hello! I'm getting this error while trying to generate densepose: ModuleNotFoundError: No module named 'detectron2.config
Edmund Kudzayi
resolved: a pip install after process ended managed to install
Edmund Kudzayi
I experienced an error during the installation of 'nvidia-pyindex', which causes the build process to fail. Specifically, the error occurs while building the wheel for nvidia-pyindex. The process is interrupted by a 'subprocess.CalledProcessError', indicating that a subprocess command returned a non-zero exit status. This error is associated with the failure to locate the 'pip' module, essential for the installation. The implication is that even after script completion, the virtual environment may not successfully run the application, as the necessary Nvidia package index is not properly installed. Any ideas on how to resolve?
Jamie Hudson
Should i ignore all this? C:\Users\Jamie\Desktop\MagicAnimate\compose_magic_animate\magic-animate\magicanimate\pipelines\pipeline_animation.py:43: FutureWarning: Importing `DiffusionPipeline` or `ImagePipelineOutput` from diffusers.pipeline_utils is deprecated. Please import from diffusers.pipelines.pipeline_utils instead. from diffusers.pipeline_utils import DiffusionPipeline Initializing MagicAnimate Pipeline... loaded temporal unet's pretrained weights from magicanimate/pretrained_models/stable-diffusion-v1-5\unet ... ### missing keys: 560; ### unexpected keys: 0; ### Temporal Module Parameters: 417.1376 M The config attributes {'addition_embed_type': None, 'addition_embed_type_num_heads': 64, 'addition_time_embed_dim': None, 'conditioning_channels': 3, 'encoder_hid_dim': None, 'encoder_hid_dim_type': None, 'global_pool_conditions': False, 'num_attention_heads': None, 'transformer_layers_per_block': 1} were passed to ControlNetModel, but are not expected and will be ignored. Please verify your config.json configuration file. It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8. It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8. It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8. It is recommended to provide `attention_head_dim` when calling `get_down_block`. Defaulting `attention_head_dim` to 8. A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton' C:\Users\Jamie\Desktop\MagicAnimate\compose_magic_animate\magic-animate\magicanimate\pipelines\pipeline_animation.py:103: FutureWarning: The configuration file of this scheduler: DDIMScheduler { "_class_name": "DDIMScheduler", "_diffusers_version": "0.21.4", "beta_end": 0.012, "beta_schedule": "linear", "beta_start": 0.00085, "clip_sample": true, "clip_sample_range": 1.0, "dynamic_thresholding_ratio": 0.995, "num_train_timesteps": 1000, "prediction_type": "epsilon", "rescale_betas_zero_snr": false, "sample_max_value": 1.0, "set_alpha_to_one": true, "steps_offset": 0, "thresholding": false, "timestep_spacing": "leading", "trained_betas": null } is outdated. `steps_offset` should be set to 1 instead of 0. Please make sure to update the config accordingly as leaving `steps_offset` might led to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` file deprecate("steps_offset!=1", "1.0.0", deprecation_message, standard_warn=False) C:\Users\Jamie\Desktop\MagicAnimate\compose_magic_animate\magic-animate\magicanimate\pipelines\pipeline_animation.py:116: FutureWarning: The configuration file of this scheduler: DDIMScheduler { "_class_name": "DDIMScheduler", "_diffusers_version": "0.21.4", "beta_end": 0.012, "beta_schedule": "linear", "beta_start": 0.00085, "clip_sample": true, "clip_sample_range": 1.0, "dynamic_thresholding_ratio": 0.995, "num_train_timesteps": 1000, "prediction_type": "epsilon", "rescale_betas_zero_snr": false, "sample_max_value": 1.0, "set_alpha_to_one": true, "steps_offset": 1, "thresholding": false, "timestep_spacing": "leading", "trained_betas": null } has not set the configuration `clip_sample`. `clip_sample` should be set to False in the configuration file. Please make sure to update the config accordingly as not setting `clip_sample` in the config might lead to incorrect results in future versions. If you have downloaded this checkpoint from the Hugging Face Hub, it would be very nice if you could open a Pull request for the `scheduler/scheduler_config.json` file deprecate("clip_sample not set", "1.0.0", deprecation_message, standard_warn=False) Initialization Done!
Hermokrates
Whenever I try to run "generate_dense_video" I get this error: Traceback (most recent call last): File "I:\AI\MagicAnimate\magic_animate\compose_dense_pose\make_dense_video.py", line 5, in import apply_net # Import the modified apply_net script File "I:\AI\MagicAnimate\magic_animate\compose_dense_pose\apply_net.py", line 13, in from detectron2.config import CfgNode, get_cfg ModuleNotFoundError: No module named 'detectron2.config'
Marcel Kuiper
or a God shines through him
Furkan Gözükara
yes i just made it. please download latest file. now automatic dense pose maker is included. spent like 12 hours straight to make it work
alex
Thanks !! can you please do a one click installer for "detectron2" too it's allow to generate the animation video for MagicAnimate https://github.com/facebookresearch/detectron2 There's DensePose too https://github.com/facebookresearch/DensePose
Furkan Gözükara
There is only 1 God. I am just a mere mortal. I am about to complete DensePose generator script as well.
Aldo Jones
You are a GOD!!!
Furkan Gözükara
let me update it for runpod installer. sadly i couldn't still make DensePose generator
Jorge Reverte Sevillano
How can i run this on runpod? 🤔
Furkan Gözükara
updated zip file. please use updated one
Furkan Gözükara
that is my today work hopefully
Furkan Gözükara
you have 0 chance to make this work on mac. it depends on so many nvidia libraries. i plan to make script for RunPod hopefully soon
Furkan Gözükara
it works with like 10 GB. not sure 8 GB VRAM
polaraju p
i have error Traceback (most recent call last): File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict output = await app.get_blocks().process_api( File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\blocks.py", line 1434, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\blocks.py", line 1335, in postprocess_data prediction_value = block.postprocess(prediction_value) File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\components\video.py", line 281, in postprocess processed_files = (self._format_video(y), None) File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\components\video.py", line 355, in _format_video video = self.make_temp_copy_if_needed(video) File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\components\base.py", line 226, in make_temp_copy_if_needed temp_dir = self.hash_file(file_path) File "D:\AI\magic_animate_installer\magic-animate\venv\lib\site-packages\gradio\components\base.py", line 190, in hash_file with open(file_path, "rb") as f: FileNotFoundError: [Errno 2] No such file or directory: 'D:\\AI\\magic_animate_installer\\magic-animate\\demo\\demo\\outputs\\2023-12-05T15-09-08.mp4'
Al H
In downloader.py, put a comma at the end of the section: os.path.join("magic-animate", "magicanimate", "pretrained_models", "stable-diffusion-v1-5","unet"): [ "https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/unet/config.json", "https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/unet/diffusion_pytorch_model.bin" ]
Erosi
How can we make pre prepared motion sequences?
Carsten Li
Could you please also add a MacOS script?
Alex Chu
downloading model files File "F:\magic_animate_installer\downloader.py", line 47 os.path.join("magic-animate", "magicanimate", "pretrained_models", "stable-diffusion-v1-5","unet"): [ ^ SyntaxError: invalid syntax. Perhaps you forgot a comma? Virtual environment made and installed properly
Rashid Omar
thanks for the quickupdate, I have rtx 3070 ti with 8 vram, any chance of running this ?
