Furkan Gözükara
patreon
1 Click Auto RunPod Installer For Rerender A Video - 1 Click Video To Anime
🕑 Added 2023-10-15 22:56:10 +0000 UTCJoin discord and tell me your discord username to get a special rank : SECourses Discord
5 February 2024 Update:
- A tutorial video published : https://youtu.be/cVf9Qf_pKks
- Please also upvote this Reddit Thread I appreciate that
- Go to your RunPod : https://runpod.io?ref=1aka98lq
- Select template RunPod Pytorch 2.1.1
- Customize deployment and set volume disk size to 50
- After pod started connect to jupyter lab interface
- Default downloaded models improved
- Default selected source model set as MeinaMix V11
- Download attached installer.sh and upload into the workspace folder
- Run below command (copy paste into new terminal).
- chmod +x installer.sh
- ./installer.sh
The above command will clone ReRender repo and install all of the requirements fully automatically for you. It will even download compiled ebsynth so you will not wait its compile.
It will also auto download modified to run on RunPod Rerender A Video web UI as well.
Moreover installer sh file will download the below model files with their proper sd_model_cfg py file so you can select them from web UI dropdown and use directly.
model_dict = {
'Realistic Vision V6' : 'models/Realistic_Vision_V6.safetensors',
'Flat 2D Animerge': 'models/Flat_2D_Animerge.safetensors',
'MeinaMix V11': 'models/MeinaMix_V11.safetensors'
}
If you also want to add more models you can download them with wget and and add them to sd_model_cfg py file. Look at the installer . sh file to see how to download from CivitAI or from Hugging Face.
After installation you need to run below code to start ReRender web UI.
It will give you a Gradio share link use it. You can turn off your Pod start again and directly use again with below command (copy paste into new terminal).
Don't forget chmod +x of ebsynth
- export HF_HOME="/workspace"
- chmod +x /workspace/Rerender_A_Video/deps/ebsynth/bin/ebsynth
- cd /workspace
- cd Rerender_A_Video
- source venv/bin/activate
- python webUI.py
Select video and click render first frame. Since it will upload video until you get correct first frame try again and again and watch the command line interface messages.
Moreover make your video FPS like 24 25 30. Do not make it like 23.976fps
You can use ffmpeg to re-encode FPS like below code
- ffmpeg -i input.mp4 -vf "fps=24" -c:v libx264 -crf 7 -c:a copy larry_24fps.mp4
Make your video minimum 2 seconds. You can test example attached 2_sec_example.mp4 file
To see all process edit video_blend py file and set OPEN_EBSYNTH_LOG = True
The result will be inside /workspace/Rerender_A_Video/result/your_uploaded_video_file_name folder
The file name will be blend.mp4
The progress will be slow and it won't show messages
You can check result folder changes to follow
First I suggest you to test 2 second clip then do full clip with same settings
Watch your VRAM usage too
Control type: Canny works better than Hed option
Bigger resolution will significantly improve quality like 768px
You can also play with Canny threshold to get more accurate output
Finally it will not copy the sound of the original video
- Put output render video and initial original video into the same folder and run below command.
- Don't forget to change file names. Also you need to have installed (add system path) or downloaded ffmpeg into the same folder
- ffmpeg -i blend.mp4 -i source_video.mp4 -c:v copy -c:a copy -map 0:v:0 -map 1:a:0 output.mp4
Comments
Furkan Gözükara
thanks a lot for support
Stephen Yip
golden
Furkan Gözükara
can you try with attached 2_sec_example.mp4 . i am also doing a fresh install right now to verify still working or not
James Smith
getting this error when trying to run first key frame Loaded model config from [./deps/ControlNet/models/cldm_v15.yaml] Loaded state_dict from [./models/control_sd15_canny.pth] /workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/safetensors/torch.py:98: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() with safe_open(filename, framework="pt", device=device) as f: Traceback (most recent call last): File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/queueing.py", line 388, in call_prediction output = await route_utils.call_process_api( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 219, in call_process_api output = await app.get_blocks().process_api( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1437, in process_api result = await self.call_function( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1109, in call_function prediction = await anyio.to_thread.run_sync( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run result = context.run(func, *args) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/utils.py", line 650, in wrapper response = f(*args, **kwargs) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/workspace/Rerender_A_Video/webUI.py", line 291, in process1 frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) cv2.error: OpenCV(4.8.1) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
Furkan Gözükara
hello. this is because your input video file name. can you try this. rename your input video file to my_video1.mp4 and upload and try again and let me know?
Prosze Spamuj
Hi! When rendering all I get this error: [...] [ WARN:0@1012.481] global loadsave.cpp:248 findDecoder imread_('result/rerender_before2/video/0055.png'): can't open/read file: check file path/integrity Traceback (most recent call last): File "/workspace/Rerender_A_Video/video_blend.py", line 318, in main(args) File "/workspace/Rerender_A_Video/video_blend.py", line 267, in main process_seq(video_sequence, i, blend_histogram, blend_gradient) File "/workspace/Rerender_A_Video/video_blend.py", line 198, in process_seq dist1s.append(load_error(bin_a, img_shape)) File "/workspace/Rerender_A_Video/video_blend.py", line 159, in load_error with open(bin_path, 'rb') as fp: FileNotFoundError: [Errno 2] No such file or directory: 'result/rerender_before2/out_51/0052.bin' Traceback (most recent call last): File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/queueing.py", line 388, in call_prediction output = await route_utils.call_process_api( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 219, in call_process_api output = await app.get_blocks().process_api( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1440, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1341, in postprocess_data prediction_value = block.postprocess(prediction_value) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/video.py", line 281, in postprocess processed_files = (self._format_video(y), None) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/video.py", line 355, in _format_video video = self.make_temp_copy_if_needed(video) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/base.py", line 226, in make_temp_copy_if_needed temp_dir = self.hash_file(file_path) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/base.py", line 190, in hash_file with open(file_path, "rb") as f: FileNotFoundError: [Errno 2] No such file or directory: 'result/rerender_before2/blend.mp4' but it does not occur always, in most cases. I was using your tutorial with A6000 GPU on runpod. Do you know how to resolve it?
Furkan Gözükara
did you start it like below exactly? chmod +x /workspace/Rerender_A_Video/deps/ebsynth/bin/ebsynth cd /workspace cd Rerender_A_Video source venv/bin/activate python webUI.py
Rasika Singal
Get the following error: video_blend.py: error: unrecognized arguments: 1 1/blend.mp4 Traceback (most recent call last): File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/queueing.py", line 388, in call_prediction output = await route_utils.call_process_api( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/route_utils.py", line 219, in call_process_api output = await app.get_blocks().process_api( File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1440, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1341, in postprocess_data prediction_value = block.postprocess(prediction_value) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/video.py", line 281, in postprocess processed_files = (self._format_video(y), None) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/video.py", line 355, in _format_video video = self.make_temp_copy_if_needed(video) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/base.py", line 226, in make_temp_copy_if_needed temp_dir = self.hash_file(file_path) File "/workspace/Rerender_A_Video/venv/lib/python3.10/site-packages/gradio/components/base.py", line 190, in hash_file with open(file_path, "rb") as f: FileNotFoundError: [Errno 2] No such file or directory: 'result/istockphoto-1278398498-640_adpp_is 1/blend.mp4'
Furkan Gözükara
i know the error. either your folder or your video file name has spaces. make your install folder like my_rerender, video file name like my_video.mp4
Filip A. M
At first everything worked, but now that I followed the tutorial exactly, this error appears when I'm about to reach the blend phase. Keyframes are ok and ready, but the blend doesn't work. I tried several runpods: " FileNotFoundError: [Errno 2] No such file or directory"
Brock Johnson
Disregard last comment pls. I was on broken runpod server.
Brock Johnson
I am getting Error on run 1st keyframe?
Furkan Gözükara
i think it would work with free kaggle. hopefully will make a tutorial stay tuned
DAVID PEREZ
Great Furkan, Could you please make a tutorial or a notebook for this to work with Kaggle free tier please, that would be awesome!
Furkan Gözükara
yes this is a full code version that clones from official repo
Gus Basurto
Thank you very much, Is this the full code version?
