Furkan Gözükara
patreon
1-Click Installers for Most Powerful Vision Model CogVLM V2 and Batch Captioning Images, Install on Windows, RunPod and Massed Compute
🕑 Added 2025-04-26 16:33:00 +0000 UTCPatreon exclusive posts index to find our scripts easily, Patreon scripts updates history to see which updates arrived to which scripts and amazing Patreon special generative scripts list that you can use in any of your task.
Join discord to get help, chat, discuss and also tell me your discord username to get your special rank : SECourses Discord
Please also Star, Watch and Fork our Stable Diffusion & Generative AI GitHub repository and join our Reddit subreddit and follow me on LinkedIn (my real profile)
=======
Info
Latest installers : CogVLM2_v10.zip
Official repository of the APP : https://github.com/THUDM/CogVLM2
This app is purely developed by me
CogVLM 2 is GPT4 level for vision tasks and it is open source and free
Check above screenshots to see how to use it
Currently the APP works amazing with 4-bit quantization very fast
I am searching to lower VRAM usage even further with like adding CPU-Offloading and other stuff if possible
All above examples generated on RTX 3090 locally on Windows with 4-bit quantization
FP16 works on 48 GB GPUs
1 May 2025 Update V10
When batch processing, it was not properly handling orientation data - like vertical image shown horizontally - this bug fixed
When batch processing, multiple GPU was not working, this bug fixed
26 April 2025 Update
All installers updated to support RTX 5000 series with newest libraries - official Torch 2.7 and CUDA 12.8
I have used my pre-compiled libraries so they should be super fast
xFormers (compiled by me for you), Flash Attention (compiled by me for you), Sage Attention (compiled by me for you on Linux), Triton
The installers will support older GPUs as well with maximum speed hopefully
If you see it is too slow, very likely that your VRAM is not sufficient, run in 4-bit mode
Also on windows now it will download models into models folder, so you can delete previously downloaded models from your Hugging Face cache folder
Windows Requirements
Python 3.10, FFmpeg, CUDA 12.4, cuDNN 8.9 and 9.4 or above, C++ tools, MSVC and Git
If it doesn't work make sure to below tutorial and install everything exactly as shown in this below tutorial
How To Install and Use:
Windows:
First watch above requirements tutorial and follow every step
Extract zip archive into a drive like c:/cogvlm2
Longer directory path causes issues
Double click and install with Windows_Install.bat and that is it
Then run with Windows_Start_App.bat
It will download around 37 GB model files into your Hugging Face cache folder when you first time load
Massed Compute (Recommend Cloud) :
Please register via this link : https://vm.massedcompute.com/signup?linkId=lp_034338&sourceId=secourses&tenantId=massed-compute
Use our coupon SECourses
Our coupon works on all GPUS now
You can see full list here : https://www.patreon.com/posts/huge-news-secourses-coupon-massed-compute-details-126671823
I recommend RTX A6000 ADA as price performance but read above post
Then select our image SECourses from Creator dropdown
Then follow Massed_Compute_Instructions_READ.txt
Same as my any other Massed Compute installer script
Example tutorial for learn how to install and use Massed Compute
(Starts at 12:58) : https://youtu.be/KW-MHmoNcqo?si=G1WbG-Qw4ujWvOtG&t=778
RunPod (Cloud):
Please register via this link : https://runpod.io?ref=1aka98lq
Then follow Runpod_Instructions_READ.txt
Same as my any other RunPod installer script
Use the template written in Runpod_Instructions_READ.txt file
Example tutorial for learn how to install and use RunPod
(starts at 22:03) : https://youtu.be/KW-MHmoNcqo?si=QN8X8Sjn13ZYu-EU&t=1323
Comments
Furkan Gözükara
sadly i dont know . i recommend compare with joy caption as well
Furkan Gözükara
yes this model is huge. i am about to publish this baby soon hopefully : https://pasteboard.co/qhrauJ4tQhLD.png
Christoph Behrmann
what prompts or prompt phrasing to use for best results if I want to focus -for example - on rims when it is about car images, not to exclude everything else, but focusing on rims, and have things like car color, etc also, but besides
Christoph Behrmann
is 140s per image using 4bit with rtx4070 fast or slow? :) usually with other image captioning tool and nearly same similar text output I have 2s/i and with CLI .py script using pre-loading images to RAM and a working flash attn on Windows within conda env I have 3i/s ... vram is at 11.3/12gb and GPU is at 90-95% without spikes, just to let you know that your "CogVLM2_v10" seems to work brilliant even if there is no batch option to set manually... and no dataloader workers option to set manually like you have added at joycaption... ##joycaption is much much faster, but I am trying to make the most out of everything, that is why I am also testing with CogVLM2 v10...
Hockey
For anyone that wonders how much VRAM 8-bit precision is using the answer is 21.6 GB. How do the 4-bit, 8-bit and full precision differ from each other in results?
Furkan Gözükara
Yes this is heavier and more powerful (better)
Yannis
Hey, is it better than JoyCaption ?
Furkan Gözükara
this error looks like gradio / network related so ignore it
meteor 1942
Got the following string of errors ... and then ... it continued and actually produced a valid description of the attached image. Wondering how to get rid of the error (selected 1,1, on both options at launch): ERROR: Exception in ASGI application Traceback (most recent call last): File "D:\CogVLM2\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 403, in run_asgi result = await app( # type: ignore[func-returns-value] File "D:\CogVLM2\venv\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 60, in __call__ return await self.app(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\fastapi\applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\applications.py", line 112, in __call__ await self.middleware_stack(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\middleware\errors.py", line 187, in __call__ raise exc File "D:\CogVLM2\venv\lib\site-packages\starlette\middleware\errors.py", line 165, in __call__ await self.app(scope, receive, _send) File "D:\CogVLM2\venv\lib\site-packages\gradio\route_utils.py", line 789, in __call__ await self.app(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\middleware\exceptions.py", line 62, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app raise exc File "D:\CogVLM2\venv\lib\site-packages\starlette\_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "D:\CogVLM2\venv\lib\site-packages\starlette\routing.py", line 715, in __call__ await self.middleware_stack(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\routing.py", line 735, in app await route.handle(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\routing.py", line 288, in handle await self.app(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\routing.py", line 76, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app raise exc File "D:\CogVLM2\venv\lib\site-packages\starlette\_exception_handler.py", line 42, in wrapped_app await app(scope, receive, sender) File "D:\CogVLM2\venv\lib\site-packages\starlette\routing.py", line 74, in app await response(scope, receive, send) File "D:\CogVLM2\venv\lib\site-packages\starlette\responses.py", line 357, in __call__ await self._handle_simple(send, send_header_only) File "D:\CogVLM2\venv\lib\site-packages\starlette\responses.py", line 386, in _handle_simple await send({"type": "http.response.body", "body": chunk, "more_body": more_body}) File "D:\CogVLM2\venv\lib\site-packages\starlette\_exception_handler.py", line 39, in sender await send(message) File "D:\CogVLM2\venv\lib\site-packages\starlette\_exception_handler.py", line 39, in sender await send(message) File "D:\CogVLM2\venv\lib\site-packages\starlette\middleware\errors.py", line 162, in _send await send(message) File "D:\CogVLM2\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 507, in send output = self.conn.send(event=h11.EndOfMessage()) File "D:\CogVLM2\venv\lib\site-packages\h11\_connection.py", line 512, in send data_list = self.send_with_data_passthrough(event) File "D:\CogVLM2\venv\lib\site-packages\h11\_connection.py", line 545, in send_with_data_passthrough writer(event, data_list.append) File "D:\CogVLM2\venv\lib\site-packages\h11\_writers.py", line 67, in __call__ self.send_eom(event.headers, write) File "D:\CogVLM2\venv\lib\site-packages\h11\_writers.py", line 96, in send_eom raise LocalProtocolError("Too little data for declared Content-Length") h11._util.LocalProtocolError: Too little data for declared Content-Length Starting image processing on GPU 0...
Furkan Gözükara
well so something is still wrong. i would recommend watch video again and apply every step. https://youtu.be/DrhUHnYfwC0 also check out the video's patreon post it shows updated system path and user path. moreover if you upgrade gold tier i can connect your pc and fix issue
T. Scott Thompson
I already have Microsoft Visual Code. I tried the installer you pointed to for MSVC, but it just updated VSC. I never got the option to select what C/etc. packages to install like you show on the video where you have check-box options on the right. I verified in file explorer that MSVC is installed (C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\bin\Hostx64\x64) and I rebooted. Still, your app gives me a warning that "Failed to find MSVC".
Furkan Gözükara
upload image again and make sure it is properly uploaded.
michele carlone
It gives me this error: Please select an option: 1. Start As 4-bit Precision 2. Start As 8-bit Precision 3. Start As 16-bit Precision Enter your choice (1-4): 1 Please select an option: 1. Start Without Gradio Share 2. Start With Gradio Share Enter your choice (1-2): 1 * Running on local URL: http://127.0.0.1:7860 To create a public link, set `share=True` in `launch()`. Starting image processing on GPU 0... Loading model for GPU 0 Loading checkpoint shards: 100%|██████████████████████████ ███████████████████████████████| 8/8 [00:13<00:00, 1.69s/it] Error processing image: expected str, bytes or os.PathLike object, not NoneType Starting image processing on GPU 0... Error processing image: expected str, bytes or os.PathLike object, not NoneType
