Furkan Gözükara
patreon
Testing Stable Diffusion Inference Performance with Latest NVIDIA Driver including TensorRT ONNX
🕑 Added 2024-05-29 14:35:45 +0000 UTC🚀 UNLOCK INSANE SPEED BOOSTS with NVIDIA’s Latest Driver Update or not? 🚀 Are you ready to turbocharge your AI performance? Watch me compare the brand-new NVIDIA 555 driver against the older 552 driver on an RTX 3090 TI for #StableDiffusion. Discover how TensorRT and ONNX models can skyrocket your speed! Don’t miss out on these game-changing results!
1-Click fresh Automatic1111 SD Web UI Installer Script with TensorRT and more ⤵️
https://www.patreon.com/posts/86307255
Tutorial video : https://youtu.be/TNR2HZRw74E
Testing Stable Diffusion Inference Performance with Latest NVIDIA Driver including TensorRT ONNX
0:00 Introduction to the NVIDIA newest driver update performance boost claims
0:25 What I am going to test and compare in this video
1:11 How to install latest version of Automatic1111 Web UI
1:40 The very best sampler of Automatic1111 for Stable Diffusion image generation / inference
1:57 Automatic1111 SD Web UI default installation versions
2:12 RTX 3090 TI image generation / inference speed for SDXL model with default Automatic1111 SD Web UI installation
2:22 How to see your NVIDIA driver version and many more info with nvitop library
2:40 Default installation speed for NVIDIA 551.23 driver
2:53 How to update Automatic1111 SD Web UI to the latest Torch and xFormers
3:05 Which CPU and RAM used to conduct these speed tests CPU-Z results
3:54 nvitop status while generating an image with Stable Diffusion XL — SDLX on Automatic1111 Web UI
4:10 The new generation speed after updating Torch (2.3.0) and xFormers (0.0.26) to the latest version
4:20 How to install TensorRT extension on Automatic1111 SD Web UI
5:28 How to generate a TensorRT ONNX model for huge speed up during image generation / inference
6:39 How to enable SD Unet selection to be able to use TensorRT generated model
7:13 TensorRT pros and cons
7:38 TensorRT image generation / inference speed results
8:09 How to download and install the latest NVIDIA driver properly and cleanly on Windows
9:03 Repeating all the testing again on the newest NVIDIA driver (555.85)
10:06 Comparison of other optimizations such as SDP attention or doggettx
10:35 Conclusion of the tutorial
NVIDIA’s Latest Driver: Does It Really Deliver?
In this video, we dive deep into NVIDIA’s newest driver update, comparing the performance of driver versions 552 and 555 on an RTX 3090 TI running Windows 10. We’ll explore the claims of speed improvements, particularly with #ONNX runtime and TensorRT integration, using the popular Automatic1111 Web UI.
What You’ll Learn:
Driver Comparison: Direct performance comparison between NVIDIA drivers 552 and 555.
Setup and Installation: Step-by-step guide on setting up a fresh #Automatic1111 Web UI installation, including the latest versions of Torch and xFormers.
ONNX and TensorRT Models: Detailed testing of default and TensorRT-generated models to measure speed differences.
Hardware Specifications: Insights into the hardware used for testing, including CPU and memory configurations.
Testing Procedure:
Initial Setup:
Fresh installation using a custom installer script which includes necessary models and styles.
Initial speed test with default settings and configurations.
Driver 552 Performance:
Speed testing on driver 552 with default models and configurations.
Detailed performance metrics and image generation speed analysis.
Upgrading to Latest Torch and xFormers:
Updating to the latest versions of Torch (2.3.0) and xFormers (0.0.26).
Performance testing after updates and comparison with initial setup.
TensorRT Installation and Testing:
Installing TensorRT extension and generating TensorRT models.
Overcoming common installation errors and optimizations.
Speed testing with TensorRT models and analysis of performance improvements.
Upgrading to Driver 555:
Step-by-step guide on downloading and installing NVIDIA driver 555.
Performance comparison between driver 552 and 555.
Analyzing the impact on speed and efficiency.
Results and Conclusions:
Performance Metrics: Detailed analysis of speed improvements (or lack thereof) with the newest NVIDIA driver.
TensorRT Benefits: How TensorRT models significantly boost performance.
Driver Update Impact: Understanding the real-world impact of updating to the latest NVIDIA driver.
Comments
Furkan Gözükara
Ye drivers suck. They need to fix it
Ec Jep
Here is some test data for you. I have experienced this before and this time I kept my old installation and created a new one with the updates. The old installation (see parameters below) runs much faster than the latest... (IDK Why) Current install v1.8.0 python3.10.11 torch 2.2.0CU121 xform 0.0.24 gradio 3.41.2 speed w/o tensorRT 6.84it/s speed w tensorRT 10.9it/s New install V1.9.3 python3.10.11 torch 2.3.0CU121 xform 0.0.26.post1 gradio 3.41.2 speed w/o tensorRT 6.72it/s speed w tensorRT 7.53it/s Win 10, RTX4090, Nvid studio driver 555.99 other comment: with same image parameters the new install does not replicate the images (similar but not exact) and has less quality with all same models and image parameters (IDKW). For me RealVisXL 4.0 and current install above generates stunning images and quality. I believe the largest increase in speed following your instructions above was to update nvid driver from 546 to 555. I have never been above 10it/s yet...
Furkan Gözükara
https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT/issues/309
Furkan Gözükara
Start with basic. see if basic works. 1024x1024 regular short prompt. then go from there. TensorRT is sadly not very flexible. for example last time only resolutions divisible to 64 was working. LoRAs were working last time i tried.
Snohoe
On top of that loras refuse to convert to work with tensorrt.
Snohoe
At first I thought it might mean the maximum tokens but I generated a RTmodel with a Max prompt token count of 750 and it still gives that error.
