tblankensmith
patreon
Custom ChatGPT-4-Turbo and Whisper(speech to text) Plugins for TouchDesigner
🕑 Added 2023-03-12 19:11:25 +0000 UTCHey! In this tutorial, we'll go over 2 new components I developed to run OpenAI's Whisper (speech-to-text) and ChatGPT within TouchDesigner. No need to install any dependencies.
Link to Open AI: https://platform.openai.com/signup
Update:
Note the component now works with the new GPT4 & GPT-4-Turbo Model as well.
Comments
Torin Blankensmith
Check out my eleven labs speech to text video. I setup a network that auto turns on and off whisper when someone talks
Torin Blankensmith
It’s using OpenAI’s servers to process, so it’s optimized already. No custom install needed
Fabrizio
Hi Torin! thank you for the plug-in! Are there any technical specifications for GPUs to use this plug-in in Windows?
Sina Catharina Oberwinster
Hey Torin, thank you for the tutorial. I'm currently trying to implement a live speech to text project with Whisper Ki. Is it possible to run the whole thing without the record on/off button? So that there is input all the time and is analyzed in the meantime, even if this is probably a bit delayed? If so, do you have any tips for me on how I can implement this in td?
Bo
I'm using a Mac, there is an error: "No such file or directory". Do you know how to solve it or show us clearly how to connect Arduino with Chat GPT APIs?
Eugene Pitts
This is such amazing work - the possibilities are staggering as you play with it. I am having the same issue as Daniel and Issac where the "record audio" causes the application to crash but the prerecorded audio works just fine. I am on Windows with the latest Stable build.
Isaac Hooper
Me too touch designer crashes every time I hit record would really appreciate a solution
Daniel
Just an update, with specified file path to a pre-recorded audio, the transcription works perfectly, the fatal error only happens when I hit record audio. Thank you.
Daniel
Hello Torin! Thank you for the plugin! I have an issue that when I hit the record voice input button on the whisper node, touch designer just returned a fatal error warning and says the application must closed. (I'm using Mac)
Torin Blankensmith
Just include the asyncIO tox anywhere in your TD project
FRANCOIS DEJARDIN - Polykrom
How do I use the async .tox file in order to make the gpt tox working ?
Djoeneuh
Really cool man, learning a bunch. Is there a way we could trigger an event based on the response ? As in, if we ask it to press 2 seperate buttons, for example for yes / no questions
Alon Hammer
And the script that can take the actual script and separate it from the text, it should be the same yeah?
Torin Blankensmith
Hey! That's strange. I specifically set this up so you don't need to install any libraries. Seems like an SSL issue with TD. What version of TD are you using?
Torin Blankensmith
Hey I just updated this to work with the new GPT-4-Turbo model as well. Yeah, I just gave it a couple of examples of what glsl code in TouchDesigner looks like. If you want to have it work with python in TD just give it examples of using python in TD
Alon Hammer
This is awesome!! I use chatgpt a lot to help with python scripts. Do you have any recommendations or direction for streamlineing that approach. I saw in the video you had a script that took the response and generated a glsl script. Is that easy to change for python?
Yulia Babanova
Hello Torin! Thank you for the plugin! I have an issue that it keeps returning a mistake whenever I try to integrate it into my project (I'm using Mac): HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/audio/transcriptions (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)'))) I've already installed new Python, openai libraries, ssl for python, updated my TD version but it keeps returning a mistake. Do you have any idea what could be the issue?
