dvvr
patreon
Releaes 1.4.9 AI Powered Voice Chat!
🕑 Added 2023-10-01 13:09:55 +0000 UTC## AI Powered Voice Chat With Any Character
Now you can talk to your favourite characters face to face!
* You can choose to use ChatGPT or any LLM that you can run with OobaBooga WebUI.
* Completely local option available. If you want it totally free and private, you can run your own LLM and use it to power the chat.
* Built-in multilingo voice recognition.
* Built-in multilingo TTS engine. We've included a English model with over 900 different voices. Other language voices availble for download.
* Built-in native lipsync.
* Spacitial audio, you can tell where the speaker is without even looking.
* Easily customize character personality and description.
* You can save and replay chat messages.
* You can also use characters (in JSON format) from popular AI role play programs like TavernAI.
Current limitations
* Voice engine is Windows only. There are no voice option on Mac, Android, Quest and Pico for now.
* Running LLM along with DanceXR does require powerful GPU and large VARM. It is recommended to use remote setup.
Demo video: https://youtu.be/Eso5TNb67ZM
Full documentation: https://vrstormlab.com/dancexr/ai_chat
Useful links
* OpenAI https://platform.openai.com/
The famous ChatGPT. To use their model for AI chat, you'll need to register and get an API key from them. Then enter that key in the Chat Config in DanceXR. The service is not free but is cheap enough. Don't be confused with ChatGPT Plus though, they are different services. You'll still be charged by usage even when you have a ChatGPT plus subscription.
* OobaBooga WebUI: https://github.com/oobabooga/text-generation-webui
This is a program that allows you to run LLMs (Large Language Model) locally.
* After installing WebUI, you need to download a model to run. Here are 2 uncensored models that we recommend.
https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GPTQ (7b, easier to run)
https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GPTQ (13b, smarter)
* Runpod: https://www.runpod.io/
This is a service that allows you to rent a GPU and run AI models. If you don't want the hassle of installing your own WebUI, or don't have powerful hardware to run LLMs, this is one of the choices.
* Voice download https://rhasspy.github.io/piper-samples/
The TTS engine we are using is called Piper. The above link has all the voices currently available. If you are interested you can also train your own voice models.
## Model improvements
* Skirt and hair physics now have more options for you to finetune the physics effect.
* Refined collision control for boobs and softbody physics.
* You can now animate the transition effect of the Dressing System with Auto Update.
## Improved Pico support
* Color passthrough available
* Bug fixes
## Other fixes & improvements
* Improved FPS when there are many actors in the scene.
Comments
TaiqiTao
(DXR.apk)Unable to run on Android 14 version, cartoon contour lines can be achieved through model tracing. I am planning to purchase Snapdragon 8gen3 model, which should be able to run Vulkan
VR Storm Lab
I'm afraid you need to find those materials and change them to transparent. The program can only guess which materials should be transparent and it's not always accurate.
Contrabardus
Seems to have trouble with models that have "nose lines". Creates a big black strip over the nose. Eyebrows occasionally have the same problem, but not nearly as often. Haven't found a setting that fixes it outside of going into materials and disabling nose lines.
VR Storm Lab
This will be available in the Pico standalone build 1.4.9, which is to be released later this month.
friendly neighbor
I searched for hours for the Pico passthrough mode but couldn't find it. 😞 Where is it?
H-BOX
No way dude, you did this ??? Gonna try right away with ooba, that rocks
TaiqiTao
There is a program called "Digital Life" on Steam that is similar to the new feature in 1.4.9. That is the lightweight version, while yours is the billionaire version.Only Android version 1.4.9 is the true pioneer.For Android phones, floating windows or semi transparent modes can be used as smart assistants, which has practical significance.
ThatGuy
The AI voice chat sounds really interesting.
