sgthale
patreon
🤖Local AI Hosting Tutorial - MyRobot
🕑 Added 2025-02-05 20:34:01 +0000 UTCThis is for January 2025 Build and above, older builds do NOT work with this.
🔠Local Text Generation
You can generate text responses (Offline and FREE) locally without internet connection using any OpenAI-Standard web API front end (Kobold, Oobabooga, LLMStudio, etc.). See Setting up an LLM (Text Generation) below.
📢Local Voice Generation
Undertale (Offline and FREE): repeating retro style phonetics (like in the game), custom voices are supported. Upload a small 0.5 second .wav file to repeat.
Patreon (Online and Members-Only): use the voice tokens from your membership to create sound for the local text you generate, custom voices are supported.
LLM Text to Speech: (Offline and FREE): use your own beefy GPU to craete sound, custom voices are supported. See Setting up an LLM (Voice Generation) below.
Setting up an LLM (Text Generation)
This is for advanced users and I recommend Kobold as it is personally the easiest one to setup. You need a good GPU to do this, I'm on a 2070 Super and text gen is near instant.
Pause Game -> AI Setup -> Text Generation -> Change dropdown to Self Host
Download https://github.com/LostRuins/koboldcpp/releases/tag/v1.83.1
Download this beginner AI Model (Scroll down to Provided files and download Q3_K_S)
Run koboldcpp_cu12.exe and select the model you downloaded in the Model section
Press the green Launch button
When ready it should automatically open your web browser to the default URL http://localhost:5001/
You can change that URL in settings, copy the URL into MyRobot, see below
Server Address - type in the URL of your frontend WebUI server then press ENTER. If it's red, it's an invalid path, green means it's valid.
Max Tokens - limit the number of tokens to process, more is smarter but slower.
Reduce prompt tokens - MyRobot injects additional messages into your chat history to help the robot visualize and understand things as the game is played, we recommend turning this ON unless you are running a strong and smart LLM model.
Voice Tokens - 1 letter = 1 token. So 150,000 tokens is about 5~6 hours of nonstop talking.
Setting up an LLM (Voice Generation)
This is for advanced users and I recommend Chatterbox as it is personally the easiest one to setup. You need a good GPU to do this, I'm on a 2070 Super and takes up 4/6 GB VRAM.
Pause Game -> AI Setup -> Text to Speech -> Change dropdown to Self Host
Download https://github.com/devnen/Chatterbox-TTS-Server/releases/tag/v1.0.0
Follow their installation instructions (Docker setup command is easiest)
Once you have it installed you should be able to test it in https://localhost:8004/
Change to Self Host in the TTS in the AI Setup Menu
Setup your server address and program the necessary parameters (image below)
Chatterbox API Docs requires the following parameters, the "text" parameter is set to "content" which is a special cyan colored value that replaces itself with the actual text to transcribe.
Custom voices with Chatterbox requires a simple reference .wav file to clone the voice. My Shinobu voice is attached below. Place it in your Chatterbox/voices folder
Enabling AI Self Hosting
You can always switch back to online generation by clicking Switch To Patreon/Steam.
Changing System Prompt
To change the system prompt for your local LLM to follow, connect your robot with the USB and head jack to the computer and in the Robot Editor, edit the system prompt in Software -> AI -> ChatAbility
Currently local LLM hosting does not support function calling so commands such as "fire your grapple hook at me" etc. only currently work with the online Patreon AI service. This is definitely being updated as soon as LLM models add robust support for function calling.
Any questions? Comment below:
Comments
Waytal
when i tested the local AI, i needed to build a small server in bettween to add the "model" parameter myself since you can't add it into the game. And it worked ! PS: I used mockoon to first see the request that the game was sending if you are curious
sgthale
You may have to reset the chat history of your robot by brianwashing it with the system prompt Robot Editor.
sgthale
Mmhmaybe if you can provide your model, hardware, and info on your setup i can attempt to recreate it. i don't actually know if OLLAMA is supported with the frontend API.
Dawn Reanimator
Ollama doesn't seem to work setting the base URI, even though it generally supports OpenAI style API prompting. Not sure if this is a me issue or not yet. But would be glad to see support considering Jetson AGX Orin is a powerhouse for LLMs and project digits will likely become relevant for this later as well, the specific nvidia docker containers being the pain point. Lmk if there's specific debug data I could gather.
foxter_exe
for some reason any llm used with kobold doesn't follow the in game prompt and will break itself within minutes for me. ill ask something or do something like poke it and the llm will start endlessly typing until the game says its lost connection. when i look at the terminal it shows that the llm repeated the same word or sentence over and over or repeated its prompt over and over. and unless i restart the kobold app and the game the llm will repeat the same answer to any question after
João Lucas
for some reason i cant make it stop speaking in *actions* even with system prompt
Felipe Moreira Ramos
There is now an Agent Leaderboard from Galileo Ai at HuggingFace. This might be useful for function calling. gemini-2.0-flash-001 is better than gpt-4o-2024-11-20, cheaper, and faster. Mistral Nemo is the last one but small and open source .
sgthale
Looking into this now
12807342180
For some reason when I use KoboldCPP as my local LLM the AI voices blast the fucking volume at 100% ear-rape. Also when I try to use oobabooga it says failed to connect to LLM.
sgthale
That is valuable information. Im gonna note that down for later. Thanks! Yes I should definitely check that out.
Kamini
So if I understand correctly, you couldn't find any local LLM models that can run do function calling? I did some limited testing with Mistral Nemo Instruct and it seems to understand both, function calling and roleplaying a character. Maybe add it as an option in the next build weather function calling is enabled or not?
