Droid Productions
patreon
Talk to me like on of your French chatbots.
🕑 Added 2023-02-11 05:48:18 +0000 UTCOkay, this was a fun little rabbit hole to pursue. This started it's life with me asking MollyBot to write a sexy script.
MollyBot is a Discord chatbot based on the Davinci-3 dataset (better known as ChatGPT), though without any of the weird objections to sex, or mayhem. She's been given a rough set of information about how Molly in Love of Magic talks, and she knows that she's a blond sorceress valley-girl. (Don't bother asking her about the ending of the game, she doesn't know any details like that).
Say hi to the nice people on Discord, Molly.
Next up, I needed a voice for this (and no, I'm not going to narrate it myself). Time to turn to another AI, https://beta.elevenlabs.io/.
Now, my initial idea was to run this through another lipsynch AI (maybe Wav2lip), but I couldn't get results I liked.
I flirted briefly with using the AniLip auto-lipsync for Daz3D, but at 2,300 frames it'd occupy my PC for days on end. Time to cheat.
This classic Disney-style way of doing Lip Synch goes back to Steamboat Willey in 1928, and provide a standardized way of designing expressions for lipsynch. The *old* way would be to manually draw them, but ain't nobody got time for that in 2023.
I'm assembling the layers in Spine2D, like this:
Then using an automated phoneme extractor (that listens to the sounds and extract the mouth-shapes). In this case I used the free Rhubarb Lip Synch system, which helpfully added the right keys back into the Spine file as keyframes (at this point I started regretting my life choices in making this 2,400 frames long).
I exported the whole thing out as a .mov file (so I could keep transparency information), and imported it into After Effects. Both the background (KD Bedroom) and the foreground (Molly) were turned into 3D objects, and I added a little camera motion to make it feel less static.
So... I had fun with this. It's not going ingame, mostly because I'm not really sure if that's Molly's true voice. But I learned new things, and hopefully you guys enjoyed the video, and the behind the scenes look at how it was put together.
If you have any questions, let me know... now I'm back to writing Act XIV+.
Comments
MattinVegas
Molly AI now thats scary
Pax
This was a great tech demo and explanation with the video showing off how it all comes together. Thanks for sharing this!
Bruce Mellish
Shame she doesn't know anything useful, though .... :D
