cebstudios
patreon
MocapImport 0.71 - Audio2face to Blender - Tutorial
🕑 Added 2021-06-06 19:38:45 +0000 UTCIntructions to import to audio2face. export from audio2face and import in blender
Important: At the end of this post, there are some info on how to use audio2face with blendshapes (a lighter alternative)
1- leave only the head of the model
2- use the option "Export to Nvidia Audio2face"
3- open in Nvidia Audio2face
4- in character Transfer tab, click on "+ Male Tamplate"
5- move the template heads to the side of the imported model
6- on Driver a2f mesh, select "mark"
7- Taget mesh = the mesh you imported
8- OpenMouth A2F mesh, select "mark_openmouth"
9- On mesh fitting, click ADD POINT
10- select a points on the green head, and the imported head. (get points near the ear, on the eyes, mouth, nose and chin)
11- When finished, click "Done Adding"
12- Press "Begin Mesh Fitting"
13- Press "Begin Post Wrap"
14- Click on Mark model (The grey head model template) -!! This is very important
15- Go to the tab Audio2Face
16- Click "+ A2F Pipeline" - It will ask if you want to attach it to Mark. You can confirm it.
Now the sounds that you play in the audio player will drive the movements from the template Marks and your imported model.
To export do these steps:
On "Data Export"
1- choose a copy of the mesh that you imported. It is the on with the "result" at the end of the name
2- choose where to save the exported file
3- choose a name, of left with the default one
4- choose the FPS
5- click "Export as Maya Cache"
Back in blender.
1- make sure that you have the model that you export is selected
2- click on "Import Nvidia Audio2Face"
3- select the file that was exported form Audio2Face
Now the head model has the movements that was exported from Audio2face
Have fun
For new version of audio2face, you have the possibility to get the blendshapes, and that is much more light weight than the previous way.
Here is a toturial from nvidia, on how to create the files to use as blendshapes
https://www.youtube.com/watch?v=Fh0qpSA4TOw
Here is a link to a video on how to add the blendshapes to create shepekeys in blender
https://www.youtube.com/watch?v=vmGOna364WI
And attached in this post, there is a blend file with the script used on the video above
Comments
Sorry, I'm soo late with other activities. Probably would take months for me to be able to do another tutorial on that subject (because I must remember things first too 😊) So, the best thing I can suggest is using it with modifier, the blendshape option wolud be much better, but I just made it work once and it was using the CC3 a2f integration, so for custom characters, i still didnt find how to do, and cant do it now, sorry.
Thank you for your answer. If you produce a tutorial video that can use lip sync data lightly, please let me know.
Yeah... I noticed that too. I see 2 solutions, or you use it with the modifier option. Or, you would have to use with the new audio 2face blendshape option. There is no automated way that i've done, but you can try to created the blendshapes and export the json. I have a script somewhere that can drive the movement, i did it a while ago, i have to search it. But, for now, go with the modifier option. (its the other option on mocap_import
It's hard to work because there are too many Shape keys. Is there a solution?
