Project Simone
Speech Graphics created Simone, a virtual English teacher with a special cutaway view revealing the world’s first high-fidelity animation of human speech. Simone appears in Saundz, an educational software product teaching American English pronunciation. Read more.
Client
Saundz
Our Task
Saundz creates software to teach correct pronunciation. A main component of their product is visual demonstration of how English speech sounds are articulated. For this to be effective they required highly realistic and accurate computer animation of speech. Moreover, they wanted not just an external view of the lips, but also an internal view of the mouth, tongue, nasal cavity and throat not visible from the outside. Since Speech Graphics leads the world in modeling and visualization of speech, we were contracted to provide this crucial technology.
Production Stages
- 3D modeling
We produced a complete head-and-bust model, with hair, textures, and specially designed blendshapes to deform the face realistically during speech. We also created a working model of the internal speech organs: mouth, tongue, palate, nasal cavity, throat. For this modeling we used a variety of data, including facial scans, MRI, and cadaver studies. The internal model required careful design choices about level of anatomical detail and how to display 3D structures in a 2D plane. The model of the tongue, an extremely deformable, volume-preserving muscle mass, required complex rigging based on deformation analysis.
- Rendering and compositing pipeline
Once the models were produced, we designed the textures and rendering pipeline for both external and internal views. The face was rendered with subsurface scattering in Mental Ray. For the internal model, we borrowed techniques from medical animation to strike the right balance between graphic detail and good taste. We also designed smoke effects to indicate nasalization and aspiration. Compositing was done in After Effects.
- Animation production
The client recorded audio of words and phrases from a female speaker of American English for their pronunciation instruction curriculum. We animated all of this audio automatically on the 3D models using our in-house audio-driven software. Both the face model and internal model of speech organs were driven in the same manner by Speech Graphics’ muscle-dynamic simulation of speech production. Even the smoke effects were controlled as if they were muscles in this system.
