1. Expressionbot: An Emotive Lifelike Robotic Face for Face-to- Face Communication Ali Mollahossenini, Gabriel Gairzer, Eric Borts, Stephen Conyers, Richard M. Voyles, Ronald Cole, and Mohammad H. Mahoor Presented by Jonathan Zhu

2. Introduction Robots are becoming more widespread Therapy, pets, entertainment Poor emotional and social interaction Face-to-Face communication Verbal and Nonverbal Price vs. Complexity ParoSimon

3. Current Problems Uncanny Valley Mona Lisa Effect

4. Mechatronic Robots Kismet Philips iCat Inflexible, unrealistic, “robot-like” Limited range of emotion

5. Android Robots Albert Hubo Zeno HRP-4C Realistic Complex, Expensive Uncanny Valley?

6. On Screen Avatar Grace Second Life Flexible and Cheap 2D Mona Lisa Effect

7. Light Projected Physical Avatar Realistic to Cartoonlike Flexible and Cheap Dome Lighthead Maskbot Furhat Bot Socibot

8. Design Neck System 3D printed face mask Modeled in Autodesk Maya Wig for aesthetics Portable Projector Animation C# facial animation Controllable eye gaze

9. Speech Bavieca speech recognizer Phonetic Transcription (IPA) Phonemes for all sounds Visually similar phonemes - visemes 20 classes /b/, /p/, /m/ - buy, pie, my Animation smoothing Need to force lip closure for certain phonemes

10. Emotion Lip Blending 6 basic expressions Anger, disgust, fear, joy, sadness, surprise Facial Action Coding System Joy: Cheek Raiser, Lip Corner Sadness: Inner Brow Raiser, Brow Lowerer, Lip Corner Depressor Conflicts surprise and phonemes /b/, etc Use weight table – upper/lower face Pre render certain conflicted groups

12. Projection Calibration Modeling done in 2D Projector and fish eye lens cause distortion Need to calibrate projection for 3D

13. Projection Calibration Distort Maya models instead of fixing at runtime Find best possible match (a) Create grid on both Maya model (b) and 3D face (c) Compute homography and affine transformation Match grid to triangle mesh on 3D (d) Use computer homography to distort original model

14. Projection Calibration

15. Experiments Assess the difference between 2D and 3D agents 1. Identifying Emotion 2. Speech Realism 3. Eye Gaze

16. Identifying Emotion Expression shown for 5 seconds Participants classify expression Joy, sadness, anger, surprise, disgust, fear or none 3D much better for identifying anger, moderately better at joy, disgust, and fear

17. Speech Realism Subjects presented with 2 speeches Margaret Thatcher speech Microsoft Anna synthetic speech Rank realism on scale of 0-5 Find the effectiveness of forced lip closure for certain phonemes 3D vs 2D agents had no significant difference Forced lip closure made speech significantly more realistic

18. Eye Gaze Find the effect of 3D light projected robot on the Mona Lisa Effect Position subjects around the head at various angles

19. Eye Gaze Shifting eye gaze only Screen Agent: 50% (24%) Physical Agent: 88% (13%) Shifting eye gaze + random head movement Screen Agent: 42% (18%) Physical Agent: 77% (15%)

20. Conclusion Only $1500 – relatively low cost Better emotion/expression recognition Smoother, more realistic speech Reduction of Mona Lisa effect