1. An evaluation of the prototype mobile phone app Pugh a 3D cartoon character designed to help deaf children to speech read Dawn Shaw Marianne Patera Eleni Paparidou Robin Wolff University of Salford, U.K. School of Computing, Science & Engineering,

2. Inspiration

3. Visual cues are very important for communicating messages to the deaf Using a signer at the bottom of the screen may disengage viewers from being immersed in the story Some of the visual cues & messages may not be “translated” successfully This technique is not effectively engaging deaf children The Problem

4. “When people observe agents, they do not just care about what the is agent doing; they want to understand the relations between the agent’s actions at various points in time.” (Phoebe Sengers, 1999, p. 5)

5. Baldi Talking Tom Cat Accurate speaker Talking Ben the Dog Non-accurate speakers

6. Concept A 3D Cartoon Character Smartphone Application Believable Friendly Engaging The Ability to Sign

7. Design Vicon markers 44 phonemes Visemes Mouth structure

8. Video of Pugh

9. Preliminary Evaluation Online Test The test consisted of: 5 visible Consonants (p, b, m, f, v) 5 Vowels (a, e, i, o b) in VCV, CVC, CV, VC words 15 high frequency words (primary school level) & 3 sentences Participants Only a small number of participants (6) evaluated Pugh All adults 2 deaf and 4 speech therapists

10. Findings At the moment Pugh is not an accurate speaker. Speech-reading from the existing version is very difficult For the first 3 parts of the test (VCV, CVC, CV, VC) a sufficient proportion of the speechreading of the consonants was correct The vowel ‘a’ was perceived correctly by all the participants Pugh cannot articulate the vowel ‘o’ Inaccuracies were caused by the limitations in the technology to animate Pugh’s lips more accurately Its eyes, eyebrows and hands do not move properly which can be misleading (expressionless face) Limitations – small number of participants, test design

11. More programming needed Further testing Work closely with specialists Collaborate with animators working on children‘s programmes Is it possible for a cartoon character to speak accurately & sign? The Challenge The Plan The findings have reinforced a need for further development of the lip movement & the facial expressions.

12. Thank you! Dawn Shaw d.m.shaw@salford.ac.uk Pugh @