1. Katholieke Universiteit Leuven - ESAT, BELGIUM The SPACE project: Speech Algorithms for Clinical and Educational Applications Hugo Van hamme SPACE symposium Antwerp

2. The SPACE project 2 2 September 2005 Outline partners What is it about ? Why does it make sense ? The educational component The clinical component Challenges – examples The technologies: foreground and background The first 6 months

3. The SPACE project 3 2 September 2005 Partners K.U.Leuven - ESAT: coordinator - speech recognition Prof. Hugo Van hamme R.U.Gent – ELIS: speech recognition Prof. Jean-Pierre Martens V.U.Brussel – ETRO: text-to-speech Prof. Werner Verhelst K.U.Leuven – ORTHO: disability, special needs education and child care Prof. Pol Ghesquière U.Antwerpen: communication disorders Prof. Marc De Bodt

4. The SPACE project 4 2 September 2005 In touch with the field … user group –Technology providers ScanSoft –Technology users Technology & Integratie Artec eXplio –Interest groups Stichting Integratie Gehandicapten (SIG) Modem this symposium

5. The SPACE project 5 2 September 2005 What ? Speech technology –automatic speech recognition (ASR) –speech synthesis (TTS) Clinical and educational –Speech therapy related. –Speech assessment Adapt technology –To suit requirements of the applications Demonstrate usefulness of technology –Automation of existing methods –New methods enabled by the technology Interdisciplinary

6. The SPACE project 6 2 September 2005 Why ? spoken interaction with the computer comes naturally –Unlike many other applications of ASR/TTS –Similar characteristics language learning pre-assessment in 2003 social relevance –role of universities –large group of beneficiaries persons with dyslexia reading skill development of all primary school pupils deaf, communication disorders

7. The SPACE project 7 2 September 2005 Why ? (2) other applications possible –language learning and language proficiency assessment –training of professional speakers –pronunciation training and stutter therapy –E-learning technology improvements applicable in other areas: –HMI with voice mode –entertainment

8. The SPACE project 8 2 September 2005 Some background Project sponsor: IWT –Instituut voor de aanmoediging van innovatie door Wetenschap & Technologie in Vlaanderen –SBO: Strategisch BasisOnderzoek 4 years: March 1, 2005 – February 28, 2009 28 person-years total effort This symposium is co-sponsored by the Nederlandse Taalunie

9. The SPACE project 9 2 September 2005 Domain of interest 1 Automated reading assessment and remedial practice reading tutor –replace human supervision in current diagnostic practice and in therapy –make assessment objective and repeatable –explore new strategies for diagnosis and remedy, enabled by speech technology use: –automate diagnosis of dyslexia => early detection –a program that helps you develop your reading skill –increase intensity (and effectiveness) of therapy –AVI reading tests in primary schools

10. The SPACE project 10 2 September 2005 Domain of interest 2 Clinical applications for speech assessment clinical practice –perceptual evaluation –subjective tests of articulation –interrater and intrarater disagreements use articulatory speech analysis compare to human judgement –reference database determine type and degree of error

11. The SPACE project 11 2 September 2005 The challenge - examples reading tutor: –mis-pronunciation –Immediate auditive feedback (cues) assessment: mis-articulation

12. The SPACE project 12 2 September 2005 Hestiations, unwanted speech Joep rijdt op zijn fiets door de straat. Het is een mooie gele fiets.Die heeft hij voor zijn verjaardag gekregen. Er zit een grote glimmende bel op.

13. The SPACE project 13 2 September 2005 The technology background: –large vocabulary speech recognizer (ESAT) –voice assessment, pronunciation modelling (ELIS) –text-to-speech and voice modification (ETRO) requirements –accurate assessment of utterance acceptance/rejection Fine-grained analysis/feedback –speech representations that give articulatory insight –modelling of imperfect speech: mis-articulations mis-pronunciations at phoneme, word or sentence level –feedback and guidance through TTS

14. The SPACE project 14 2 September 2005 Approaches: acoustics optimize acoustic models for children model the disfluencies –non-phonemes articulatory analysis of speech –voicing, high/low, lip rounding, … –estimated from wave form –relevant for articulation assessment accurate phonetic classification –phonetic hypotheses generated in phoneme lattice –phoneme-specific features and tests added

15. The SPACE project 15 2 September 2005 Approaches: miscues lexical mispronunciation models –exploit prior knowledge on reading mistakes orthography frequency: rare words substituted by common semantics: read-by-guessing strategy –data driven: at word level or by transformation rules sentence level misreading models –hestitations, restarts …

16. The SPACE project 16 2 September 2005 Approaches: TTS TTS for –providing pronunciation examples –providing reading cues –synchronised reading special reading mode speech synthesis –spelling mode (letter/phoneme) –syllable mode (isolated/lengthened) extremely slow speech special stress patterns

17. The SPACE project 17 2 September 2005 Where are we ? articulatory speech analysis data collection: –dyslalia, dysarthria, hearing loss –reading exercises: content, tools TTS: public domain software analysis reading tutor prototype –children’s acoustic model –track reading progress –model for word skips and restarts –model for unintended speech –model for lexical errors: swap of letters, phoneme substitution …

18. The SPACE project 18 2 September 2005 conclusion the SPACE project –has challenging objectives –interdisciplinary –will deepen insights in new speech modelling approaches –will develop prototypes in both application areas –has mainly a social relevance, also economic spin-off activities possible improvement in accuracy and robustness of ASR additional speaking modes and synchronisation in TTS