2. SUMMER SCHOOL OF SCIENCE VIŠNJAN, 2003. SPEECH PERCEPTION AND COCHLEAR IMPLANTS Damir Kovačić International School for Advanced Studies, Trieste & SUVAG Polyclinic, ZAGREB

3. Language ------ >>>>> Speech Perception One morning, when Gregor Samsa woke from troubled dreams, he found himself transformed in his bed into a horrible vermin. He lay on his armour-like back, and if he lifted his head a little he could see his brown belly, slightly domed and divided by arches into stiff sections. The bedding was hardly able to cover it and seemed ready to slide off any moment. His many legs, pitifully thin compared with the size of the rest of him, waved about helplessly as he looked. from Franz Kafka: Metamorphosis

4. THE ESSENCE OF LANGUAGE: **ARBITRARINESS OF THE SIGN **INFINITE USE OF FINITE MEDIA

5. Language processing entails many types of knowledge (grammatical, lexical, phonological etc.) and representations (sentences, words, phonemes, etc.) Linguistics, Psycholinguistics, Neuropsychology, Cognitive sciences, Information Theory, Neural Networks, etc…… How are these processes and units represented in the brain? Cognitive Neuroscience (Brain/Neuro Imaging)

6. To have a language: A speaker/listener has stored knowledge (but where?) of about approximately ?????? words of his/her native language. This knowledge includes: how the words sound the meaning of words the syntactic (grammatical) properties of words how to say/articulate words MENTAL LEXICON --  >>> HUGEEEEEE KNOWLEDGE!!! UNIQUE TO HUMANS!

7. From Haagort’s talk

8. SPEECH PERCEPTION: We will make a small experiment ;) - to find out speech signal - and its spectrum

9. Speech Perception: Concerns the earliest stage in the language processing when listener maps the time-varying acoustic signal on discrete linguistics (more abstract) representations in brain Phonemes (vowels and consonants) Syllables Morphemes Words Sentences

10. Word Segmentation Categorical Perception Perceptual Constancy - instantaneous adaptations to variations in speech signal SPEECH PERCEPTION – some basic phenomena Coarticulation Redundancy

11. The steady drip is worse than a drenching rain

12. NOISE AS SPEECH???

13. Brain at work: What happens when you hear a word? Most theories of lexical access agree in the following: Hearing a word elicits: 1)an automatic activation of many lexical entries that match or partially match with the input, which then undergo 2)a competition until one of them becomes selected

14. The mental lexicon sport figure sing door carry turf turtle gold turk turkey turn water turbo turquoise turnip turmoil

15. TURN sport figure sing door carry turf turtle gold turk turkey water turn turbo turquoise turnip turmoil

16. TURN sport figure sing door carry turf turtle gold turk turkey water turn turbo turquoise turnip turmoil

17. Ear Anatomy

18. Cochlea scalae vestibuli, media, tympani cochlear partition (Basilar membrane, organ of Corti) fluids: perilymph, endolymph Auditory neurons transforms sound into neural signals very sensitive organ, 35 mm long it looks like a snail’s house

19. Organ of Corti – the residence of hair cells

20. (Inner hair cells) (Outer Hair cells) IHC – aferent neurons (periphery  brain), 4000 OHC – eferent neurons (brain  periphery !!), 12000

23. Frequency-location map

24. “Travelling waves” in the cochlea 4000 Hz400 Hz

26. Stvaranje živčanih signala - percepcija zvuka “Traveling waves” u pužnici dovode do otklona bazilarne membrane, a time i tzv. svežnjeva dlačica unutrašnjih slušnih stanica mehaničko-električni kanali unutar dlačica slušnih stanica se otvaraju i zatvaraju dolazi do promjena receptorskih struja i potencijala što dovodi do otpuštanja neurotransmitera, čime se stvaraju akcioni potencijali u živčanim stanicama nastaje serija akcionih potencijala, a mozak razaznaje zvuk po različitim brzinama stvaranja tih živčanih signala - pattern of neural firing rates

27. COCHLEAR IMPLANTS

32. Cochlear implantation shows a huge variability in its follow-up success and usefulness Excellent and poor outcomes are attributable to the age of implantation: 1) Prelingually deafened children - excellent results 2) Prelingually deafened adults - poor, sometimes they demand the removal of the implant

33. THE RISE OF MACHINES: