A Lecture about… Phonetic Acquisition Veronica Weiner May, 2006

What is phonetic acquisition? Learning what sounds are part of a language Learning what sounds are part of a language Learning to categorize sounds across different speakers and contexts Learning to categorize sounds across different speakers and contexts Learning what groups of sounds constitute words Learning what groups of sounds constitute words Learning what prosodic cues (pitch, intonation, etc.) are part of a language Learning what prosodic cues (pitch, intonation, etc.) are part of a language There are developmental stages during the first year of life There are developmental stages during the first year of life Phonetic acquisition continues until adulthood Phonetic acquisition continues until adulthood Available tools to language learners Available tools to language learners Statistical regularities in the auditory input Statistical regularities in the auditory input Motherese, social interaction Motherese, social interaction Multimodal statistics (lip reading) Multimodal statistics (lip reading) “Learning by doing” (babbling) “Learning by doing” (babbling) Specialized neural apparatus? Specialized neural apparatus?

Some research questions How different sound features (prosody, transition probabilities, multimodal correlations) interact at different stages of development How different sound features (prosody, transition probabilities, multimodal correlations) interact at different stages of development The relationship between production and perception The relationship between production and perception Brain activity related to phonetic acquisition Brain activity related to phonetic acquisition Differences between L1 and L2 Differences between L1 and L2 and more… and more…

What I will cover ( Agenda! ) Part I: “Phacts about Phonemes” Part I: “Phacts about Phonemes” Some properties of auditory input that I think are interesting / useful / good to know Some properties of auditory input that I think are interesting / useful / good to know Part II: Some key results in the field Part II: Some key results in the field Infant timeline Infant timeline 5 important papers 5 important papers Part III: Other interesting papers Part III: Other interesting papers The role of social interaction The role of social interaction Babbling in sign language Babbling in sign language Phonetic acquisition in blind children Phonetic acquisition in blind children

Part I: “Phacts” about Phonemes a.k.a. “Phun” with Phonemes

The components of language Phonemes Phonetic units  Phonetic units  are combined into phonemic categories or just phonemes Example - /r/ and /l/ are phonetic units that compose : Example - /r/ and /l/ are phonetic units that compose : different phonemes in English different phonemes in English the same phoneme in Japanese. the same phoneme in Japanese. Each language uses about 40 phonemes Each language uses about 40 phonemes

Properties of phonemes: across languages “General auditory perceptual abilities provided the ‘basic cuts’ that influence the choice of sounds for the phonetic repertoire of the world’s languages” (Kuhl, NRN, 2003) Infants are more sensitive to phonemic boundaries than other sounds at an “equivalent distance,” even in languages they have not heard. (Eimas, 1988) Image source:

Phonemes, phonetic units, who cares? To distinguish all the words in a language, a person must be able to distinguish all the phonemes of that language To distinguish all the words in a language, a person must be able to distinguish all the phonemes of that language Why? Phonemes are defined as the minimum element of contrast between words Why? Phonemes are defined as the minimum element of contrast between words ‘rake’ vs. ‘lake’ ‘rake’ vs. ‘lake’ Coding the phonemes of speech data is a difficult problem for computers Coding the phonemes of speech data is a difficult problem for computers

Telling phonemes apart Image source: Wikipedia entry for Formant

Telling phonemes apart Are auditory properties ambiguous? Figure source: Kuhl P. Nat Rev Neuro, 2004.Original figure: Peterson & Barney, 1952

Telling phonemes apart Auditory properties: maybe less ambiguous than we thought (at least for vowels). Figure source: Hillenbrand et al 1995 Hillenbrand’s group added sound duration and formant contours. Higher dimensional space of auditory features.

Telling phonemes apart Distinguishing most phonemes across speakers and contexts is still a very difficult problem, though. Source: Tony Ezzat, CBCL Male (Adler) Female, high voice, a bit slower (Mary Pat)

Open research question What are the relevant features, then, for acquiring phonemes (or distinguishing them with a computer)? What are the relevant features, then, for acquiring phonemes (or distinguishing them with a computer)? Auditory Auditory Articulatory (production-related) Articulatory (production-related)

Part II: Some Key Results in Phonetic Acquisition since 1983

1. Acquisition over the 1 st year: an overview Figure source: P Kuhl. Nat Rev Neurosci. 04.

2. Acquisition in older children Hazan and Barrett (2000) showed that phonetic acquisition occurs even after the age of 12. Hazan and Barrett (2000) showed that phonetic acquisition occurs even after the age of 12. They used a minimal pair procedure: They used a minimal pair procedure:

2. Acquisition in older children Hazan and Barrett (2000) A sound is synthesized that “morphs” in equal steps between Sue and Shoe. A sound is synthesized that “morphs” in equal steps between Sue and Shoe. The cues that distinguish the sounds can be varied independently or together. The cues that distinguish the sounds can be varied independently or together. Subjects are tested on their choice of Sue or Shoe at each step. Subjects are tested on their choice of Sue or Shoe at each step. Sue vs. shoe - Friction frequency - F 2 Transition % Sue Bottom figure: P Kuhl. NRN. 04.

2. Acquisition in older children Hazan and Barrett (2000) Slope of psychometric function Children 6-12 vs. Adults Adult perception is significantly more “categorical” than 12 y.o.’s Different 50% equivalence point in children and adults

3. Babies can distinguish fewer sounds as they age From: Werker and Tees (1983) Thompson (Salish) is a Native Indian language spoken in British Columbia. Thompson phonemes are not distinguishable by English speaking adults. They are distinguishable by babies. Experiment 1

3. Babies can distinguish fewer sounds as they age From: Werker and Tees (1983) Experimental procedure: Conditioned Head Turn Babies are conditioned to look at the speaker when there is a sound change by getting visual reinforcement for correct head turns Procedure works only in around 25% of babies Photo source: Werker lab website

3. Babies can distinguish fewer sounds as they age From: Werker and Tees (1983) Cross sectional data: subjects, M&F approx split, for each bar in figure. Longitudinal data: 3 M, 3 F Experiment 2 Tested more ages Tried Salish and Hindi Cross sectional data Longitudinal data

4. Babies sensitive to frequency distributions in sounds From: Maye, Werker, and Gerken (2002) Synthetic sounds ranging from [da] to [ta]. Training set 1: There are more sounds on the ends near [da] and [ta] (bimodal distribution). Training set 2: Most sounds are in the middle (unimodal distribution).

4. Babies sensitive to frequency distributions in sounds From: Maye, Werker, and Gerken (2002) Experimental procedure: Looking time experiment comparing alternating and non-alternating stimuli Experimental procedure: Looking time experiment comparing alternating and non-alternating stimuli Babies are 6-8 months old. Babies are 6-8 months old. Results: Results: Babies look longer at [ta] and [da] pairs after they have been trained on the bimodal distribution.

4. Babies sensitive to frequency distributions in sounds From: Maye, Werker, and Gerken (2002) Interesting corollary: Interesting corollary: After hearing unimodal stimuli, babies actually discriminate [ta] and [da] worse than most infants their age After hearing unimodal stimuli, babies actually discriminate [ta] and [da] worse than most infants their age Does this mimic what happens when infants hear a unimodal distribution in their ambient language? Does this mimic what happens when infants hear a unimodal distribution in their ambient language? Only 6 blocks of 16 training sounds were used here Only 6 blocks of 16 training sounds were used here

5. Using statistical information to learn word boundaries From: Saffran, Aslin, and Newport (1996) One key problem in word segmentation: Strategies: - Statistical ‘pretty baby’ transitional probabilities - Prosodic (word stress) - Prosodic (word stress) Image source: P Kuhl. NRN. 04.

5. Using statistical information to learn word boundaries From: Saffran, Aslin, and Newport (1996) Can 8 month old babies extract transitional probabilities from synthesized speech that contains no breaks, pauses, stress differences, or intonations? Can 8 month old babies extract transitional probabilities from synthesized speech that contains no breaks, pauses, stress differences, or intonations? Experimental procedure: a Looking Time experiment Experimental procedure: a Looking Time experiment Babies hear 2 minutes of speech Babies hear 2 minutes of speech Looking at a light elicits a ‘word,’ ‘part word,’ or ‘non word’ on repeat Looking at a light elicits a ‘word,’ ‘part word,’ or ‘non word’ on repeat Looking time is measured Looking time is measured

5. Using statistical information to learn word boundaries From: Saffran, Aslin, and Newport (1996) Training: pabikugolatupabikudaropi Testing: word part-word pabiku kudaro Figure source: P Kuhl. NRN. 04.

6. Prosody vs. statistics to segregate words From: Johnson and Jusczyk (2001) Replicated Saffran et al.’s results Replicated Saffran et al.’s results Added intonation and stress cues Added intonation and stress cues 90% of English words stress the first syllable 90% of English words stress the first syllable When statistics says, “isn’t a word,” but prosody says “is a word,” who wins?? When statistics says, “isn’t a word,” but prosody says “is a word,” who wins?? Experiment 1: Replication of Saffran et. al Experiment 1: Replication of Saffran et. al Experiment 2: Part-words had a stressed first syllable Experiment 2: Part-words had a stressed first syllable Experiment 3: Part-words were coarticulated (spoken together), while statistical words were not. Experiment 3: Part-words were coarticulated (spoken together), while statistical words were not. Experiment 4: Control: statistical words were coarticulated, part words were not. Experiment 4: Control: statistical words were coarticulated, part words were not.

6. Prosody vs. statistics to segregate words From: Johnson and Jusczyk (2001) Experiment 1: Replication of Saffran et. al Experiment 2: Part-words had a stressed first syllable Experiment 3: Part-words were coarticulated (spoken together), while statistical words were not. Experiment 4: Control: statistical words were coarticulated, part words were not. In 8 month olds, prosody wins.

6. Prosody vs. statistics to segregate words From: Johnson and Jusczyk (2001) To what extent do these results generalize to Different developmental stages?. Different developmental stages?. Different input conditions? Different input conditions?

Part III: Odds and Ends (Other interesting results)

Social effects on phonetic acquisition Kuhl et al. (03) show that English speaking 9 month old babies can learn Mandarin phonetic contrasts from a live person but not a video of that person. Goldstein et al. (03) show that 8 month old babies receiving positive feedback from their mothers vocalized more than yoked controls. Figure source: P Kuhl. NRN. 04.

Phonetic acquisition in the blind Evidence of multimodal phonetic acquisition. Rowland (83) shows that blind children make fewer vocalizations but begin babbling at the same time as sighted children. Slide source: N Pitchford lecture notes, Nottingham University

Babbling in sign language learners Pettito et al. (91) show that deaf babies exposed to sign language make ten times more “babbling” hand gestures than controls.

Summary

What we have learned 1. Languages are distinguished in part by their sets of phonemes 2. Discriminating phonemes is a challenging computational problem 3. Phonetic learning and performance proceeds in stages, beginning in early infancy 4. Phonetic learning continues into adulthood 5. Babies can distinguish fewer sounds as they age From: Werker and Tees (1983) 6. Babies are sensitive to frequency distributions in sounds From: Maye, Werker, and Gerken (2002) 7. Statistical information can be used to learn word boundaries From: Saffran, Aslin, and Newport (1996) 8. Prosody can be compared with statistics to segregate words From: Johnson and Jusczyk (2001) 9. Phonetic learning is affected by social context 10. Some facts about phonetic acquisition in blind and deaf learners

Thanks!