Stress, Focus, and the Roots of Segmental Variation Kenneth de Jong Indiana University Work reported here in collaboration with Eric Oglesbee, Kyoko Okamura, Noah Silbert and Bushra Zawaydeh, and supported by NSF: BCS Slides at:

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Segmental Variation: Problem or Solution? Everything varies, and the variation is the most interesting part EXAMPLE FROM MOTOR LITERATURE: Variation in production tells us about linguistic goals: –Abbs et al (1984, among others) -> trading relations in production tell us about motor synergies organized around consonantal closure –Perkell et al (1993); de Jong (1997) -> trading between lip rounding and tongue-body retraction suggests a common goal of acoustic timbre depression Current research: stress variation similarly may be informative about why and how stress and syllables go together

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress as Attentional Focus Stress is conventionalized Attentional Focus (de Jong, 2000) STRESS –Syllabic domain –Grammatical property of a language –Inheres in lexical items FOCUS –Can operate in various domains –Property of production –Inheres in particular utterances BOTH –Variation modeled in terms of local modulation along Lindblom’s H&H dimension

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Lindblom’s H r + H o Model (Lindblom 1990) Hyperarticulation (Hr) - Hypoarticulation (Ho) Hr = output oriented constraints Ho = production-oriented factors Not OR …; but AND

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress extension of H r +H o (de Jong, 1991; 1995; 1998) Stress effects specified as hyperarticulation At that time: This was Uhglee - super complicated Essentially stress is open-ended adaptation, how do we predict the effect of stress? Hyper-end: Requires a model of the phonological contrast space Hypo-end: Requires a model of motor constraints and optimization

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress extension of H r +H o (de Jong, 1991; 1995; 1998) TODAY the reverse: If H+H characterization is right; Stress & Focus variation can tell us about phonological contrast space and motor constraints Specifically: Variation suggests how phonological contrast expression and motor constraints might contribute to creating a relationship between stress and syllables

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -Connect contrastive dimensions to focus modulations: onset voicing and VOT & F0 in voicing contrasts (Oglesbee, 2008) -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -Connect contrastive dimensions to focus modulations: onset voicing and VOT & F0 in voicing contrasts (Oglesbee 2008) -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Oglesbee (2008) Measured perceptual sensitivities in a goodness estimation task to VOT, F0 contour, and intensity dynamics for /p/ and /b/ Compared productions of /p/ and /b/ in focus and non-focus positions Subjects created larger differences with focus in dimensions with perceptual sensitivity Dimensions: –All: VOT –S1: vowel amplitude ramp (/b/ sharper) –S2: F0 contour (/p/ higher)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality (de Jong & Zawaydeh, 2002; de Jong, 2004) -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Non-high front vowels -> Vowel quality Stress by voicing Dashed arrows indicate voicing effects Solid arrows indicate effects going from: Unstressed -> secondary stressed Secondary -> primary stressed - F2 (Hz) - F1 (Hz)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre (Silbert & de Jong, 2008) - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Silbert & deJong (JASA, 2008) Examined English /f/ /v/ /s/ & /z/ Prevocalic, post-vocalic & focus, non-focus Measured intensity, duration, spectral quality and dynamics of quality Focus affects only duration and intensity -> NO spectral effects

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: coda voicing and vowel duration (de Jong & Zawaydeh, 2002; de Jong, 2004)

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Voicing -> Vowel duration Stress by voicing Focus by voicing

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Focus & Stress Effects -VOT, F0, and onset amplitude cues to voicing become more extreme with focus, in parallel with perceptual sensitivity in goodness tasks -VOWELS: vowel quality becomes more extreme with focus and stress -> vowel space is a gradient contrast space with weak motor constraints -CONSONANTS: fricative timbre is not affected by focus -> rigid motor constraints - More CONSONANTS: other consonantal cues are heavily affected

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Generalizations so far Stress effects strongly localized in vocalic nucleii Consonant effects similarly most readily apparent in nucleii Consonant acoustics affected in terms of durational and intensity properties All of which suggests a model of syllable nucleii as motorically fluid repositories of variation Consonantal margins are motorically constrained

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Types of Explanations (deJong, 2007) Hardware Physiological facts and constraints on control Activeware Facts about action and coordination. Motoric propensities which encourage certain articulatory coordinations OR … Shareware Facts about how talkers and listeners couple to share information. Requirements of communication with a listener OR … Ancestorware It’s the way mom & dad did it. Languages are shared repositories for an enormous amount of information.

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress variation - Attentional dynamics Many previous models of variation at the syllable level have Hardware or Activeware explanations (e.g. de Jong, 2003, where syllabic units are coodinative gestural constraints) The attentional focus model is a Shareware explanation for stress-related phonetic variability.

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress system A stress system = a convention in which both speakers and listeners pay more attentional focus to certain syllables than to others Productions of stressed items are given more attentional focus (de Jong, 1991; 1995; 1998) Perceptions of stressed items are more important for intelligibility (Cole, et al, 1978, Cole and Jakimik, 1980; Bond and Garnes, 1980; Terken and Nooteboom, 1987) Producers and perceivers are coupled during spoken communication. This is a shareware system.

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt But … not the whole story Okamura (ms); de Jong (2004); de Jong & Zawaydeh (2002) Stress differs across languages: Arabic and English focus effects concentrated in stressed syllables. Not true of (say) Japanese. Though stress and focus effects are similar; stress effects more consistent across subjects than focus effects

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt General Model Summary Suggests a model with (at least) two components 1)Shareware: Attentional variation happens according to speakers’ perceptions of listeners’ ability to perceive -This hyperarticulation in general is unevenly distributed in the signal 2)Ancestorware: Attentional variation gets contributed to the pool of experiences that the listener has with particular lexical items -Shareware factors gets conventionalized in Ancestorware

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Why syllables? Syllables might be implicated in both parts Shareware: syllables may act as integral units in the apportionment of attention. Syllabic nucleii are ‘acoustic objects’ which exhibit attentional integration. (See online slides for more on this part of model.) Ancestorware: production variation is more readily and systematically encoded in vocalic nucleii, punctuated by less gradiently variable consonantal margins. Focus variation along H+H lines is localized in syllable nucleii, so over time, nucleii get stress

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt End

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt ADDENDUM: Attentional Dynamics in Audition Auditory attention work by Mari Jones & others (Jones, 1976; Jones & Boltz, 1989; Jones & Yee, 1993; Large & Jones, 1997) –attentional selectivity -> focus –attentional capture -> prominence –attentional integration -> syllabic units –temporal expectancy -> rhythmic alignment

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress variation - Attentional dynamics Part 1. attentional selectivity: some parts of a stimulus are more readily acted upon than others Stress => some syllables are attentionally selected

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress variation - Attentional dynamics Part 2. attentional selectivity arises from ---> attentional capture: parts which suddenly change in salient dimensions tend to garner such selective advantages Attention is attracted to acoustic events where sudden changes take place F0: Pitch accents dock to stressed locations Amplitude: stress is a property of syllables, which are characterized by sonority rises

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress variation - Attentional dynamics Part 3. attentional selectivity also ---> may exhibit attentional integration: aspects which get attended to as a unit are those which work together to define an object or event works on portions of speech which are auditorially cohere Syllables might be such auditorily coherent units

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt Stress variation - Attentional dynamics Part 4. Attentional modulation is generally governed by temporal expectancy. High attention areas may, under appropriate conditions, come at temporally predictable intervals. Hence stress is sometimes governed by rhythmic, metrical patterning.

.//jones.ling.indiana.edu/~kdejong/ ASA_Paris3.ppt References Abbs et al (1984). Control of multimovement coordination: Sensorimotor mechanisms in speech motor programming. Journal of Motor Behavior, 16(2): Bond & Garnes (1980). Misperceptions of fluent speech. In R. Cole (ed.), Perception and Production of Fluent Speech. Hillsdale, NJ: Erlbaum, pp Cole et al (1978). Perceptibility of phoneic features in fluent speech. JASA, 64: Cole & Jakimik (1980). How are syllables used to recognize words? JASA, 67: De Jong (1991). The Oral Articulation of English Stress Accent. Ohio State University Ph.D. thesis. De Jong (1995) The supraglottal articulation of prominence in English: Linguistic stress as localized hyperarticulation. JASA, 97: De Jong (1997). Labio-velar compensation and acoustically-based motor equivalence. JASA, 101: De Jong (1998). Stress-related variation in the articulation of coda alveolar stops: Flapping revisited. Journal of Phonetics, 26: De Jong (2000). Attention modulation and the formal properties of stress systems. In J. Boyle, J-H. Lee, and A. Okrent (eds.), Chicago Linguistic Society 36, Vol. 1, pp. 71 – 91. Chicago: Chicago Linguistics Society. De Jong (2004). Stress, lexical focus, and segmental focus in English: Patterns of variation in vowel duration. Journal of Phonetics, 32: 493 – 516. De Jong & Zawaydeh (2002). Comparing stress, lexical focus, and segmental focus: Patterns of variation in Arabic vowel duration. Journal of Phonetics, 30: Jones (1976). Time, our lost dimension: Toward a new theory of perception, attention, and memory. Psychological Review, 83: Jones & Boltz (1989). Dynamic attending and responses to time. Psychological Review, 96: Jones & Yee (1993). Attending to auditory events: the role of temporal organization. In S. McAdams and E. Bigand (eds.), Thinking in Sound: the Cognitive Psychology of Human Audition. Oxford: Clarendon Press, pp Large & Jones (1997). The dynamics of attending: How we track time varying events. Psychological Review, 106: Lindblom (1990). Explaining phonetic variation: a sketch of the H&H Theory. In H.J. Hardcastle, and A. Marchal (eds.), Speech Production and Speech Modeling, NATO ASI Series D: Behavioural and Social Sciences, Vol. 55 (Kluwer A.P., Dordrecht). Oglesbee (2008). Multidimensional Stop Categorization in English, Spanish, Korean, Japanese, and Candian French. Indiana University Ph.D. thesis. Okamura (ms). The effects of lexical focus and phonological focus on pitch accent in Japanese. Information available from kokamura at Indiana dawt edu. Perkell et al (1993). Trading relations between tongue-body raising and lip rounding production in the vowel /u/: a pilot "motor equivalence" study. JASA, 93, Silbert & de Jong (2008). Focus, prosodic context, and phonological feature specification: Patterns of variation in fricative production. JASA, 123: 2769 – Terken & Nooteboom (1987). Opposite effects of accentuation and deaccentuation on verification latencies for given and new information. Language and Cognitive Processes, 2: