The prosodic marking of the contrast between restrictive and appositive clause in Dutch Vincent J. van Heuven With the help of: Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic, Marjoleine Sloos, Hanna de Vries Linguistics Program, Universiteit Leiden

Towards the phonetics of the relative clause in Dutch Vincent J. van Heuven With the help of: Crit Cremers, Hanna Gauvin, Constantijn Kaland, Eddin Najetovic, Marjoleine Sloos, Hanna de Vries Linguistics Program, Universiteit Leiden

van Heuven, NVFW/MPI 3 Introduction  Framework: improving prosody of text-to- speech (TTS) systems  Our parser/generator Delilah computes rich syntactic and semantic structures  Can be interfaced to standard TTS systems for Dutch  Opportunities for phonetic optimalisation of output

van Heuven, NVFW/MPI 4 Introduction  Grammatical distinction in relative clause (1) Restrictive clause Within scope of antecedent Rel. pronoun is not subject of relative clause Not preceded by comma in text Suggests relatively shallow prosodic boundary E.g. No journalist [who signed the petition], was arrested.

van Heuven, NVFW/MPI 5 Introduction  Grammatical distinction in relative clause (2) Appositive clause Not within scope of antecedent Rel. pronoun is subject of relative clause Preceded by comma in text input Suggests relatively deep prosodic boundary, E.g. Michael, [who beat up his girl friend], was arrested.

van Heuven, NVFW/MPI 6 Introduction  Questions How is the contrast between restrictive and appositive clause coded in the prosody? Does the listener associate a specific prosodic structure with each type of clause? Does correct prosodic marking yield better evaluation scores of text-to-speech system?

van Heuven, NVFW/MPI 7 Prosodic boundaries  The deeper a prosodic boundary, the longer the physical silence (pause) immediately before the boundary... the stronger the domain-final lengthening... the more disruptive the intonation pattern

van Heuven, NVFW/MPI 8 Approach (1)  Perceptual evaluation of prosodic marking of Synthetised speech with restrictive vs. appositive clauses With / without pause With / without domain-final lengthening Four different melodies

van Heuven, NVFW/MPI 9 Approach (2)  Four melodies at utterance-internal boundary Cohesion marker H*L No/neutral marker H*+L Moderate break H* %L Strong break H*L H% %L

van Heuven, NVFW/MPI 10 Approach (3)  Two sentence structures that force just one interpretation onto relative clause Restrictive (with downward quantifiers) No journalist who signed the petition, was arrested Few students who are are fraternity members, dislike a beer Appositive (with proper names and/or evidentials) Michael, who beat up is girl friend, was arrested The woman, who – by the way – is wearing a pink dress, is from Germany

van Heuven, NVFW/MPI 11 Approach (4)  Judgment/evaluation task for listeners How do you rate the way the speaker reads the following sentence? Scale from 0 to 10 0: very poor/inappropriate 10: perfect

van Heuven, NVFW/MPI 12 Hypotheses  Appositive clause correlates with Physical silence Pre-final lengthening Maximally disruptive melodic pattern Strong break > Moderate break > Neutral > Cohesion Three factors are additive i.e. better scores as more factors reinforce one another

van Heuven, NVFW/MPI 13 Hypotheses  Restrictive clause correlates with Absence of physical silence Absence of domain-final lengthening Minimally disruptive melody Cohesion > Neutral > Moderate break > Strong break Factors are additive

van Heuven, NVFW/MPI 14 Methods (1)  Factors Appositive vs. restrictive clause Two lexically different instantiations With / without pause (of 200 ms) With / without domain-final lengthening (+40%) Four different melodies  Total 2 x 2 x 2 x 2 x 4 = 64 types Presented twice = 128 tokens per listener

van Heuven, NVFW/MPI 15 Methods (2)  Materials spoken by native speaker of Dutch  Digital tape splicing for removal/insertion of lengthening and pause  PSOLA for duration manipulation and import of computer-generated melodies (imported from Fluency text-to-speech)

van Heuven, NVFW/MPI 16 Methods (2)  Listeners 20 native speakers of Dutch  Presentation through loudspeakers  11-point quality judgment scale  Demonstration of sound files:

GEEN journalist die de verklaring onderTEkend had, werd OPgepakt ‘No journalist who the statement signed had, was arrested’ No pause, +pause, No length +length Disaster: the red condition (strong break) was inadvertently generated as the blue one (neutral). It will therefore be absent from the results section. My student Constantijn Kaland is now re-running the complete experiment for his BA thesis.

van Heuven, NVFW/MPI 18 Results  First: main effects Yes / no physical pause Yes / no domain-final lengthening Type of melodic boundary marking  Second: interactions i.e. breaking factors down

Effect of physical pause  Absence of pause is preferred for restrictive clause  Presence of pause is disfavored for restrictive clause  Difference is significant  Both presence and absence of pause are OK for appositive clause

Effect of domain-final lengthening  Tiny effects in predicted direction No lengthening preferred for restrictive clause Lengthening preferred for appositive clause  Effects are statistically insignificant

Effect of melodic configuration  Order of conditions reasonably in line with prediction Indeed: green > yellow > red for restrictive clause But should be red > yellow > green for appositive clause  Effects are statistically insignificant

van Heuven, NVFW/MPI 22 Summary of results so far  Only one effect Listeners prefer restrictive clauses without a physical pause separating clause from antecedent  All other prosodic markings are OK for both types of clause  Now let us look at interactions

van Heuven, NVFW/MPI 23 Results: interactions  Effects of (i) lengthening and (ii) melody evaluated separately for presence vs. absence of physical pause

Interaction of pause and lengthening  Effect of lengthening as predicted (though not always significant)  More clearly when pause is absent than when present

Interaction of pause and melody  Preference for cohesive H*L with restrictive clause  But only when pause is absent Physical pause absentPhysical pause present

van Heuven, NVFW/MPI 26 Summary of results  Effects of (i) lengthening and (ii) melodic configuration are as predicted  But only if pause is absent  Indicates hierarchical structure physical pause is the primary cue lengthening and melody are subsidiary cues

van Heuven, NVFW/MPI 27 Conclusions  Appositive and restrictive relative clauses in Dutch are marked by different prosodic boundaries, which are implemented by different phonetic cues  More specifically, …

van Heuven, NVFW/MPI 28 Conclusions  … the evaluation study shows that the syntax-prosody interface should assign different prosodic boundaries to restrictive and appositive clauses: deeper boundary to appositive clause shallower boundary to restrictive clause  Therefore, …

van Heuven, NVFW/MPI 29 Discussion  … quality of (Dutch) text-to-speech systems can be improved Never insert physical pause before restrictive clause Optionally accompany by appropriate secondary cues Do not lengthen syllable before restrictive clause Use cohesive melody before restrictive clause

van Heuven, NVFW/MPI 30 Discussion  But Implementation of secondary cues is not very effective Fast and dirty implementation requires proper use of pause only  Rerun needed with strong prosodic break  Challenge left for computational linguists: How to compute status of relative clause from text input?

van Heuven, NVFW/MPI 31 The end  Thank you for your attention