1. Expressivity in Sound and Music Roberto Bresin, Sofia Dahl, Anders Friberg KTH, Stockholm – SOb project partner {roberto, sofia, andersf}@speech.kth.se http://www.speech.kth.se/music/performance Music Acoustics Group Speech Music Hearing Dep. (TMH) - Royal Institute of Technology (KTH) - Stockholm

2. R Bresin: SPM2001 Dead-pan version by computer and sampler Performed by Alfred Brendel Why does it sound good? Brendel I OI deviations (%) First 8 bars Schumann’s Träumerei

3. R Bresin: SPM2001 Performance rules obtained mainly with 2 methods: analysis-by-measurement analysis-by-synthesis  Generative grammar for automatic music performance Music performance: analysis and synthesis

4. R Bresin: SPM2001 MUSIC SCORE (MIDI) PERFORMED MUSIC (MIDI) DIRECTOR MUSICES (performance rules) PROGRAMMER PROFESSIONAL MUSICIAN NEW / MODIFIED RULE K values (Rule quantity) Analysis-by-synthesis of music performance Example: punctuation rule

5. R Bresin: SPM2001 Director Musices performance rules (~30 rules) Differentiation Rules Example: Duration Contrast Rule Grouping Rules Example: Phrase Articulation Rule Synchronization/Ensemble Rules Example: Ensemble timing Other Rules Example: Repetition Articulation Rule

6. R Bresin: SPM2001 Director Musices

7. R Bresin: SPM2001 Pianist Rules Phrase Arch Rule Fitted to Three Famous Pianists

8. R Bresin: SPM2001 Body control of music and sound Music performance and body motion

9. R Bresin: SPM2001 Friberg, A., & Sundberg, J. (1999) Journal of the Acoustical Society of America, 105 (3) Final ritardando Runners’ deceleration Final ritardando alludes to a stopping runner Three different melodies with a final ritardando curve from the stopping runners

10. R Bresin: SPM2001 Legato and walking Staccato and running Legato and staccato allude to walking and running? Control model example:

11. R Bresin: SPM2001 Accents in drumming: kinematics and timing. Frontal view of stick motion trajectory

12. R Bresin: SPM2001 Comparisons between subjects Motion trajectories of the stick seen from the side.

13. R Bresin: SPM2001 Two singers sing excerpt from Gounod’s Faust 1. Nicolai Gedda 2. Thomas Burns Wrong body motion: Vocal ugliness

14. R Bresin: SPM2001 Gedda Burns Frequency (kHz) Mean sound level (dB) Long-term average spectra

15. R Bresin: SPM2001 Virtual fixing of Burn’s problems 1. Synthesis of original 2. Exaggerated glottal adduction Medication: enhance fundamental! 3. Irregular vibrato & out of tune Medication: increase vibrato regularity! apply equally tempered tuning! 4. Horrible formants Medication : Cluster F 3, F 4, F 5! 5. Raise singer’s formant by 200 Hz!

16. R Bresin: SPM2001 Happy or sad music?

17. R Bresin: SPM2001 A.Gabrielsson and P.Juslin suggested a list of important cues for the simulation of emotions in music performance (Psychology of Music, 1996, vol. 24) No expression Tenderness Solemnity Happiness Sadness Anger Fear Performance Rules: Synthesis of Emotional Expression

18. R Bresin: SPM2001 Synthesis of Emotion: SADNESS (1/2)

19. R Bresin: SPM2001 DEAD-PAN IOI deviations dB deviations SAD Synthesis of Emotion: SADNESS (2/2)

20. R Bresin: SPM2001 Synthesis of Emotion: Listening Test Results Chance Level: 14%

21. R Bresin: SPM2001 Emotional expression can be derived directly from the music score, simply by enhancing music structure  We want to recognize a meaning!! Conclusions

22. R Bresin: SPM2001 http://www.speech.kth.se/music/performance More info, sound examples, sw...

23. R Bresin: SPM2001 The End www.soundobject.org