1. Application of Emotion Recognition Methods in Automotive Research Matthias Wimmer, Christian Peter, Jörg Voskamp, Martin A. Tischler Institut für Informatik 2nd Workshop “Emotion and Computing”, 10.9.2007, Osnabrück

2. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"2 Theoretical Background of Driving Pleasure Driver‘s Objective: Optimal Activation Task Demands Driveability Sportiveness Comfort Capability Task Difficulty Resources und Restrictions Vehicle Environment Emotional estimation Transfer of Driver‘s Request Jordan (2000): “Four Pleasures”  Physio-pleasure  Psycho-pleasure  Socio-pleasure  Ideo-pleasure based on Fuller (2005)

3. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"3 Measurement of Emotions in Vehicles EMOTION Subjective component Verbal report Expressive component Video/Audio- recording Behavioral component Driving dynamics Physiological component Measurement Cognitive component Verbal report  Facial expressions  Gestures  Changes in voice ECG, EMG, EDA Questioning during driving Driving style and operator control action Cognitive appraisal of the vehicle and the situation

4. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"4 Pilot Study „Driving Pleasure W204“ 2007 C220 CDI (W204)1982 190E (W201)

5. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"5 Cooperation

6. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"6 Subjects: 8 non professional drivers (age: 33-53)

7. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"7 Task: Drive on three courses Bosch Proofing Ground Boxberg (near Heilbronn) Autobahn Country road Handling course

8. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"8 Technical Setup driving dynamics meter EREC glove video camera microphone cell phone handsfree set EREC recording unit

9. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"9 The Methods in Detail Physiological measurement system EREC (skin resistance, heart rate, temperature) Interviews and questionnaires after driving each car Video and audio recorder on the back seat Drivers were asked to “think aloud”

10. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"10 Subjective Driving Experience 1,03,05,07,0 subjective control no worries about safety drove too risky vehicle is predictable vehicle met my desires MB 190E MB C220 strongly agreestrongly disagree N=8, Method: questionnaire

11. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"11 Facial Expression Recognition Institut für Informatik Mercedes-Benz 190E Mean expression of happiness: 4,2% Peak value: 15,0% Mercedes-Benz C-Class 220 Mean expression of happiness: 5,5% Peak value: 19,8% 0% = neutral face 100% = maximum expression

12. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"12 1 13579 A C H A H C MB 190E MB C-Class 220 A Autobahn C Country road H Handling Valence Activation Speech Analysis + + - - Most important parameters changes of pitch intensity energy changes over frequency bands Average confidence 1.1-1.2 for valence 0.9-1.0 for arousal

13. Tischler, Peter, Wimmer & Voskamp, Application of Emotion Recognition Methods in Automotive Research, 2nd Workshop "Emotion and Computing"13 Conclusions Methods show corresponding results Speech analysis Easy to apply, but speech is not a natural product while driving Facial expression recognition Challenge in fitting the face model and classifying the facial expression Problems: changing light and background, non emotional-related head movements, driver is talking Advantages of tested methods Non disturbing Continuous data collection Useful completion to psychological methods