1. Combination of Speech and Tangible Interfaces for Automotive Dialog Systems: An Experimental Study Margarita Pentcheva

2. Combination of Speech and Tangible Interfaces for Automobile Dialog Systems Motivation increase of on-board and accessory devices; infotainment services variety of assistance and comfort functions

3. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Motivation interaction between driver and car interfaces → the smallest possible distraction effect.

4. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Combining Tangible Interfaces and Spoken Interaction „Faster!“ Using tangible interaction to set context for speech dialog

5. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Combining Tangible Interfaces and Spoken Interaction “Front right window” “All windows” “Rear seat heating” “Volume” Using speech to set context for tangible interaction (turn-and-push dial)

6. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Why a combination of turn-and-push dial and speech? turn-and-push dial is used for controlling comfort functions. BUT: Hierarchical menus need to be browsed before the dial does what the driver wants it to do.

7. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Why a combination of turn-and-push dial and speech? speech is used as a means of interaction suitable for in-car use. BUT: speech alone is not the most intuitive means of interaction in every case.

8. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Experimental Study on the Effect of Manual vs. Speech vs. Multimodal Input on Driver Distraction air condition fan seat heating radio windows

9. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Hypotheses Hypothesis 1: The speech-only and combined (multimodal) conditions outperform the manual condition in terms of distractiveness and convenience. Hypothesis 2: The combined (multimodal) condition leads to a more precise accomplishment of the secondary task.

10. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Experimental Study Mercedes R-Class used for the experiment. HMI

11. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Lane-Change Test Area shows the quality of driving Area is influenced by – Perception (missed sign) – Reaction – Maneuver – Lane keeping Standardization ISO TC22/SC13/WG8 (Road vehicles-Ergonomic aspects of transport information and control systems)

12. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Driving Activity Load Index (DALI) Questionnaire Global attention demand- demand required to complete the task  visual demand only  auditory demand only  tactile demand: originally related to vibrations, here adapted to manual handling  stress  temporal demand  interference

13. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Experimental method Participants 24 subjects (11 men, 13 women) age range: 21-60 non-professional drivers no experience with driving simulator novices in Mercedes R-Class driving

14. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Procedure approx. duration Vehicle and driving simulator briefing10 min. Practice drive 3 min. Baseline 1 3 min. Experimental part (not presented here)10 min. Baseline 2 3 min. Drive X (X,Y, or Z balanced); manual 3 min. Questionnaire manual 5 min. Drive Y(X,Y, or Z balanced); speech-only 3 min. Questionnaire speech-only 5 min. Drive Z(X,Y, or Z balanced); multimodal 3 min. Questionnaire multimodal 5 min. Baseline 3 3 min. General Questionnaire (all conditions) 5 min. sum 60 min.

15. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Results of the experiment with respect to the driving performance (LCT) F(4, 20)= 26.73, p<.001

16. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Results of the experiment with respect to the driving performance (LCT) F(4, 20)= 26.73, p<.001

17. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Results of the experiment with respect to the driving performance (LCT) F(4, 20)= 26.73, p<.001

18. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Performance in the secondary task measured on the number of completed and uncompleted tasks

19. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Breakdown of errors in terms of imprecision

20. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Subjective rating of demand (DALI)

21. Margarita PentchevaCombination of Speech and Tangible Interfaces for Automobile Dialog Systems Summary How can speech and tangible interfaces be combined in order to provide effective multimodal interaction in vehicles? Experimental investigation of the effects of manual, speech-only and multimodal interaction with the car's comfort functions on driver distraction. Multimodal interfaces have potential to reduce mental and visual demands. Multimodal input is for many precise operations faster and more efficient than speech or manual input.

22. Thank you! Questions?