Download presentation
Presentation is loading. Please wait.
1
Queen Mary, University of London
Automatic generation of multimodal interaction models from behavioural data Marie-Luce Bourguet Computer Science Dept. Queen Mary, University of London
2
Multimodal Interaction Models
Application Multimodal Engine … Speech recognition Gesture recognition
3
Modeling Multimodal Interaction
“remove” pen down pen move pen up gesture delete speech remove reset
4
Modeling Multimodal Interaction: Synchronisation patterns
pen down pen move pen up gesture delete speech remove reset “remove” pen down pen move pen up gesture delete speech remove reset pen down pen move pen up gesture delete speech remove reset
5
Modeling Multimodal Interaction: Mutual disambiguation
pen down pen move pen up gesture delete speech remove reset pen down pen move pen up speech move reset
6
Design Tool
7
Simulation Tool
8
Automatic Generation of Interaction Models
Training Module … Speech recognition Gesture recognition
9
Conclusion Building interaction models from user interaction data presents numerous advantages: It bypasses the difficult task of designing multimodal interaction models The resulting interaction models are highly personalised They should enable natural and robust interaction The FSM based formalism is well adapted because very simple analysis of the data should permit the automatic generation of interaction models.
Similar presentations
