1. Ten Myths of Multimodal Interaction
Presented By:Steven Dang
Sharon Oviatt Ten myths of multimodal interaction. Commun. ACM 42, 11 (November 1999),  

2. Sharon Oviatt Phd in Experimental Psychology (1979)
Specialized in infant cognition and language comprehension

3. Language Learning and Comprehension Assessment
Her Thesis Work
Language Learning and Comprehension Assessment
Taught words to prelanguage infants
Measured learning and understanding through a combination of eye gaze, body movement, and vocalizations
Preferential Looking
(Fantz, R. L. 1956)

4. Sharon Oviatt Phd in Experimental Psychology (1979)
Worked at SRI International until 1994
Worked on modelling task-language dynamics
Professor at Oregon Health and Sciences University until 2005
Research Director at Incaa Designs (current)
Educational implications of multimodal technology interactions

5. Goals of the paper
Dispel common “misconceptions” about how multimodal interaction system design
Shift multimodal design away from only multimodal versions of the mouse
Introduce behavioral research to replace misconceptions with design guidelines
Target Audience: readers of “Communications of the ACM”

6. Put that There (1980) 59 seconds
Points to how signal integrity can be improved through contextual cues (ie: other modalities)

7. Are they misconceptions?

8. Myth #2: Speech and pointing is the dominant multimodal integration pattern.
DREAMSPACE (IBM, 1998)
ICONIC (1993)

9. Myth #3: Multimodal input involves simultaneous signals.
Put that there (Bolt, 1980)

10. Myth #4: Speech is the primary input mode in any multimodal system that includes it.  
DREAMSPACE (IBM, 1998)

11. Myth #6: Multimodal integration involves redundancy of content between modes
“Make it this big”

12. Myth #1: If you build a multimodal system, users will interact multimodally.
Gaze+Gesture (Harrison, 2015)

13. Myth #7: Individual error-prone recognition technologies combine multimodally to produce even greater unreliability.
“they[Bolt, 1980] discovered that by integrating speech and gesture recognition with contextual understanding, neither had to be perfect provided they converged on the user's intended meaning” (Billinghurst, 1998)

14. Dreamspace (IBM, 1998)
3min 32 secs

15. Gaze+Gesture (2015)
31 seconds

16. Voelker (2015)
55 seconds

17. The Ideal
1 min 24 sec

18. Discussion
Have we really moved past multi-modal interaction as an analog to mouse and keyboard?
When is multimodal interaction appropriate?
Why aren’t there more multimodal systems?
How might multi-modal interaction enable a different human-computer relationship?

19. 10 Misconceptions
Myth #1: If you build a multimodal system, users will interact multimodally.
Myth #2: Speech and pointing is the dominant multimodal integration pattern.
Myth #3: Multimodal input involves simultaneous signals.
Myth #4: Speech is the primary input mode in any multimodal system that includes it.  
Myth #5: Multimodal language does not differ linguistically from unimodal language.
#1: In reality, users prefer to communicate some information, such as spatial information, multimodally while communicating other information unimodally. Understanding the nature of your information will help you design your input modality(s).
#2 In reality, very little time is spent doing speak-and-point during natural human communication and thus machines are missing many opportunities to capture and enable richer HCI.
#3: In reality, users don't often point to what they are talking about, and when they do, the timing is not following current intuitions. Relying on this signal may force users into unnatural interaction patterns or application errors.
#4: In reality, not all information is relayed through speech, and treating gesturing as strictly a secondary channel for pointing information misses opportunities for richer communication.
#5: In reality, users might use an eleborate verbal command given only a verbal interface, but a verbal and gesture interface might involve both a consise verbal command accompanied by brief gesture.

20. 10 Misconceptions (cont.)
Myth #6: Multimodal integration involves redundancy of content between modes
Myth #7: Individual error-prone recognition technologies combine multimodally to produce even greater unreliability.  
Myth #8: All users’ multimodal commands are integrated in a uniform way.
Myth #9: Different input modes are capable of transmitting comparable content.
Myth #10: Enhanced efficiency is the main advantage of multimodal systems.
#6: In reality, users may convey components of a message through separate channels independently and nonredundantly so that the full message can only be understood by synthesizing all channels.
#7: In reality users adapt to the limitations of a given communication channel choosing the most conducive channel first and by varying the channels used upon error, thus repeating information across channels of different abilities.
#8: In reality, different people might tend to prefer to communicate and/or interact with a system using each modal channel slightly differently, while intra-individual variability is generally low.
#9: In reality some information is easier to see spatially and other information is easier to hear or write/type.
#10: In reality, there are more metrics that can show system benefits such as error-rates, and user enjoyment. Considering metrics beyond efficiency for a particular application may be valuable.