User Interface Design Tools for the Future Multimodal UI Research in the HCC James A. Landay Jason Hong, Scott Klemmer, Jimmy Lin, Mark Newman, & Anoop Sinha HCC Retreat July 5, 2000
7/5/20002 Several Projects on Design UIs of the Future 4 Designer’s Outpost 4 SUEDE 4 Multimodal Design Assistant
7/5/20003 The Future: Information-centric UIs
7/5/20004 Designer’s Outpost: Tangible Tools for Information Design
7/5/20005 Designer’s Outpost: Virtual Feedback on Physical Layouts
7/5/20006 Designer’s Outpost: Virtual Feedback on Physical Layouts
7/5/20007 Designer’s Outpost: Virtual Linking of Related Information
7/5/20008 Designer’s Outpost: Export to Next Stage Tools (DENIM)
7/5/20009 Designer’s Outpost: Infrastructure for Next Prototype 4 Two video/still cameras *still camera above desk/wall +captures ink / IDs *video camera below desk/wall +occlusion free +captures structure 4 SmartBoard *1280 x 1024 resolution +looking for a projector donation *direct pen input 4 Cross iPen tablet *high-res ink capture
7/5/ One Future: Our Access to Information will be via Speech Read my important
7/5/ SUEDE: Low-fidelity Prototyping for Speech-based UIs
7/5/ A Better Future: Our Access to Info will be via Multimodal UIs 4 Take advantage of the best mode of input/output for the task/context at hand 4 How do we design so that these new devices will not seem impoverished relative to desktop computers?
7/5/ A Better Future: Our Access to Info will be via Multimodal UIs 4 How do we combine speech, gesture, etc. into a UI design? *rapid production of “rough cuts” +informal (sketching / “Wizard of Oz”) +iterative design (user testing/fast mods) *generate initial code +UIs for multiple devices +designer adds detail / improve UI 4 Study *uses of these novel modes alone +e.g., SUEDE *construction of multimodal apps +e.g., a multimedia notebook 4 Early stages so far *based on inferring models Model
More information on these design tools at