Monday, April 2, 2007 Multisensory & Multimedia Multisensory & Multimedia

Monday, April 2, senses: - Vision, hearing, touch, smell & taste - Connection with computers “sensory deprived and physically limited” - Multimedia experience: multisensory Interviews: - Hiroshi Ishii (Associate professor of Media Arts and MIT) - Durrell Bishop (Teacher of Product Royal College of Art in London) - Joy Mountford (Member of Human Interface Apple) - Bill Gaver (Professor of Interaction Royal College of Art in London ) 5 senses: - Vision, hearing, touch, smell & taste - Connection with computers “sensory deprived and physically limited” - Multimedia experience: multisensory Interviews: - Hiroshi Ishii (Associate professor of Media Arts and MIT) - Durrell Bishop (Teacher of Product Royal College of Art in London) - Joy Mountford (Member of Human Interface Apple) - Bill Gaver (Professor of Interaction Royal College of Art in London ) Overview

Monday, April 2, Interaction requires 2 key components: Controls & representation of results - Graphical UI: representation  pixels on screen control  remote: mouse, keyboard,… -Tangible UI: representation & control: tangible & coupled - GUI  TUI: Painted bits  tainted bits by giving physical form to digital information - Multisensory multimedia interactions: fun & help us do things better - Interaction requires 2 key components: Controls & representation of results - Graphical UI: representation  pixels on screen control  remote: mouse, keyboard,… -Tangible UI: representation & control: tangible & coupled - GUI  TUI: Painted bits  tainted bits by giving physical form to digital information - Multisensory multimedia interactions: fun & help us do things better Hiroshi Ishii

Monday, April 2, 2007 Tangible User Interfaces: example “The reacTable, is a state-of-the-art multi-user electro- acoustic music instrument with a tabletop tangible user interface.”

Monday, April 2, 2007 Tangible User Interfaces - [open][open] - Hiroshi Ishii’s latest project: Jabberstamp

Monday, April 2, 2007 Tangible User Interfaces - Control & representation of results: in a more physical form compared to GUI’s - Directly grab & manipulate results - Abacus: feel, manipulate, touch information - Control & representation of results: in a more physical form compared to GUI’s - Directly grab & manipulate results - Abacus: feel, manipulate, touch information

Monday, April 2, 2007 Tangible User Interfaces - musicBottles: - weather forecast bottle

Monday, April 2, 2007 Durrell Bishop - “Things should be themselves”: physical or virtual objects should be self-evident - Consumer electronics: universally black boxes of standard size - Labels say what a button does: merely an action on a word - Monopoly vs. VCR - “Things should be themselves”: physical or virtual objects should be self-evident - Consumer electronics: universally black boxes of standard size - Labels say what a button does: merely an action on a word - Monopoly vs. VCR

Monday, April 2, 2007 Durrell Bishop - TUI: Marble answering machine

Monday, April 2, 2007 Durrell Bishop - Picking up tagged objects  physical pointers

Monday, April 2, 2007 Joy Mountford - Goal of Apple: design consistent interface for all users ’s: Spreadsheets, word processing - Why not include pictures, sounds, movies?  QuickTime - Designers design for themselves instead of for audience: Hand controller dilemma - Navigable movies (became QuickTime VR) -Object Maker: 3D object scanner: almost all viewpoints around objects - Goal of Apple: design consistent interface for all users ’s: Spreadsheets, word processing - Why not include pictures, sounds, movies?  QuickTime - Designers design for themselves instead of for audience: Hand controller dilemma - Navigable movies (became QuickTime VR) -Object Maker: 3D object scanner: almost all viewpoints around objects  

Monday, April 2, 2007 Joy Mountford - Beyond video: audio & music - Soundscapes Interval Research: Bead Box - Beyond video: audio & music - Soundscapes Interval Research: Bead Box

Monday, April 2, 2007 Bill Gaver - Designing sound: SonicFinder (Apple) - Auditory representations of real-world objects - Enhanced experience examples: - dragging a file - copying a file sounds like pouring water - Properties of a file determines it’s sound, for instance: size - Designing sound: SonicFinder (Apple) - Auditory representations of real-world objects - Enhanced experience examples: - dragging a file - copying a file sounds like pouring water - Properties of a file determines it’s sound, for instance: size

Monday, April 2, 2007 Equator project - History tablecloth

Monday, April 2, 2007 Equator project - Key table/picture frame (capturing emotions)

Monday, April 2, 2007 Equator project - Drift table