1. Monday, April 2, 2007 Multisensory & Multimedia Multisensory & Multimedia

2. Monday, April 2, 2007 5 senses: - Vision, hearing, touch, smell & taste - Connection with computers “sensory deprived and physically limited” - Multimedia experience: multisensory Interviews: - Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT) - Durrell Bishop (Teacher of Product Design @ Royal College of Art in London) - Joy Mountford (Member of Human Interface Group @ Apple) - Bill Gaver (Professor of Interaction Research @ Royal College of Art in London ) 5 senses: - Vision, hearing, touch, smell & taste - Connection with computers “sensory deprived and physically limited” - Multimedia experience: multisensory Interviews: - Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT) - Durrell Bishop (Teacher of Product Design @ Royal College of Art in London) - Joy Mountford (Member of Human Interface Group @ Apple) - Bill Gaver (Professor of Interaction Research @ Royal College of Art in London ) Overview

3. Monday, April 2, 2007 - Interaction requires 2 key components: Controls & representation of results - Graphical UI: representation  pixels on screen control  remote: mouse, keyboard,… -Tangible UI: representation & control: tangible & coupled - GUI  TUI: Painted bits  tainted bits by giving physical form to digital information - Multisensory multimedia interactions: fun & help us do things better - Interaction requires 2 key components: Controls & representation of results - Graphical UI: representation  pixels on screen control  remote: mouse, keyboard,… -Tangible UI: representation & control: tangible & coupled - GUI  TUI: Painted bits  tainted bits by giving physical form to digital information - Multisensory multimedia interactions: fun & help us do things better Hiroshi Ishii

4. Monday, April 2, 2007 Tangible User Interfaces: example “The reacTable, is a state-of-the-art multi-user electro- acoustic music instrument with a tabletop tangible user interface.”

5. Monday, April 2, 2007 Tangible User Interfaces - http://tangible.media.mit.edu/projects/ [open][open] - Hiroshi Ishii’s latest project: Jabberstamp

6. Monday, April 2, 2007 Tangible User Interfaces - Control & representation of results: in a more physical form compared to GUI’s - Directly grab & manipulate results - Abacus: feel, manipulate, touch information - Control & representation of results: in a more physical form compared to GUI’s - Directly grab & manipulate results - Abacus: feel, manipulate, touch information

7. Monday, April 2, 2007 Tangible User Interfaces - musicBottles: - weather forecast bottle

8. Monday, April 2, 2007 Durrell Bishop - “Things should be themselves”: physical or virtual objects should be self-evident - Consumer electronics: universally black boxes of standard size - Labels say what a button does: merely an action on a word - Monopoly vs. VCR - “Things should be themselves”: physical or virtual objects should be self-evident - Consumer electronics: universally black boxes of standard size - Labels say what a button does: merely an action on a word - Monopoly vs. VCR

9. Monday, April 2, 2007 Durrell Bishop - TUI: Marble answering machine

10. Monday, April 2, 2007 Durrell Bishop - Picking up tagged objects  physical pointers

11. Monday, April 2, 2007 Joy Mountford - Goal of HIG @ Apple: design consistent interface for all users - 1980’s: Spreadsheets, word processing - Why not include pictures, sounds, movies?  QuickTime - Designers design for themselves instead of for audience: Hand controller dilemma - Navigable movies (became QuickTime VR) -Object Maker: 3D object scanner: almost all viewpoints around objects - Goal of HIG @ Apple: design consistent interface for all users - 1980’s: Spreadsheets, word processing - Why not include pictures, sounds, movies?  QuickTime - Designers design for themselves instead of for audience: Hand controller dilemma - Navigable movies (became QuickTime VR) -Object Maker: 3D object scanner: almost all viewpoints around objects  

12. Monday, April 2, 2007 Joy Mountford - Beyond video: audio & music - Soundscapes team @ Interval Research: Bead Box - Beyond video: audio & music - Soundscapes team @ Interval Research: Bead Box

13. Monday, April 2, 2007 Bill Gaver - Designing sound: SonicFinder (Apple) - Auditory representations of real-world objects - Enhanced experience examples: - dragging a file - copying a file sounds like pouring water - Properties of a file determines it’s sound, for instance: size - Designing sound: SonicFinder (Apple) - Auditory representations of real-world objects - Enhanced experience examples: - dragging a file - copying a file sounds like pouring water - Properties of a file determines it’s sound, for instance: size

14. Monday, April 2, 2007 Equator project - History tablecloth

15. Monday, April 2, 2007 Equator project - Key table/picture frame (capturing emotions)

16. Monday, April 2, 2007 Equator project - Drift table