© 2007 Masahiko INAMI The University of Electro-Communications 1 Masahiko Inami Department of Mechanical Engineering and Intelligent Systems The University of Electro-Communications (UEC Tokyo) Beyond Visual Augmentation (Spatial Augmented Reality)

© 2007 Masahiko INAMI The University of Electro-Communications 2 Augmented Reality Real head + CT image AR is a technology which deals with the combination of physical and artificial stimulus. Real world Virtual world + + AR world

© 2007 Masahiko INAMI The University of Electro-Communications 3 Augmented (visual) Reality

© 2007 Masahiko INAMI The University of Electro-Communications 4 Beyond Visual Augmentation VR ≠ Visual Reality We have “five” senses. –Sight –Hearing –Taste –Smell –Touch

© 2007 Masahiko INAMI The University of Electro-Communications 5 Introduction: Examples of Multi-modal AR Touching Chemical Boundaries Unconscious Avoidance

© 2007 Masahiko INAMI The University of Electro-Communications 6 Visuo-Haptic Augmentation

© 2007 Masahiko INAMI The University of Electro-Communications 7 Head-Mounted Projector (HMP) for Retroreflective Projection Technology (RPT), Masahiko Inami (UEC) Naoki Kawakami (Univ. of Tokyo) Susumu Tachi (Univ. of Tokyo)

© 2007 Masahiko INAMI The University of Electro-Communications 8 Optical Camouflage(1999)

© 2007 Masahiko INAMI The University of Electro-Communications 9 Visuo-Haptic display using RPT

© 2007 Masahiko INAMI The University of Electro-Communications 10 Off On TelesarPHONE (TELESAR II) S. Tachi, et al. (Univ. of Tokyo 2005) Humanoid as a Visuo-Haptic Display

© 2007 Masahiko INAMI The University of Electro-Communications 11 TelesarPhone (2005) Susumu Tachi et al. (Univ. of Tokyo)

© 2007 Masahiko INAMI The University of Electro-Communications 12 Haptic and Tactile Augmentation

© 2007 Masahiko INAMI The University of Electro-Communications 13 SmartTools(2001) Masahiko Inami (UEC) Takuya Nojima (Univ. of Tokyo) Susumu Tachi (Univ. of Tokyo) Sensor Tool Force Display User Subject

© 2007 Masahiko INAMI The University of Electro-Communications 14 Sensors: –Using real time sensors –Installed at the working area of the tool A tool and a haptic display –Using a conventional tool –Free space around the working area of the tool Design of the SmartTool The user holds this tool We need free space around the tool

© 2007 Masahiko INAMI The University of Electro-Communications 15 SmartTool

© 2007 Masahiko INAMI The University of Electro-Communications 16 Virtual Jig Tool Normal TaskWith Augmented Haptics Virtual ruler SmartTool ruler

© 2007 Masahiko INAMI The University of Electro-Communications 17 Normal mode

© 2007 Masahiko INAMI The University of Electro-Communications 18 Smart mode

© 2007 Masahiko INAMI The University of Electro-Communications 19 SmartFinger(2002) Force Sensor Actuator Masahiko Inami (UEC) Hideyuki Ando (NTT) Taro Maeda (Osaka Univ.) Susumu Tachi (Univ. of Tokyo)

© 2007 Masahiko INAMI The University of Electro-Communications 20 Masahiko Inami (UEC) Hideyuki Ando (NTT) Taro Maeda (Osaka Univ.) Susumu Tachi (Univ. of Tokyo)

© 2007 Masahiko INAMI The University of Electro-Communications 21 SmartTouch(2003) Hiroyuki Kajimoto (UEC) Masahiko Inami (UEC) Naoki Kawakami (Univ. of Tokyo) Susumu Tachi (Univ. of Tokyo)

© 2007 Masahiko INAMI The University of Electro-Communications 22 Forehead Retina System Hiroyuki Kajimoto (UEC) Yonezo Kanno (EyePlusPlus.Inc.) Susumu Tachi (Univ. of Tokyo) 2006

© 2007 Masahiko INAMI The University of Electro-Communications 23 Goal: Forehead Retina System

© 2007 Masahiko INAMI The University of Electro-Communications 24 System structure Intel Xscale 400MHz, 4MB Embedded Linux Renesas H8 30MHz & Driver Circuit Sunglass Camera NTSC, 40  FOV.

© 2007 Masahiko INAMI The University of Electro-Communications 25 Olfactory Augmentation Wearable olfactory display by Tanikawa, Hirose et al. SpotScents Yasuyuki Yanagida(Meijo Univ.), Haruo Noma(ATR) et al.

© 2007 Masahiko INAMI The University of Electro-Communications 26 SPOTSCENTS Track the nose position Deliver the scents from some nearby place: “ free space instead of tubes ” Yanagida IEEE VR2004

© 2007 Masahiko INAMI The University of Electro-Communications 27 Launching a Scented Vortex Ring Inject scented air into the cylinder instead of body of air cannon Scented air in the cylinder completely launched by the shot Prevent air mixture and back-flow by shutters

© 2007 Masahiko INAMI The University of Electro-Communications 28 SPOTSCENTS - Basic Concept - Generate a small “Spot” olfactory field in free space. Provide spatio-temporal control of the olfactory field.

© 2007 Masahiko INAMI The University of Electro-Communications 29 Video – Breaking vortex rings

© 2007 Masahiko INAMI The University of Electro-Communications 30 Vestibular Augmentation Galvanic Vestibular Stimulation(GVS) can produce a virtual perception of acceleration. V-acceleration toward the anode by DC (~ 2mA) Electrodes placed on the mastoids behind the ears.

© 2007 Masahiko INAMI The University of Electro-Communications 31 Galvanic Vestibular Stimulation (Shaking the world) Taro Maeda (Osaka Univ.) Hideyuki Ando (NTT) Maki Sugimoto (UEC) Masahiko Inami (UEC)

© 2007 Masahiko INAMI The University of Electro-Communications 32 Demonstration at Sig05 Etech

© 2007 Masahiko INAMI The University of Electro-Communications 33 Coming Up Next Part A: Course Introduction and Overview (Raskar) Part B: Introduction to current Approaches (Bimber) Part C: New Directions in Augmentation (Raskar) Part D: Beyond Visual Augmentation (Inami) Part E: Augmentation on Everyday Surfaces(Bimber) Part F: Prototypes and Experiences (all) Q&A