Interactive Systems Technical Design Seminar work: Augmented reality Juha-Matti Malila Matti Yli-Olli

Introduction Definition An augmented reality system generates a composite view for the user. It is a combination of the real scene viewed by the user and a virtual reality scene generated by the computer that augments the scene with additional information.

Motivation Augmented Reality enhances a user's perception of and interaction with the real world The virtual objects display information that the user cannot directly detect with his own senses. The information conveyed by the virtual objects helps a user perform real-world tasks

Implementation Generic components of AR device Tracking Graphics Display

Implementation Example: requirements of a mobile AR system Head-mounted display with a resolution of at least 800x600 pixels Ability to digitize at least one video stream with 256 colors at a frame rate of 256 pics/second Network connection with a minimum transfer speed of 100 kbit/s System architecture has to be modular and scalable The speed of the system bus has to be at least 19,5 MB/s Self-sustained in energy and small enough to be carried along Includes position and orientation detection

Applications Medical visualization Maintenance and repair Entertainment Military applications Navigation

Applications Image guided surgery Collaboration between MIT AI lab and Brigham and Women’s Surgical Planning lab Will enable surgeons to visualize internal structures through an automated overlay of 3D reconstructions of internal anatomy on top of live video views of a patient Construct 3D models using scans such as MRI (Magnetic Resonance Imaging) and CT (Computed Tomography Imaging) Segment data to distinguish between tissue types and surface render the data

Image guided surgery Set up in operating room consists of: Portable cart containing a Sun UltraSPARC Hardware to drive the laser scanner Flashpoint tracking system

Image guided surgery Laser scanner used to collect 3D data of patient’s scalp surface as positioned on operating table to register the patient to the segmented MRI skin Enhances visualization because the surgeon has “x-ray vision” Medical instruments are tracked

Strengths / Advantages Used for over 200 neurosurgical cases Used 1-2 times/week Achieves high positional accuracy Reduces operating time from eight hours to five Monetary savings are huge Makes surgeons more confident

Limitations / Weaknesses Still need a human to operate Expensive

Selected Industrial Players SporTVision – AR applications for sport TV Microvision – NOMAD NOMAD Videos: http://www.microvision.com/vid_nmd_testim_index.htm

Selected International Research Groups and Projects National University of Singapore – Mixed Reality Lab Research projects Empodied Mixed Reality Interaction Wearable Computer Interaction Natural Feature Tracking Human Electronics 3D Live Head Mounted Display 3D Video Table - Video Clip

Selected Finnish Research Groups and Projects VTT Information Technology – Multimedia Group VIAPOR (Video Image Augmenting, Processing and Object Recognition) http://mango2.vtt.fi:84/tte/samba/project/tte32/index.html Research topics: Entertainment applications Gesture interaction and CAD conferencing Displays and trackers Object recognition methods VTT Video Clip

Companies and Research Groups in Oulu CyPhone – Media Phone Prototype Outside navigation and displaying device Uses GPS, GSM and WAP Cyphone user case

Future Developments AR products become mass-products Mobile AR systems - CyPhone Hologramms in few years Straight connection to human brains in few decades