Overview and Current Systems Augmented Reality Overview and Current Systems

Motivation: A Brief Scenario

Motivation: A Brief Scenario The Problems: The surgeon can’t look at the patient and the screen at the same time. Looking at the screen while working on the patient isn‘t ergonomic. The screen is 2D whereas the patient‘s body is 3D.

Motivation: A Brief Scenario The Vision The information should be provided where it is needed. The user should be enabled to access and interact with the information ergonomically. The information should be in 3 dimensions like the application’s subject.

Roadmap Defining Augmented Reality A Taxonomy of Reality and Virtuality Technologies Fields of Application Current Systems

Defining Augmented Reality Combining Real and Virtual Interactive in Real Time Registered in 3-Dimensions

Defining Augmented Reality: Combining Real and Virtual Attention: Combining does not neccesarily mean adding. Combining does not only include visual information (Navi-Project).

Defining Augmented Reality: Interactive in Real Time „Using AR technology, users can interact with a combination of real and virtual objects in a natural way.“ In „Confluence of Computer Vision and Interactive Graphics for Augmented Reality“

Definining Augmented Reality: Registered in 3-Dimensions Lightning reacts to the position of the tangible sun.

A Taxonomy of Reality and Virtuality The Reality-Virtuality Continuum

A Taxonomy of Reality and Virtuality                  Centricity: Exocentric or Egocentric?

A Taxonomy of Reality and Virtuality Control-Display Congruence Question: How intuitive is the control? In AR: When I move the real object to the right hand side, in what direction does the virtual one go? Conclusion: AR is situated near the real-role of the continuum. AR uses an egocentric viewpoint in most cases. AR tries to maximize the control-display congruence.

Technologies User Interfaces Visualization Tracking

User Interfaces Visual Acoustic Haptic/Tangible

User Interfaces: Visual Gesture Recognition Example: MIT‘s KidsRoom: Temporal integration

User Interfaces: Acustic 2 Parts Speech Synthesis Speech Recognition Odyssey 2001‘s dream has not come true, yet. So, Acoustic UIs are combined with other modalities. Example: SHEEP Insert!

User Interfaces: Haptic/Tangible Joystick, 3D-Mouse Tangible Objects Examples: CAR, SHEEP and the Studierstube‘s PIP Touch gloves

Visualization: Ways of Visualizing Visualization Devices HMDs PDAs Projection Monitors 3-Dimensionality Separeted displays Polarized projection Anaglyphic projection Time division

Visualization: Mixing Real and Virtual Optical see through Video see through Chroma keying: Mixing with video hardware Mixing in the Renderer What about occlusion? Tracking all real objects Depthmaps construction via stereo images Special Projection Techniques

Visualization: Challenges Contrast Brightness Resolution Field of View Safety

Tracking: Ways of Tracking Optical Tracking Markers IR-reflecting spheres Cards Types of optical Tracking Inside out Outside in

Tracking: Ways of Tracking Time of Flight GPS, DGPS, Galileo Ultrasound Magnetic Field Sensing Inertial Tracking Gyro Meters Accelerometer Special: Eye Tracking

Tracking: Challenges Static Errors Dynamic Errors Optical distortion Improper parameter configuration Mechanical misalignment Strategies Carefullness Algorithms for autocalibration Dynamic Errors Hardware related delays Software related delays Strategies Higher processing speed Video stream matching Rendering in advance Predicting the future

Fields of Application Annotating the Environment Displaying Auxiliary Sensor Information Visualizing Artificial Data

Annotating the Environment Navigation Columbia‘s „Touring Machine“ Police and Military BARS „… the success of a military operation in an urbanized environment depends crucially on being able to provide navigation and coordination information to the individual marine level.“ The US-Army's Concepts Division of the Marine Corps Combat Development Command

Annotating the Environment Repair, Maintenance and Production BMW: The „Intelligent Welding Gun“ Boeing: „Augmented wiring“

Displaying Auxiliary Sensor Information Medical University of North Carolina: Ultrasound images of a fetus. The heARt-Project: Minimal invasive heart surgery. MIT: MRI and EEG to support neurosurgery. Successfully used

Visualizing Artificial Data Design Studierstube‘s Virtual Showcase

Visualizing Artificial Data Scientific Visualization Education Studierstube: 3D-Construct Application

Current Systems Libraries ARToolKit Frameworks Studierstube DWARF

Current Systems DWARF a Sample Framework DWARF, a network of distributed dynamically cooperating services. Distribution Heterogenity Modularity

Current Systems DWARF a Sample Framework Services Needs Abilities Service Descriptions Middleware Service Manager Visual Frontend

Current Systems DWARF a Sample Framework Subsystems Tracking Presentation Interaction Context World Model Application

The Future HMDs in the near future? AR in sports and television

Большое спасибо за Ваше внимание. Vielen Dank für Ihre Aufmerksamkeit! Thank you very much for paying attention.

Discussion Questions? Excursus CAR, SHEEP and Co. as motion pictures. A close look at DWARF: The Source Code of a Service. What does it cost? A sample HMD and a sample Tracking System.