A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira Fischler Department of Psycholgy Paul Fishwick Department of CISE Benjamin Lok Department of CISE

Anesthesia Education Current Learning Process The Virtual Anesthesia Machine (VAM) – an abstract 2D simulation A real Anesthesia Machine – a physical simulation

Knowledge Types TypeAbstractConcrete Skill Set Concepts that can be applied to many real machine models Procedural, tactile, and psychomotor skills Example Invisible gas flowPerforming a procedure with a specific machine Transfer Knowledge?

Problem: Merging Knowledge Types Anesthesia machine trainees must mentally merge:  Abstract Concepts  Procedural Skills Mixed Reality (MR) can merge real and virtual spaces  Virtual space = Abstract  Real Space = Concrete Can MR’s Merging of spaces improve users’ merging of abstract and concrete knowledge?

Merging Abstract and Concrete Merge the abstract physical simulation with the physical device

The Augmented Anesthesia Machine Uses a magic lens approach to merge:  Abstract VAM simulation  Real Anesthesia Machine TUI

Overview The Augmented Anesthesia Machine The Augmented Learning Process User Study: VAM vs. AAM MR enables abstract + concrete knowledge merging

Previous Work Tangible User Interfaces  Two Main Parts: Computational Media Physical Interface  Interface is meaningful Representative of a part of the computational media As opposed to an abstract interface (i.e. a mouse)  AAM Computational media = VAM Physical Interface = real anesthesia machine [Ishii 1997, Ullmer 2000]

Previous Work Magic Lenses  2D to 3D  Applications in AR/MR AAM  Uses a magic lens to visually merge the abstract and concrete spaces [Bier 1993, Looser 2004]

Motivation 75% of Anesthesia Machine Related Accidents resulting in death or brain damage are due to user error.  Many anesthesia providers do not fully understand how the machine functions internally VAM was created to reduce this number

Motivation for the VAM Created by Dr. Samsun Lampotang Advantages of the VAM  visualize the invisible (i.e. gas flow)  Spatial layout simplified for ease of visualization  Web disseminated  Increases understanding of internal function [Fishcler et. al. 2006] Registered Users One Billion Hits per year ( Problems still exist in knowledge transfer from the VAM to the real machine

Motivation VAM to AAM: A Mapping Problem A A B B

MR: Enabling Multiple Represetations Multiple Representations improve overall comprehension of a concept.  VAM: Abstract Representation  Anesthesia Machine: Concrete Representation  AAM: Combined Representation Proposed Augmented Learning Process: Abstract Representation (VAM) Combined Representations (AAM) Concrete Representation (Anesthesia Machine)

The AAM Representation Tracked 6DOF magic lens shows overlaid VAM simulation Real machine interaction

Anesthesia Machine Tracking System 2D Optical Tracking with OpenCV  Infrared Markers for knobs  Infrared LEDs connected to buttons  Color tracking for gauges (bright red) 3 Unibrain webcams positioned around the machine in positions of minimal occlusion

Magic Lens Display HP TC1100  3.7 lbs  10” screen Not See through  Uses a 3D model of the machine, registered to the real machine  Easier to enable consistent registration  Less hardware than video see through

Magic Lens Tracking System Outside-Looking-in Optical Tracking Infrared Markers Specifications:  2 Unibrain Fire-I Webcams 640x480 at 30 fps  Tracking Volume: 3x3x3 m  Accuracy: 1cm  Jitter: 5mm  Latency: 70 ms

User Study Can AAM-Concrete help users to merge abstract and concrete knowledge? Between subjects study (n=20)  VAM Training Group  AAM Training Group

Population and Environment 20 Psychology Students  4 males, 16 females  Received class credit  Knew nothing about anesthesia machines Conducted in a quiet lab environment

Procedure Informed Consent Introduction Machine Functions 5 Training Exercises VAM AAM Spatial Cognition Tests Self Evaluation of Training Written Test Real Machine Intro VAM AAM Hands-on Fault Test Questionnaires Day 1 (90 min) Day 2 (60 min)

Metrics Efficiency of Training  Time to complete 5 training exercises Abstract Knowledge Acquisition  Written Anesthesia Machine Test Short Answer and muliple choice From the Anesthesia Patient Safety Foundation workbook Concrete Knowledge Acquistion  Hands-on fault test Faulty inspiratory valve

Fault Test Results Hypothesis 1: The AAM is more effective at teaching concrete concepts. AAM group found the faults significantly more often than VAM group Accept Hypothesis 1: The AAM trains concrete knowledge more effectively than the VAM Group# Participants Successful AAM6 out of 10 VAM1 out of 10

Written Test Results Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts There were no significant differences in Written test scores (p<0.21)

Training Time Results Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts VAM Group trained significantly faster (p<0.002) Accept Hypothesis 2: The VAM trains abstract knowledge more effectively than the AAM

Discussion Hypothesis 3: The AAM improves transfer to the real machine by enabling the merging of abstract and concrete knowledge. AAM group performed significantly better in the fault test To solve the fault test, participants had to merge:  Concrete knowledge: notice the inspiratory valve was missing  Abstract knowledge: understand the effect on the gas flow of the machine – harmful to the patient Accept Hypothesis 3: AAM helps to merge abstract and concrete knowledge

Conclusions leverage MR to merge simulation types  Combining Abstract simulation with the corresponding physical device  i.e. merging the VAM with the real machine MR enables multiple representations MR can help users to merge abstract and concrete knowledge  Improves training transfer into real world domains