AR Dance and Engineering in Augmented Reality Presented by: Michael Korostelev

the Computer Fusion Laboratory Director: Dr. Li Bai Exchange scholar Dr. Guohua Huang Students: James Ren Michael Korostelev Michael Huang Fengcheng Wu David Osinski CFL Lab

Project Showcase `` ````` `````````` Septa Electronic Payment Project 2004 – now ($150K + $1.0M) Dance and Engineering in Augmented Reality 2010 – now ($50K) Wireless Distributed Operating System 2007 – now ($400K) Wind-Solar Hybrid LED Lighting System now Sensing Technology for Import Food Inspection now Wireless Industrial Process Monitoring 2008 – now ($200K) Project Showcase

Art and Engineering Purely creative

Art and Engineering Entertainment

Art and Engineering Practical and profitable

Art and Engineering Educational

CFL Lab Artist Visions Film Festival Visual Treasure Hunt Visitors given clues about the town of Lambertville NJ They explore the town to find QR codes Scan QR codes and attach their own message Visitors’ input becomes collaborative story

Art and Engineering These projects bring engineers close to artists and act as a bridge between the disciplines Each having a goal to relieve the apprehension laymen have to technology Develop tools for creative people, rather than develop creative works

Motivation Create a medium for rich creative works Inspire ideas from artists and engineers alike Enhance education and increase outreach Establish framework for fundamental research

Project Goal Enhance visual experience of a live performance with AR Visually enhance storytelling Engage and involve audience Convey complex ideas Reduce production costs Unbind creative limitations

What Is AR ? Display Techniques Head Mounted Displays Spatial Displays Handheld Display

Dance and Engineering in Augmented Reality Camera/Projector InstallationAugmented Stage High frame rate digital capture High lumen projector with wide focal range PC with accelerated graphics Marker less stationary objects IR led marked moving objects Volumetric objects Dynamic lighting Floor projection

2D/3D Projector-Camera Calibration

Homography (perspective transformation) Stereo Calibration

Kinect “Hacks”

RGBD Sensor Based Calibration Calibrate RGBD sensor Uncalibrated overlayDepth and Image camera separated physically so images do not match Two different approaches to create image overlay - Homography - Stereo Calibration

Projector – Sensor Extrinsics Real Scene Sensor Capture Projection

Determining Projection Matrix Determining the projection matrix

Determining Projection Matrix

Displaying the scene

Displaying the Scene