WIEN Building the Augmented Classroom Components for everyday mobile collaborative AR

Flo Ledermann WIEN Components for building the Augmented Classroom - 2/21 The Augmented Classroom - Overview n Spontaneous collaboration of mobile users n Optical Tracking, use of markers as tangible user interfaces

Flo Ledermann WIEN Components for building the Augmented Classroom - 3/21 Standard User Equipment n Mobile AR System u Backpack with CPU & connectors u Optical See-through HMD u Head mounted camera

Flo Ledermann WIEN Components for building the Augmented Classroom - 4/21 User Equipment Enhancements n Pen is replaced by „Pinch Gloves“ n PIP is mounted to the wrist of the left hand n -> Both Hands are free to interact with the real world

Flo Ledermann WIEN Components for building the Augmented Classroom - 5/21 User Equipment Details (1/6) Pinch Gloves: Evolution n First Generation: Markers sewed onto glove

Flo Ledermann WIEN Components for building the Augmented Classroom - 6/21 User Equipment Details (2/6) Pinch Gloves: Evolution n Second Generation: Rigid structure carrying markers

Flo Ledermann WIEN Components for building the Augmented Classroom - 7/21 User Equipment Details (3/6) Pinch Gloves: Evolution n Second Generation: Rigid structure carrying markers Hypothesis: Constant offset in space

Flo Ledermann WIEN Components for building the Augmented Classroom - 8/21 User Equipment Details (4/6) Pinch Gloves: Evolution n Third Generation: u Finger-free u Cables covered u Only one button n Lightweight Version u 3 Buttons on thumb u Markers attached seperately

Flo Ledermann WIEN Components for building the Augmented Classroom - 9/21 User Equipment Details (5/6) Modified PIP n Wrist-attached to keep hands free n Due to inacurate optical tracking, a Track-pad is used for 2D input.

Flo Ledermann WIEN Components for building the Augmented Classroom - 10/21 User Equipment Details (6/6) Headset n Uses new Firefly FW camera with 4mm lens. n Nearly 90° FOV n Tracking area bigger than HMD FOV

Flo Ledermann WIEN Components for building the Augmented Classroom - 11/21 Table for overhead tracking n Lightweight table with camera for overhead tracking

Flo Ledermann WIEN Components for building the Augmented Classroom - 12/21 Software Enhancements n Improvements in usability introduced by u One-shot quick calibration for HMD & gloves u Shared tracking using multiple cameras

Flo Ledermann WIEN Components for building the Augmented Classroom - 13/21 Software Details (1/9) HMD Calibration n Optical tracking must be calibrated in ARToolkit n Remaining problem: offset between camera coordinate system and eye coordinate system. ECS CCS

Flo Ledermann WIEN Components for building the Augmented Classroom - 14/21 Software Details (2/9) HMD Calibration n SoCameraCalibrationKit: Calibration Marker Tracked position in overlay image Calibration reference Align marker with calibration reference

Flo Ledermann WIEN Components for building the Augmented Classroom - 15/21 Software Details (3/9) HMD Calibration n SoCameraCalibrationKit: Press Button -> Offset is calculated and written to OT node for camera offset Marker and tracked position are now aligned correctly.

Flo Ledermann WIEN Components for building the Augmented Classroom - 16/21 Software Details (4/9) Glove Calibration n A Similar tool is provided to calibrate the „Pinch Point“ of the gloves n The offset varies for every user, and can be configured with a single click

Flo Ledermann WIEN Components for building the Augmented Classroom - 17/21 Software Details (5/9) Shared Tracking n Optical tracking fails: u Marker (partially) out of FOV of camera u Marker occluded u Marker oriented away from camera u Marker not recognized

Flo Ledermann WIEN Components for building the Augmented Classroom - 18/21 Software Details (6/9) Shared Tracking n Solution / Improvement: u Use several cameras, sharing their tracking data over the network u Dynamically determine reference markers (seen by 2 or more cameras) u Calculate position of hidden marker from reference + remote coordinates

Flo Ledermann WIEN Components for building the Augmented Classroom - 19/21 Software Details (7/9) Shared Tracking n Opentracker implementation (1/2):

Flo Ledermann WIEN Components for building the Augmented Classroom - 20/21 Software Details (8/9) Shared Tracking n Opentracker implementation (2/2):

Flo Ledermann WIEN Components for building the Augmented Classroom - 21/21 Software Details (9/9) Shared Tracking n Results:

Flo Ledermann WIEN Components for building the Augmented Classroom - 22/21 Thank you!