3D Laser Stripe Scanner or “A Really Poor Man’s DeltaSphere” Chad Hantak December 6, 2004

Overview Introduction Acquisition Device Calibration Software Framework Processing Limitations Future Work

Introduction or “What’s the project about?” 3D Scanning System Similar to DeltaSphere Range Samples from fixed COP Acquire sparse depth with color Acquisition Device Cheap Simple to operate

DeltaSphere Dense Depth Time of Flight Acquisition Collects Range Samples over 360 degrees of Azimuth and 135 degrees of elevation Cons Slow Scanning Processing Expensive

Acquisition Device Components Laser Level Available at hardware stores Used for leveling (pictures, shelves, …) Emits a plane of laser light (Vertical or Horizontal) Camcorder Canon GL2 (3 CCD) Little overkill for this

Acquisition Device Construction Camcorder rotates about fixed point Laser Level is fixed next to camcorder

Depth Range Acquisition “The Theory” Calibrated Rig Know laser stripe’s plane equation in camera’s coordinate system During Rotation Determine amount of rotation Find laser’s stripe points in image Determine where image rays intersect stripe plane Yields depth samples At same time acquire color

Calibration Need to know laser stripe plane in camera coordinate system Record calibration grid with laser stripe activated

Bouget’s Matlab Toolkit Camera Intrinsics Standard Bouget’s Toolkit Extras from toolkit Calibration plane in camera’s coordinate system for each Image

Laser Stripe’s Plane For each image Find stripe points (user clicks) Determine world rays Intersect with grid’s plane for world points Take all the world points and fit them to a plane (least squares) Result is laser stripe plane

Software Framework Implemented in C++ Libraries OpenCV Image Processing DirectShow Interface to Camcorder y/en-us/directshow/htm/introductiontodirectshow.asphttp://msdn.microsoft.com/library/default.asp?url=/librar y/en-us/directshow/htm/introductiontodirectshow.asp

Two Executables depthPanorama Hook into camcorder Acquires / processes frames Saves into shared memory panoramaView Reads from shared memory Displays result to user

depthPanorama 3 Systems RotationTracker Estimates the amount of rotation ColorProcessor Creates the color panorama DepthProcessor Creates the depth panorama

RotationTracker Estimates the amount of rotation between two frames Uses Lucas-Kanade Point Tracking Estimate rotation if “enough” points moved Uses RANSAC to estimate the amount of rotation

ColorProcessor Updates the Color Panorama Process Only update after “enough” rotation Extract and resize left half of frame into panorama’s memory Right half of frame contains the laser stripe

Depth Processor Updates the Depth Panorama Process Find the laser stripe points in the frame (samples every x scan lines) Turn image points into world rays Intersect rays with laser stripe plane For valid intersections, place depth value in depth panorama For “some” invalid intersections, attempt to interpolate

Finding Laser Stripe Image Points or “The Hard Part” Know in the frame where points are (right half) Extension to ModelCamera system Process scan line looking for candidate peaks with symmetry Exploit fact camcorder is just rotating Determine homography from one frame to the next, warp previous frame, subtract from current frame Homography is based on amount of rotation between frames Since laser stripe moves differently it will stand out better in resultant image There is still noise in the image Need some image processing Edge Detection, Erosion, Dilation

Turing Image Points into Depth Samples Know the camcorder intrinsics Allows image points to be turned into rays into world from camcorder’s COP Intersect these rays with plane Splat depth to neighboring depth samples Associate a confidence level with each depth sample Perfect; confidence = 1.0 Interpolated; confidence = 0.75 Splatted; confidence = 0.6 When updating if incoming confidence higher, use it

No Intersection Due to noise, image points of stripe may not be correct World ray may not intersect stripe plane If neighboring samples are valid, interpolate missing depth value Lower confidence

panoramaView Simply reads values sent by depthPanorama Displays texture-mapped quads of depth values Updates continually from shared memory

Results

Current Problems Rotation tracking Off by a few degrees at end of 360 degree pan (< 10 degrees) Tweaking LK Tracking parameters Smooth out the rotation (use LK & correlation to determine when rotation complete) Stripe point determination Adjusting image processing parameters Depth discontinuities Take them into account when rendering

Future Work Fix current problems Better laser light / Point detection Stronger at distance Want longer range scans Combine different scans Introduces new slew of problems

References R. Laganiere, “Programming computer vision applications: A step-by-step guide to the use of the Intel OpenCV library and the Microsoft DirectShow technology”, V. Popescu, E. Sacks and G. Bahmutov, “Interactive Modeling from Dense Color and Sparse Depth”, 3DPTV, S. Sinha and M. Pollefeys, “Towards Calibrating a Pan-Tilt-Zoom Camera Network”, 5th Workshop on Omnidirectional Vision, Camera Networks, and Non-classical Cameras (OMNIVIS), May 16, 2004.