1. 3D Laser Stripe Scanner or “A Really Poor Man’s DeltaSphere” Chad Hantak December 6, 2004

2. Overview Introduction Acquisition Device Calibration Software Framework Processing Limitations Future Work

3. Introduction or “What’s the project about?” 3D Scanning System Similar to DeltaSphere Range Samples from fixed COP Acquire sparse depth with color Acquisition Device Cheap Simple to operate

4. DeltaSphere Dense Depth Time of Flight Acquisition Collects Range Samples over 360 degrees of Azimuth and 135 degrees of elevation Cons Slow Scanning Processing Expensive

5. Acquisition Device Components Laser Level Available at hardware stores Used for leveling (pictures, shelves, …) Emits a plane of laser light (Vertical or Horizontal) Camcorder Canon GL2 (3 CCD) Little overkill for this

6. Acquisition Device Construction Camcorder rotates about fixed point Laser Level is fixed next to camcorder

7. Depth Range Acquisition “The Theory” Calibrated Rig Know laser stripe’s plane equation in camera’s coordinate system During Rotation Determine amount of rotation Find laser’s stripe points in image Determine where image rays intersect stripe plane Yields depth samples At same time acquire color

8. Calibration Need to know laser stripe plane in camera coordinate system Record calibration grid with laser stripe activated

9. Bouget’s Matlab Toolkit Camera Intrinsics Standard Bouget’s Toolkit Extras from toolkit Calibration plane in camera’s coordinate system for each Image

10. Laser Stripe’s Plane For each image Find stripe points (user clicks) Determine world rays Intersect with grid’s plane for world points Take all the world points and fit them to a plane (least squares) Result is laser stripe plane

11. Software Framework Implemented in C++ Libraries OpenCV Image Processing http://www.intel.com/research/mrl/research/opencv/ DirectShow Interface to Camcorder http://msdn.microsoft.com/library/default.asp?url=/librar y/en-us/directshow/htm/introductiontodirectshow.asphttp://msdn.microsoft.com/library/default.asp?url=/librar y/en-us/directshow/htm/introductiontodirectshow.asp

12. Two Executables depthPanorama Hook into camcorder Acquires / processes frames Saves into shared memory panoramaView Reads from shared memory Displays result to user

13. depthPanorama 3 Systems RotationTracker Estimates the amount of rotation ColorProcessor Creates the color panorama DepthProcessor Creates the depth panorama

14. RotationTracker Estimates the amount of rotation between two frames Uses Lucas-Kanade Point Tracking Estimate rotation if “enough” points moved Uses RANSAC to estimate the amount of rotation

15. ColorProcessor Updates the Color Panorama Process Only update after “enough” rotation Extract and resize left half of frame into panorama’s memory Right half of frame contains the laser stripe

16. Depth Processor Updates the Depth Panorama Process Find the laser stripe points in the frame (samples every x scan lines) Turn image points into world rays Intersect rays with laser stripe plane For valid intersections, place depth value in depth panorama For “some” invalid intersections, attempt to interpolate

17. Finding Laser Stripe Image Points or “The Hard Part” Know in the frame where points are (right half) Extension to ModelCamera system Process scan line looking for candidate peaks with symmetry Exploit fact camcorder is just rotating Determine homography from one frame to the next, warp previous frame, subtract from current frame Homography is based on amount of rotation between frames Since laser stripe moves differently it will stand out better in resultant image There is still noise in the image Need some image processing Edge Detection, Erosion, Dilation

18. Turing Image Points into Depth Samples Know the camcorder intrinsics Allows image points to be turned into rays into world from camcorder’s COP Intersect these rays with plane Splat depth to neighboring depth samples Associate a confidence level with each depth sample Perfect; confidence = 1.0 Interpolated; confidence = 0.75 Splatted; confidence = 0.6 When updating if incoming confidence higher, use it

19. No Intersection Due to noise, image points of stripe may not be correct World ray may not intersect stripe plane If neighboring samples are valid, interpolate missing depth value Lower confidence

20. panoramaView Simply reads values sent by depthPanorama Displays texture-mapped quads of depth values Updates continually from shared memory

21. Results

22. Current Problems Rotation tracking Off by a few degrees at end of 360 degree pan (< 10 degrees) Tweaking LK Tracking parameters Smooth out the rotation (use LK & correlation to determine when rotation complete) Stripe point determination Adjusting image processing parameters Depth discontinuities Take them into account when rendering

23. Future Work Fix current problems Better laser light / Point detection Stronger at distance Want longer range scans Combine different scans Introduces new slew of problems

24. References R. Laganiere, “Programming computer vision applications: A step-by-step guide to the use of the Intel OpenCV library and the Microsoft DirectShow technology”, http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/. V. Popescu, E. Sacks and G. Bahmutov, “Interactive Modeling from Dense Color and Sparse Depth”, 3DPTV, 2004. S. Sinha and M. Pollefeys, “Towards Calibrating a Pan-Tilt-Zoom Camera Network”, 5th Workshop on Omnidirectional Vision, Camera Networks, and Non-classical Cameras (OMNIVIS), May 16, 2004.