1 Intelligent Robotics Research Centre (IRRC) Department of Electrical and Computer Systems Engineering Monash University, Australia Visual Perception.

Slides:



Advertisements
Similar presentations
DEPTH FROM DISTORTION: PERFORMING 3D RECONSTRUCTION USING A CURVED MIRROR AND SINGLE CAMERA DREXEL UNIVERSITY DEARTMENT OF MATHEMATICS ASST. PROF. ANDREW.
Advertisements

System Integration and Experimental Results Intelligent Robotics Research Centre (IRRC) Department of Electrical and Computer Systems Engineering Monash.
Stereo Many slides adapted from Steve Seitz. Binocular stereo Given a calibrated binocular stereo pair, fuse it to produce a depth image Where does the.
MASKS © 2004 Invitation to 3D vision Lecture 7 Step-by-Step Model Buidling.
Computer vision: models, learning and inference
Hybrid Position-Based Visual Servoing
Where has all the data gone? In a complex system such as Metalman, the interaction of various components can generate unwanted dynamics such as dead time.
Intelligent Systems Lab. Extrinsic Self Calibration of a Camera and a 3D Laser Range Finder from Natural Scenes Davide Scaramuzza, Ahad Harati, and Roland.
Stereo.
December 5, 2013Computer Vision Lecture 20: Hidden Markov Models/Depth 1 Stereo Vision Due to the limited resolution of images, increasing the baseline.
A Versatile Depalletizer of Boxes Based on Range Imagery Dimitrios Katsoulas*, Lothar Bergen*, Lambis Tassakos** *University of Freiburg **Inos Automation-software.
Laser Scan Matching in Polar Coordinates with Application to SLAM
Last Time Pinhole camera model, projection
Monash University Dept Research Forum Active Sensing for Mobile and Humanoid Robots - Lindsay Kleeman Active Sensing for Mobile and Humanoid Robots.
SIGGRAPH Course 30: Performance-Driven Facial Animation Section: Markerless Face Capture and Automatic Model Construction Part 2: Li Zhang, Columbia University.
CS6670: Computer Vision Noah Snavely Lecture 17: Stereo
Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2005 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.
Stereoscopic Light Stripe Scanning: Interference Rejection, Error Minimization and Calibration By: Geoffrey Taylor Lindsay Kleeman Presented by: Ali Agha.
Epipolar geometry. (i)Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point.
Srikumar Ramalingam Department of Computer Science University of California, Santa Cruz 3D Reconstruction from a Pair of Images.
Structure from motion. Multiple-view geometry questions Scene geometry (structure): Given 2D point matches in two or more images, where are the corresponding.
Uncalibrated Geometry & Stratification Sastry and Yang
Vision Computing An Introduction. Visual Perception Sight is our most impressive sense. It gives us, without conscious effort, detailed information about.
Multi-view stereo Many slides adapted from S. Seitz.
3D from multiple views : Rendering and Image Processing Alexei Efros …with a lot of slides stolen from Steve Seitz and Jianbo Shi.
COMP 290 Computer Vision - Spring Motion II - Estimation of Motion field / 3-D construction from motion Yongjik Kim.
3D Rigid/Nonrigid RegistrationRegistration 1)Known features, correspondences, transformation model – feature basedfeature based 2)Specific motion type,
CSE473/573 – Stereo Correspondence
Stereo Guest Lecture by Li Zhang
Advanced Computer Vision Structure from Motion. Geometric structure-from-motion problem: using image matches to estimate: The 3D positions of the corresponding.
Project 1 artifact winners Project 2 questions Project 2 extra signup slots –Can take a second slot if you’d like Announcements.
Multiple View Geometry : Computational Photography Alexei Efros, CMU, Fall 2006 © Martin Quinn …with a lot of slides stolen from Steve Seitz and.
Image Based Rendering And Modeling Techniques And Their Applications Jiao-ying Shi State Key laboratory of Computer Aided Design and Graphics Zhejiang.
Stereo vision A brief introduction Máté István MSc Informatics.
Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #15.
Computer Vision Spring ,-685 Instructor: S. Narasimhan WH 5409 T-R 10:30am – 11:50am Lecture #15.
Mohammed Rizwan Adil, Chidambaram Alagappan., and Swathi Dumpala Basaveswara.
Lecture 11 Stereo Reconstruction I Lecture 11 Stereo Reconstruction I Mata kuliah: T Computer Vision Tahun: 2010.
Announcements Project 1 artifact winners Project 2 questions
Lecture 12 Stereo Reconstruction II Lecture 12 Stereo Reconstruction II Mata kuliah: T Computer Vision Tahun: 2010.
Automatic Registration of Color Images to 3D Geometry Computer Graphics International 2009 Yunzhen Li and Kok-Lim Low School of Computing National University.
December 4, 2014Computer Vision Lecture 22: Depth 1 Stereo Vision Comparing the similar triangles PMC l and p l LC l, we get: Similarly, for PNC r and.
Stereo Many slides adapted from Steve Seitz.
A General-Purpose Platform for 3-D Reconstruction from Sequence of Images Ahmed Eid, Sherif Rashad, and Aly Farag Computer Vision and Image Processing.
Lec 22: Stereo CS4670 / 5670: Computer Vision Kavita Bala.
Computer Vision Lecture #10 Hossam Abdelmunim 1 & Aly A. Farag 2 1 Computer & Systems Engineering Department, Ain Shams University, Cairo, Egypt 2 Electerical.
CSE 185 Introduction to Computer Vision Stereo. Taken at the same time or sequential in time stereo vision structure from motion optical flow Multiple.
Chapter 5 Multi-Cue 3D Model- Based Object Tracking Geoffrey Taylor Lindsay Kleeman Intelligent Robotics Research Centre (IRRC) Department of Electrical.
3D Object Modelling and Classification Intelligent Robotics Research Centre (IRRC) Department of Electrical and Computer Systems Engineering Monash University,
Lecture 16: Stereo CS4670 / 5670: Computer Vision Noah Snavely Single image stereogram, by Niklas EenNiklas Een.
Based on the success of image extraction/interpretation technology and advances in control theory, more recent research has focused on the use of a monocular.
Multiple Light Source Optical Flow Multiple Light Source Optical Flow Robert J. Woodham ICCV’90.
1Ellen L. Walker 3D Vision Why? The world is 3D Not all useful information is readily available in 2D Why so hard? “Inverse problem”: one image = many.
Linearizing (assuming small (u,v)): Brightness Constancy Equation: The Brightness Constraint Where:),(),(yxJyxII t  Each pixel provides 1 equation in.
Project 2 due today Project 3 out today Announcements TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA.
Motion / Optical Flow II Estimation of Motion Field Avneesh Sud.
Lec 26: Fundamental Matrix CS4670 / 5670: Computer Vision Kavita Bala.
MAN-522 Computer Vision Spring
Processing visual information for Computer Vision
Instructor: S. Narasimhan
제 5 장 스테레오.
José Manuel Iñesta José Martínez Sotoca Mateo Buendía
CS4670 / 5670: Computer Vision Kavita Bala Lec 27: Stereo.
Rogerio Feris 1, Ramesh Raskar 2, Matthew Turk 1
Shape Recovery Using Robust Light Striping
Epipolar geometry.
Announcements Midterms graded (handed back at end of lecture)
Common Classification Tasks
Chapter 11: Stereopsis Stereopsis: Fusing the pictures taken by two cameras and exploiting the difference (or disparity) between them to obtain the depth.
Stereo vision Many slides adapted from Steve Seitz.
Presentation transcript:

1 Intelligent Robotics Research Centre (IRRC) Department of Electrical and Computer Systems Engineering Monash University, Australia Visual Perception and Robotic Manipulation Springer Tracts in Advanced Robotics Chapter 3 Shape Recovery Using Robust Light Striping Geoffrey Taylor Lindsay Kleeman

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 2 Contents Motivation for stereoscopic stripe ranging. Benefits of our scanner. Validation/reconstruction framework. Image-based calibration technique. Experimental results. Conclusions and future work.

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 3 Motivation Allow robot to model and locate objects in the environment as first step in manipulation. Capture registered colour and depth data to aid intelligent decision making.

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 4 Conventional Scanner Triangulate from two independent measurements: image plane data and laser stripe position. Depth image constructed by sweeping stripe. Camera Image sweep stripe Stripe generator Camera Scanned object B D

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 5 Difficulties Light stripe assumed to be the brightest feature: –Objects specially prepared (matte white paint) –Scans performed in low ambient light –Use high contrast camera (can’t capture colour) Noise sources invalidate brightness assumption: –Specular reflections –Cross talk between robots –Stripe-like textures in the environment For service robots, we need a robust scanner that does not rely on brightness assumption!

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 6 Related Work Robust scanners must validate stripe measurements Robust single camera scanners: –Validation from motion: Nygårds et al, 1994 –Validation from modulation: Haverinen et al,1998 –Two intersecting stripes: Nakano et al, 1988 Robust stereo camera scanners: –Independent sensors: Trucco et al, 1994 –Known scene structure: Magee et al, 1994 Existing methods suffer from assumed scene structure, acquisition delay, lack of error recovery

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 7 Stereo Scanner Stereoscopic light striping approach: –Validation through three redundant measurements: Measured stripe location on stereo image planes Known angle of light plane –Validation/reconstruction constraint: There must be some point on the known light plane that projects to the stereo measurements (within a threshold error) for these measurements to be valid. –Reconstructed point is optimal with respect to measurement noise (uniform image plane error) –System parameters can be calibrated from scan of an arbitrary non-planar target

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 8 Validation/Reconstruction Left Camera L P Scanned surface Right Camera R P Right Image Plane Left Image Plane Laser plane at known position and angle Unknown 3D reconstruction (constrained to light plane) LxLx X RxRx Right measurement Left measurement  Light plane params

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 9 Validation/Reconstruction Unknown reconstruction projects to measurements: L x = L PX, R x = R PX Find reconstruction X that minimizes image error: E = d 2 ( L x, L PX) + d 2 ( R x, R PX) –Subject to constraint  T X = 0 ie. X is on laser plane –d 2 (a, b) is Euclidean distance between points a and b If E < E th then ( L x, R x) are valid measurements and X is the optimal reconstruction The above constrained optimization has analytical solution in the case of rectilinear stereo

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 10 Calibration System params: p = (k 1, k 2, k 3,  x,  z, B 0, m, c) –k 1, k 2, k 3 relate to laser position and camera baseline –  x,  z, B 0 relate to plane orientation –m, c relate laser encoder counts e to angle,  x = me + c Take scan of arbitrary non-planar scene –Initially assume laser is given by brightest feature Form total reconstruction error E tot over all points: E tot =  j=frames  i=scanlines E( L x ij, R x ij, e j, p)

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 11 Calibration Find p that minimizes total error (assuming L x ij, R x ij, e j are fixed): p* = min p [E tot (p)] –Use Levenberg-Marquardt numerical minimization Refinement steps: –Above solution will be inaccurate due to incorrect correspondences caused by brightness assumption –Use initial p* to validate ( L x ij, R x ij ) and reject invalid measurements, then recalculate p* –Above solution also assumes no error in encoder counts e j. Error removed by iterative refinement of e j and p*.

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 12 Implementation Right Camera Left Camera Optical Encoder Laser Stripe

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 13 Image processing Left field Right field Image difference Extract maxima Left candidates Right candidates Raw stereo image When multiple candidates appear on each scan line, calculate reconstruction error E for every candidate pair and choose pair with smallest E < E th

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 14 Scanning Process

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 15 Results Refection/cross-talk mirror experiment: mirror generates bright false stripe measurements

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 16 Results Laser extracted as brightest feature per line All bright candidates extracted and matched using validation condition.

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 17 More Results Office phone mirror scan results: Brightest feature without Validation With Validation

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 18 Specular Objects Tin can scan with specular reflections

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 19 Specular Objects Tin can scan results: Brightest feature without Validation With Validation

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 20 Depth Discontinuities Depth discontinuities cause association ambiguity

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 21 Depth Discontinuities Depth discontinuity scan results: Without ValidationWith Validation

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 22 Conclusions We have developed a mechanism for eliminating: –sensor noise –cross talk –‘ghost’ stripes (reflections, striped textures, etc) Developed an image based calibration technique requiring an arbitrary non-planar target. Operation in ambient light allows registered range and colour to be captured in a single sensor. Experimental results validate the above techniques.

Taylor and Kleeman, Visual Perception and Robotic Manipulation, Springer Tracts in Advanced Robotics 23 Future Directions Use multiple simultaneous stripes to increase acquisition rate –Multiple stripes can be disambiguated using the same framework that provides validation Perform object segmentation, modeling, and recognition on scanned data to support grasp planning and manipulation on a service robot.