1. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Martin Bujňák MartinB@DataExpert.sk Supervisor : RNDr. Martin Samuelčík On-line Structure from Motion

2. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Aim Reconstruction of sparse 3D model of the scene using video sequence Input : video sequence captured with standard hand-held camera As small as possible user interactions Output : –Sparse 3D model –Camera calibration parameters Intrinsic Extrinsic

3. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Feature detection

4. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Feature detection Harris based detector –Removing features that can be interchanged with its neighborhood –Introducing feature orientation

5. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Features matching Find the nearest (distance) similar (correlation) feature in the next frame Matching ambiguities –two neighboring frames don’t differ a lot –„outliers“ are effectively removed in further processes (up to 30% of bad pairs is expected) ??

6. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Guided searching F-matrix : two view geometry –Restricts searching for matching pair to 1D search region in the second frame –Using “stereo” principle Epipolar line ordering constrain Ratio of corresponding segments is near 1 for small motion between two frames ? x

7. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Guided searching

8. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Projective reconstruction Canonical camera pair from F-matrix –F-matrix calculated using RANSAC paradigm –4 free parameters calculated so that camera pair satisfies calibration conditions Principal point at image center Aspect ration of 1 Zero skew Unknown varying focal length

9. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Projective reconstruction sparse 3D reconstruction obtained from camera pair –Intersection of sightlines passing through known feature pairs Quasi calibration –Angles are near to “calibrated” case Estimation of precision of the 3D point (size of area where 3D point can be put)

10. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Structure from motion (SfM) Merging camera pairs –Merge using common 2D- 3D feature points –Search for projective transformation H Find H or H -1 depending on quality of 3D points All measurements are performed in 2D RANSAC paradigm –Linear solution New more precise SfM –Recalculating structure –Recalculating camera motion New pair Scene 4x4 homography H H -1 3px noise added to images

11. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Self-Calibration ** ** projection constraints Pinhole camera : –Expressed by 3x4 matrix –Can be decomposed into calibration 3x3 (K), rotation 3x3 (R) and translation (T) 3x1 matrix Self-calibration –Means searching for calibration parameters – only from input images –Math : Dual Absolute Quadrics(Ω * ) KK T = P(H Ω * H T )P T KK T = (PH) Ω * (H T P T ) –Linear solution with one cubical constrain

12. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Experiment Detection and correction of lens radial distortion –Polynomial model : X 1 = (1 + a(x 0 2 +y 0 2 ))* x 0 Y 1 = (1 + a(x 0 2 +y 0 2 ))* y 0 –Search for ‘a’ using F- matrix minimizing distance of matching feature points from their corresponding epipolar lines –Numerical minimization simulated annealing

13. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk Conclusion Contribution –New feature tracker –Enhanced guided matching –Sequential SfM without need of searching for initial frame –Lens radial distortion correction Results –VideoVideo Current state –videovideo

14. © 2005 Martin Bujňák, MartinB@DataExpert.sk, http://www.DataExpert.sk/ResearchMartinB@DataExpert.sk