Cherevatsky Boris Supervisors: Prof. Ilan Shimshoni and Prof. Ehud Rivlin

 AGVs can be used in: ◦ Industrial applications to move materials around a manufacturing facility or a warehouse. ◦ Transport medicine in hospitals ◦ Transporting containers in ports  Self – driving car.  Automatic house cleaner.

 We have a robot + camera.  The robot is allowed to capture images and save them in memory.  It can move only on the floor (plane) by rotation and translation.  It can use only images for navigation  We need 2 images for rotation angles and translation direction.  For recovering translation distance 3 rd image has to be captured and matched to 2 others.

Image from 1 st pose. Image from 2 nd pose.

 The navigation is performed sequentially eventually we reach the target.  We concentrate on one move from the whole sequence.

 Why can’t we find distance from two images ?  Solution: take another image at a known distance  Use the 3 images to calc the distance by triangulation. (will be explained further)

 Autonomous Visual navigation often suffers from lack of accuracy (more often when the scene is planar).  We would like to analyze this problem from theoretical POV, and build an algorithm which will improve the accuracy of every navigation step in sequential visual navigation.  A better estimation of essential matrix and an homography in planar robot motion.

 When we have an images of a planar scene there is a homography transformation between them:  The homography can be decomposed as:  At least 4-points correspondence is required for estimation. (DLT algorithm)

 Suppose the relative motion is hence the essential matrix is  For each point correspondence it holds:  Can be estimated with the eight-point algorithm(H&Z), and the essential matrix requires at least five-points. (D.Nister)

 A homography H is compatible with fundamental matrix F iff the matrix is skew-symmetric  There are 8-DOFs in F, 5 in which leaves 3 DOFs for H, which are parameterized by 3-parameter family of planes in – (N/d).  Our robot has many walls orthogonal to the floor !! What will happen in our special case ??

 How will the homography\essential matrix look like ?  How many point correspondences required for calculation ?  What is the connection between them ?  Robot Motion …

 Robot moves only on a plane (X-Z plane) and rotates in Y plane.  The camera is calibrated, hence we look for E,H. ZX Y

 The structure of H hence 2 points at least are required !

 It is shown by (E.Rivlin, I. Shimshoni and E. Smolyar) that the essential matrix in planar motion is given by  We propose a more “robust” form: ◦ Requires only two-points for solving.

 These equation defines 2 independent problems:  The values M, and b derived from the point- correspondences.  When we already computed H, we need to recover the values of.  This can be done by computing intersection of 2 circles where:

 One homography -> 2 solutions, except one special case  One special case where the are ∞ planes through the 2 points : (0,0)

 The problem is:  Can be solved by finding intersection points of 2 ellipses derived from the point pairs: Can have either 4 solutions, no- solution or infinitely many solutions !! 1.Infinitely many – there is a pure rotation between scenes. 2.4 solution – we choose the 2 where

 Minimization Problem:  Assume we don’t have any noise (reprojection error).  What is the meaning of rank(A)? ◦ Rank(A) = 2  all features are on plane ! ◦ Rank(A) = 3  Not all features are on plane ! ◦ Rank(A) = 4  Noise was added !

 [U,D,V] = svd(A)   If rank(A)=3, we take  If rank(A)=2 ◦ We get the same problem as for n=2, but ◦ This corresponds to the case where all the features are found on a dominant plane.  If rank(A)=4  there is a noise and the more likely solution type is chosen according to singular values.

 There are 2 essential matrices which are compatible with a given H (The 2 decompositions of H).  There is a 1-parameter family of homographies, parameterized by the angle ψ which are compatible with a given essential matrix.  Conclusion 1: In a planar scene, the two solutions of the essential matrix, will score the same amount of inliers in RANSAC, and we will not be able to find the correct solution.  Conslusion 2: The matrix C encodes all the information for H, hence it is possible to extract everything from this matrix:

baselineorthogonal  The correct plane normal is not changing neither in motion in the direction of the baseline and in the direction which is orthogonal to the baseline.  Validated experimentally:

 Distance from source pose to middle target is known – use triangle relationship to recover d.  Triangle scaling problem ! source target middle

Planar scene Non- Planar scene Planar scene

 100 iterations !  Rotate(60), Move(2.7), Rotate(-70)  Real angles:

Avg(err_8pt) = 0.47 Avg(err_3pt) = 0.11 Avg(err_2pt) = 0.10

2-pt: gives the right solution. Rank(A)=2 3-pt: bounces between the correct solution and the wrong one. Rank(A)=2. 8-pt: at this case it returns any combination of the correct and wrong solution. Rank(A)=6, null space is 2D. Avg(err_8pt) = Avg(err_3pt) = 40.7 Avg(err_2pt) = 0.61

 A batch of 500 iterations was executed with reprojection error of 0.5 pixels.  The variance in the φ direction is twice larger then in the θ direction !

 A batch of 500 iterations was executed with reprojection error of 0.5 pixels. A line of solutions for the eight point algorithm The correct solution The incorrect solution

 500 RANSAC batches were executed.  Each batch 400 RANSAC iterations ran at most, and the index of the first “promising” iteration was recorded.  In (d) the number of iterations is shown in sorted order. Avg = 208 Avg = 53 Avg = 90

 A motion where was compared in 3 methods.  It is easy to see the numerical problem with the 3-point method in a side motion.

 Noise values were equally sampled in the range [0,2], and averaged over 200 runs.  The proposed error measure is: ◦ where is the estimated essential matrix, and is the correct one, normalized to a unit length.

 Divided into 3 phases: 1.Room estimation – we approximate the 3D structure of the room from pairs of images with a predefined distance between them. Distance to the wall is computed as the median distance of features. 2.Searching for targets, and Roadmap building. 3.Navigation to a predefined target from any unknown position.

Target 1 Target 2 Reached target 1Reached target 2

 Totally there were 67 frames captured and it took about 9 minutes to build the roadmap and reach both targets.  When the 3-point algorithm was used instead it took 12 minutes and 95 frames were captured, in the same environment.  Two distance measures were proposed to evaluate the algorithm: 1.Distance between images ( ). 2.Distance between poses in cm and degrees for angles.

Target 1 Target 2 Reached target 1Reached target 2 D= 5 D= 11

 He The distance between the reached target and the correct was < 1cm, and less then 1 deg. Here the X axis and angle are almost identical, but a difference of 3cm in the Y axis direction.

 It took 5 iterations to find the first target, and 3 iterations to find the second, during phase 2 of the algorithm.

 Three different poses were chosen for emphasizing the plane\no-plane scene classification.  Odometer sensor readings are displayed below.

70.5% of the features matched 20.3% of the features matched 58.6% of the features matched A simple heuristic rule, if > 50% then planar scene. About 83% of images were classied correctly and we had classication error of 17%. 6 out of 34 pairs were misclassied in the experiment.

 A novel algorithm for estimating epipolar geometry and homography was developed and implemented.  The algorithm was adapted to the special case where the motion is planar and many features are present on walls.  Fewer point correspondences were suggested for essential matrix and homography estimation, and a theoretical relation between E and H was analyzed in this special case. 

 A geometric interpretation for estimating E and H was given and proven.  The algorithm deals with H-degeneracies and is able to find the correct motion which the robot has to make in order to reach the next target. Even if all features are on a dominant plane, it is possible to get the correct solution from three images.