Center for Machine Perception Department of Cybernetics, Faculty of Electrical Engineering Czech Technical University in Prague MAKING MINIMAL SOLVERS FAST Zuzana Kukelova, Martin Bujnak, Tomas Pajdla TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A AA A AAA A

2/24 Motivation Recognition & Tracking Placeholder text 3D Reconstruction Augmented reality Robotics

3/24 Motivation Systems of polynomial equations Camera geometry problems - calibration, relative/absolute pose estimation

4/24 Properties Requirements  all “non-degenerate” instances of the problem result in systems of equations of the same form - consisting of the same polynomials and differing only in coefficients  Contaminated input measurements => problems have to be solved for many different inputs (RANSAC)  High or even real-time performance  Contaminated input measurements => problems have to be solved for many different inputs (RANSAC)  High or even real-time performance Solve many instances of the same problem, of the same system of polynomial equations only with different coefficients, very fast Conclusion Camera geometry problems

5/24 Complicated systems 5-point relative pose problem

6/24 General methods Existing software – well known general Grőbner basis or resultant algorithms

7/24 Specific methods Design special algorithms to achieve numerical robustness and computational efficiency Intersection ellipse-line No need to use general algorithms Closed form solution

8/24 Online solver Efficient solver for solving systems of polynomial equations of one form Not general but fast This is what the user call Offline phase Study the problem Solve it in some finite prime field Design a specific efficient solver Needs to be performed only once Specific methods

9/24 Grőbner basis specific methods Parameters identification Number of solutions, basis Computations in finite prime field - fast, stable Elimination template design Which polynomials should be multiplied with which monomials and eliminated to obtain the solution G-J elimintation of the found template With concrete coefficients Eigenvalue computations Computation of eigenvalues and eigenvactors of the action matrix created from the eliminated template

10/24 Offline phase 25min 15min  Offline phase & Elimination template – hard to understand and implement  Requires knowledge from algebraic geometry 1+1=? ideal multiplication matrix Quotient ring xy 2 +7y+.. field

11/24 Automatic generator Final “online” solver List of known coeffs List of unknowns Equations Offline phase

12/24 How to make the final solver fast Reduce the size of the elimination template Kukelova Z., Bujnak M., Pajdla T., Automatic Generator of Minimal Problem Solvers, ECCV 2008, Marseille, France, October 12-18, Sparse G-J elimintation of the found template The slowest part Idea: Exchange it for computation of roots of a single-variable polynomial using Sturm-sequences Bujnak M., Kukelova Z., Pajdla T., Making Minimal Solvers Fast, CVPR 2012.

13/24 Grőbner bases of the ideal System of input polynomial equations The ideal generated by is a set of all polynomials that can be generated as polynomial combinations of where are arbitrary polynomials from

14/24 Grőbner bases of the ideal An ideal can be generated by many different sets of generators which all share the same solutions Grőbner bases - special bases Useful in solving system of polynomial equations

15/24 Lexicographic GB Graded Reverse Lex GB  Contains polynomial in one variable  Very expensive to compute  Not feasible for computer vision problems  Contains polynomial in one variable  Very expensive to compute  Not feasible for computer vision problems  Doesn’t contain a single variable polynomial  Special multiplication matrix – eigenvalues & eigenvectors give solutions  Easier to compute  Used to solve CV problems  Doesn’t contain a single variable polynomial  Special multiplication matrix – eigenvalues & eigenvectors give solutions  Easier to compute  Used to solve CV problems Grőbner bases of the ideal

16/24 We have the Graded Reverse Lexicographic GB and multiplication (action) matrix 1. FGLM conversion algorithm We have We want the variable polynomial from the Lexicographic GB We want Well known FGLM algorithm for converting grevlex GB to Lexicographic GB Needs polynomial division Doesn’t bring a speed up over eigenvalue computations Solution

17/24 Replace time consuming polynomial division performed in standard FGLM algorithm with efficient matrix-vector multiplication using the action matrix New Matrix FGLM algorithm Trick Algorithm Significant speedup over the eigenvalue computation Only feasible solutions – positive feasible focal lengths, depths or radial distortion coefficients Positives

18/24 2. Characteristic polynomial method We have We want a single variable polynomial We want Characteristic polynomial of the action matrix Krylov’s method Faddeev-Leverrier method Danilevsky method Solution

19/24 2. Characteristic polynomial method Faddeev-Leverrier method Well-known method, compute traces of matrices Suffers from a large numerical instability Faddeev Danilevsky method Transforms input matrix to its companion matrix by s − 1 similarity transformations very efficient and numerically stable Solution Krylov

20/24 5-point relative pose problem Nister – SOTA “closed form solution” GB+eig – standard Grőbner basis solution with eigenvalue computation New “single variable polynomial”solutions – ~5x speedup over GB+eig Comparable to the closed form solution Speedup over the eigenvalue method Danilevsky 14.2μs mFGML 13.7μs Faddeev 17.2μs GB+eig 61.2μs Nister 10.6μs

21/24 6-point focal length relative pose problem New “single variable polynomial”solutions ~8x speedup over GB+eig Speedup over the eigenvalue method Danilevsky 22.6μs mFGML 21.3μs GB+eig 176.3μs

22/24 P4P+f absolute pose problem New “single variable polynomial”solutions – ~3x speedup over GB+eig Speedup over the eigenvalue method Danilevsky 47.4μs mFGML 46.2μs Faddeev 51.2μs Sparse GB 82.9μs GB+eig 127.4μs

23/24 Many problems in computer vision and other fields require fast solvers of systems of polynomial equations General methods – not feasible Specific solvers – not general but fast Conclusion Automatic generator of Grőbner basis solvers New methods for speeding up Grőbner basis solvers Matrix FGLM method + Sturm sequences Characteristic polynomial method + Sturm sequences Significant speed up over existing GB solvers

24/24 THANK YOU FOR YOUR ATTENTION

25/24 We want single variable polynomial from the Lexicographic GB Well known FGLM algorithm for converting grevlex GB to Lexicographic GB Needs polynomial division Doesn’t bring speed up over eigenvalue computations FGLM conversion algorithm We have the Graded Reverse Lexicographic GB and multiplication (action) matrix

26/24 Grőbner bases of the ideal Lexicographic GB