Correspondence-Free Determination of the Affine Fundamental Matrix (Tue) Young Ki Baik, Computer Vision Lab.

2 Correspondence-Free Determination of the Affine fundamental Matrix References Correspondence-Free Determination of the Affine Fundamental Matrix Stefan Lehmann et. al. PAMI 2007 Radon-based Structure from Motion Without Correspondences Ameesh Makadia et. al. CVPR 2005 Robust Fundamental Matrix Determination without Correspondences Stefan Lehmann et. al. APRS 2005

3 Correspondence-Free Determination of the Affine fundamental Matrix Contents The conventional method of SfM Features of the proposed method Theory of the proposed algorithm Experimental results Discussion

4 Conventional SfM Image Sequence Feature Extraction/ Matching Relating Image Projective Reconstruction Auto-Calibration Dense Matching 3D Model Building Correspondence-Free Determination of the Affine fundamental Matrix

5 The Problem of conventional SfM The high sensitivity of fundamental matrix Noise and outlier correspondences in feature data severely affect the precision of the fundamental matrix Incomplete 3D reconstruction

6 Correspondence-Free Determination of the Affine fundamental Matrix The Key Feature Correspondence-free Finding Correspondence (X) Illumination changes-free (?) Intensity value (X) Position of features (O) Limitation Occlusion ? (X) Affine camera only!!

7 Correspondence-Free Determination of the Affine fundamental Matrix Parallel projection Orthographic projection

8 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Assumption We have 3-dimensional N features. The 3D feature space is represented by,

9 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Assumption Parallel projection model determines the 2D feature projections along the lines that are running parallel to the view axis (z-axis) of the camera. The model considers a continuous projection plane with infinite extent. The corresponding 2D projection data is…

10 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Fourier spectra The Fourier spectra of and can be denoted as

11 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model 2-view case The 3D correspondence feature point Relation between images The 3D frequency vector

12 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model 2-view case Relation between 3D spectrums The equation shows that rotation R also establishes the transformation between corresponding frequency indices in the 3D Fourier spaces of the original and the transformed spectrum or scene.

13 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Matching line The magnitudes of two spectra along these lines will be identical, while the phases will show a linear offset dependent upon the translational component of transformation. The proposed method is to detect these matching lines.

14 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Matching line angle pair Angle pair of the matching lines with respect to the axes of the frequency spectra F and F’, respectively.

15 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Analysis of the transformation parameters as the corresponding frequency locations along the matching lines of the spectrum F of the first and the spectrum F’ of the second set of 2D features, respectively. It follows that,

16 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Analysis of the transformation parameters

17 Correspondence-Free Determination of the Affine fundamental Matrix Mathematical Model Derivation of a 3D rotation matrix

18 Correspondence-Free Determination of the Affine fundamental Matrix Estimation of the fundamental matrix By using 3D rotation matrix, we can obtain the relation between 2D projection point (x’,y’) of a 3D feature (x,y,z) with translation.

19 Correspondence-Free Determination of the Affine fundamental Matrix Estimation of the fundamental matrix In the orthographic projection case, all epipolar lines are parallel. Then we can denote the epipolar line of 2D feature point (x,y) as

20 Correspondence-Free Determination of the Affine fundamental Matrix Estimation of the fundamental matrix

21 Correspondence-Free Determination of the Affine fundamental Matrix Estimation of the fundamental matrix

22 Correspondence-Free Determination of the Affine fundamental Matrix Estimation of matching line angle For the practical purpose, corresponding discrete spectra should be defined as follows.

23 Correspondence-Free Determination of the Affine fundamental Matrix Estimation of matching line angle The final object function Discrete Fourier-Mellin transformation method To find out the matching line (According to the well known shift theorem of the FT, a shift in the space domain corresponds to a phase shift in the frequency domain.)

24 Correspondence-Free Determination of the Affine fundamental Matrix Overall flow

25 Correspondence-Free Determination of the Affine fundamental Matrix Experimental result test images : telephoto lens Feature points : Harris corner detection method Ideal epipolar lines are the horizontal lines. The proposed method shows us good result relative to conventional methods.

26 Camera Calibration Methods for Wide Angle view Discussion Key feature Correspondence-free method for obtaining the fundamental matrix is presented. Matching line exists between the Fourier transformed data. Limitation Proposed method Considers only affine projection model Does not treat occlusion problem Future work Applying projective projection model