A Robust Algorithm For Measuring Tie Points On The Block Of Aerial Images Andrey Sechin Scientific Director RACURS Alexey Chernyavskiy Alexander Velizhev Anton Yakubenko Graphics & Media Lab, MSU IX th International Scientific and Technical Conference From Imagery to Map: Digital Photogrammetric Technologies October 5–8, 2009, Attica, Greece

Area based cross correlation

Benefits and Drawbacks + High subpixel accuracy - Needs a good initial guess - Works on smooth surfaces - Fails on periodic structures Zheltov S. Y., Sibiryakov A. V Adaptive Subpixel Cross-correlation in a Point Correspondence Problem. Optical 3-D Measurement Techniques IV, Wichmann Verlag, Heidelberg, pp Two formulae are equivalent V.N. Adrov, A.D.Checkurin, A.Yu.Sechin, A.N.Smirnov, J-P. Adam-Guillaume, J-A. Qussette, Program PHOTOMOD: digital photogrammetry and stereoscopic images synthesis on a personal computer., Digital Photogrammetry and Remote Sensing ‘95, ISPRS Proceedings, Vol

Segmentation/boundary correlation Segments matching Correlation coefficient R - similarity function W 2005 PHOTOMOD 4.0 M. Drakin, A.Elizarov, A. Sechin, A.Zelenskiy AUTOMATIC STEREO POINTS MEASUREMENTS USING TWO-DIMENSIONAL FEATURE EXTRACTION, Optical 3-D Measurement Techiques VIII, v I, p , Zurich 2007.

New approach – Detectors, Descriptors, RANSAC N (N > 2) strips Images are ordered inside strips No information on strips ordering The problem: find tie points with subpixel accuracy Introduction

Universality Algorithm should work with: – Any terrain type (buildings, fields, mountains, forests, …) – Digital, scanned, space imagenary – arbitrary overlap

Detector Reduce image resolution Reduce image resolution Detector – find «corners» (1D features) on all images Detector – find «corners» (1D features) on all images – We use classic corner detectors – We select N (~1000) uniformly spaced best corners

Descriptor (SIFT/SURF/DAISY….) Calculate gradients in the neighborhood of 1D feature (corner) (gradients are invariant to lightness shift) Calculate gradients in the neighborhood of 1D feature (corner) (gradients are invariant to lightness shift) Select one (or a couple) of main gradient directions (invariance to rotations) Select one (or a couple) of main gradient directions (invariance to rotations) Calculate histograms of gradients (good neighbourhood desciption) Calculate histograms of gradients (good neighbourhood desciption) Normalize histograms (invarience to contast) Normalize histograms (invarience to contast)

Candidates for matching (1) A A’ For all points A on the first image we select the nearest (with respect to descriptor) point A’ on the second image For all points A on the first image we select the nearest (with respect to descriptor) point A’ on the second image

Candidates for matching (2) For points A’ on the second image we find the nearest point (with respect to descriptor) A” on the first image For points A’ on the second image we find the nearest point (with respect to descriptor) A” on the first image A’’ A’

Candidates for matching (3) If A and A’’ coincide, the pair (A, A’) fits for the nest stage If A and A’’ coincide, the pair (A, A’) fits for the nest stage B A, A’’ B’’ B’ A’

RANdom SAmpling Consensus (RANSAC, PROSAC,…) N (iterations number) times repeate – Randomly select pairs. The number of seleted paires must be enough for model calculation (homography, fundamental matrix, relative orientation – Calculate the model for selected matches – Calculate errors for all possible pairs for the found model – Ellimination of “bad” matches (outliers) – that do not fit the threashould – Calculate the number of “good” matches (inliers) and RMS Select the best model from all iterations Refinу the model using inliers

Example of found matches

Speedup/Reliability increase Distance filtering (desriptors) Metric filtering Topological filtering Reinforcement matching

Approximate overlap definition Consider all candidate pairs with approximately the same distance on both images. Angle voting. Shift voting with respect to x,y.

Finding matches on many images Finding conflicts CR algorithm (conflict resolution) Resolving conflicts and adding new matches

Finding “features” on images with initial resolution/subpixel refinement Several “features” should be found in the neighborhood. Repeat algorithm on initial resolution, take into account all found restrictions

Conclusion The algorithm is fast and reliable on reduced resolutions Calculation of detectors/discriptors/image overlap/RANSAC needs only several seconds of CPU time per image. CR algorithm needs some speedup (the solution is to split block into sub-blocks). To DO CR algorithm speedup. Speedup of the initial resolution part of the algorithm. Subpixel detectors (experiments to be performed).

Thank you for attention!