Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Deformation Invariant Image Matching Haibin Ling and David W. Jacobs Center for Automation Research Computer Science Department University of Maryland, College Park Oct, 20, 2005, ICCV

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Outline  Introduction  Deformation Invariant Framework  Experiments  Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 General Deformation One-to-one, continuous mapping. Intensity values are deformation invariant. –(their positions may change)

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Our Solution A deformation invariant framework –Embed images as surfaces in 3D –Geodesic distance is made deformation invariant by adjusting an embedding parameter –Build deformation invariant descriptors using geodesic distances

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Related Work Embedding and geodesics –Beltrami framework [Sochen&etal98] –Bending invariant [Elad&Kimmel03] –Articulation invariant [Ling&Jacobs05] Histogram-based descriptors –Shape context [Belongie&etal02] –SIFT [Lowe04] –Spin Image [Lazebnik&etal05, Johnson&Hebert99] Invariant descriptors –Scale invariant descriptors [Lindeberg98, Lowe04] –Affine invariant [Mikolajczyk&Schmid04, Kadir04, Petrou&Kadyrov04] –MSER [Matas&etal02]

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Outline  Introduction  Deformation Invariant Framework  Intuition through 1D images  2D images  Experiments  Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, D Image Embedding 1D Image I(x) EMBEDDING I(x)  ( (1-α)x, αI ) αIαI (1-α)x Aspect weight α : measures the importance of the intensity

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Geodesic Distance αIαI (1-α)x p q g(p,q) Length of the shortest path along surface

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Geodesic Distance and α I1I1 I2I2 Geodesic distance becomes deformation invariant for α close to 1 embed

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Image Embedding & Curve Lengths Depends only on intensity I  Deformation Invariant Image I Embedded Surface Curve on Length of Take limit

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Computing Geodesic Distances Fast Marching [Sethian96] Geodesic level curves  Moving front Varying speed p

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Fast Marching [Sethian96] is the marching speed F T is the geodesic distance T=1 T=2 T=3 T=4 p Narrow band algorithm

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Geodesic Distance for 2D Images Computation –Geodesic level curves –Fast marching [Sethian96] is the marching speed Geodesic distance –Shortest path –Deformation invariant F T is the geodesic distance T=1 T=2 T=3 T=4 p q

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Deformation Invariant Sampling Geodesic Sampling 1.Fast marching: get geodesic level curves with sampling interval Δ 2.Sampling along level curves with Δ p sparse dense Δ Δ Δ Δ Δ

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Deformation Invariant Sampling Geodesic Level Curves Geodesic Sampling 1.Fast marching: get geodesic level curves with sampling gap Δ 2.Sampling along level curves with Δ p

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Geodesic Distance & Fast Marching

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Deformation Invariant Descriptor p q p q Geodesic-Intensity Histogram (GIH) geodesic distance intensity geodesic distance intensity

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Real Example p q

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Deformation Invariant Framework Image Embedding ( close to 1) Deformation Invariant Sampling Geodesic Sampling Build Deformation Invariant Descriptors (GIH)

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Practical Issues Lighting change –Affine lighting model –Normalize the intensity Interest-Point –No special interest-point is required –Extreme point (LoG, MSER etc.) is more reliable and effective

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Invariant vs. Descriminative

Dealing with Illumination Change Affine lighting change Normalization Sampling

Dealing with Lighting H p,1 H p,2 H p,n GIH’s at p H q,1 H q,2 H q,n GIH’s at q

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Outline  Introduction  Deformation Invariance for Images  Experiments  Interest-point matching  Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Data Sets Synthetic Deformation & Lighting Change (8 pairs) Real Deformation (3 pairs)

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Interest-Points * Courtesy of Mikolajczyk, Interest-point Matching Harris-affine points [Mikolajczyk&Schmid04] * Affine invariant support regions Not required by GIH 200 points per image Ground-truth labeling Automatically for synthetic image pairs Manually for real image pairs

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Descriptors and Performance Evaluation Descriptors We compared GIH with following descriptors: Steerable filter [Freeman&Adelson91], SIFT [Lowe04], moments [VanGool&etal96], complex filter [Schaffalitzky&Zisserman02], spin image [Lazebnik&etal05] * Performance Evaluation ROC curve: detection rate among top N matches. Detection rate * Courtesy of Mikolajczyk,

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Synthetic Image Pairs

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Real Image Pairs

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Study of Interest-Points

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Outline  Introduction  Deformation Invariance for Images  Experiments  Conclusion and Future Work

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Conclusion and Future Work  Conclusion  A new deformation invariant framework  Deformation invariant descriptor (GIH)  Future Work  Understanding how to effectively vary α  Noise & Occlusion  Fast algorithm  Real application ……

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Acknowledgement Krystian Mikolajczyk and Cordelia Schmid for the feature extraction code. Paolo Favaro and Kevin S. Zhou for discussion. NSF (ITR ). The Horvitz Assistantship.

Haibin Ling and David Jacobs, Deformation Invariant Image Matching, ICCV, Oct. 20, 2005 Thank You!