On the Complexity of Affine Image Matching STACS 2007, Aachen University of Lübeck Institute for Theoretical Computer Science Lübeck, Germany Thursday, February 22th 2007 Christian Hundt and Maciej Liśkiewicz

Foto: Andreas Herrmann What is Image Matching? On the Complexity of Affine Image Matching ? ? ?

Overview On the Complexity of Affine Image Matching 1.Notation and the Problem 2.Structure of the Searchspace 3.Polynomial Time Algorithm

Section 1 On the Complexity of Affine Image Matching Notation and the Problem

R 2 -Functions and Images On the Complexity of Affine Image Matching A: R  R  R

Notation and the Problem R 2 -Functions and Images On the Complexity of Affine Image Matching A: Z  Z  R Discretization of A ( step I ):

Notation and the Problem R 2 -Functions and Images On the Complexity of Affine Image Matching A: Z  Z  R

Notation and the Problem R 2 -Functions and Images On the Complexity of Affine Image Matching A: Z  Z  R

Notation and the Problem R 2 -Functions and Images On the Complexity of Affine Image Matching A: Z  Z  Z Discretization of A ( step II ):

Notation and the Problem Image Metric On the Complexity of Affine Image Matching BA  : R  R  R ( e.g.  (a,b) = |a-b| )  (A, B) =    ( A(x,y), B(x,y) ) dx dy -  

Notation and the Problem Image Metric On the Complexity of Affine Image Matching BA  (A, B) =    ( A(x,y), B(x,y) ) N N Discretization of  for Images: 1 4N 2 y=-N x=-N

Notation and the Problem Transformations On the Complexity of Affine Image Matching f(A)(x,y) = A( f -1 (x,y) ) Af(A) f f -1 18

Notation and the Problem Transformations On the Complexity of Affine Image Matching f(A)(x,y) = A( f -1 (x,y) ) Af(A) f -1 ??? and with Images?

Notation and the Problem Transformations On the Complexity of Affine Image Matching Af(A)

Notation and the Problem Transformations On the Complexity of Affine Image Matching Af(A) Nearest Neighbour Interpolation!!!

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (3,1)(0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (3,1)(2,1)(0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (3,1)(2,1)(1,1)(0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (3,1)(2,1)(1,1)(0,1)(0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (3,1)(2,1)(1,1)(0,1)  (0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching  f (i, j) = [ f -1 (i, j) ], if in (-N... N) 2  sonst Discretization of f: (3,1)(2,1)(1,1)(0,1)     (0,2) (0,3) (1,2) (1,3) (2,2) (2,3) (3,2) (3,3) (0,0) (0,1) (0,2) (0,3) (1,0)(2,0)(3,0)(4,0) (1,1)(2,1)(3,1)(4,1) (1,2)(2,2)(3,2)(4,2) (1,3)(2,3)(3,3)(4,3)

Notation and the Problem Transformations On the Complexity of Affine Image Matching Discretization of F:  N (F) = {  f | f  F and f injecive }

Notation and the Problem Image Matching On the Complexity of Affine Image Matching BA f ? Input:Image A, distorted image B, set of admissible transformations F Solution:Transformation f in F Goal: min  (f(A), B) = min    ( A (  (x,y) ), B ( x,y ) ) 1 4N 2 y=-N x=-N N N f  F    N (F)

Notation and the Problem How to... On the Complexity of Affine Image Matching 1.enumerate  N (F) ? 2.estimate |  N (F)| ?

Notation and the Problem Elastic Image Matching On the Complexity of Affine Image Matching f f(A)A Set of transformation F: ||(x,y)-(x‘,y‘)||  1  ||f(x,y)-f(x‘,y‘)||   [Keysers; Unger; 2002] Elastic Image Matching is NP-complete.

Notation and the Problem Projective Image Matching On the Complexity of Affine Image Matching f(A)A Set of transformation F: f(x,y) = (a/c, b/c) and (a,b,c) = M(x,y,1) Open Problem. f

Notation and the Problem Affine Image Matching On the Complexity of Affine Image Matching f(A)A Set of transformation F: f(x,y) = M(x, y) + t Our result: Affine Image Matching can be solved in polynomial time. f

Notation and the Problem More Image Matching Classes On the Complexity of Affine Image Matching f f fff Translation: Scaling: Rotation: Rotation & Scaling: Linear Transformation: f(x,y) = s0 0s cos  sin  -sin  cos  s cos  s sin  -s sin  s cos  x y t1t1 t2t2 f(x,y) = x y x y x y a1a1 a2a2 a3a3 a4a4 x y

Section 2 On the Complexity of Affine Image Matching Structure of the Search Space

Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a1a1 a2a2 a3a3 a4a4 x y t1t1 t2t2 Transformation: Vector in R 6 : v = () T

Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a2a2 a3a3 a4a4 x y t1t1 t2t2 Transformation: Vector in R 6 : a1,a1,v = () T

Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a2a2 a3a3 a4a4 x y t1t1 t2t2 Transformation: Vector in R 6 : a1,a1,v = () T

Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a3a3 a4a4 x y t1t1 t2t2 Transformation: Vector in R 6 : a 1, a 2,v = () T

Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a3a3 a4a4 x y t1t1 t2t2 Transformation: Vector in R 6 : a 1, a 2,v = () T

Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y Transformation: Vector in R 6 : a 1, a 2, t 1, a3a3 a4a4 t2t2

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a3a3 a4a4 x y Transformation: Vector in R 6 : t2t2 a 1, a 2, t 1,

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y Transformation: Vector in R 6 : t2t2 a 1, a 2, t 1, a 3, a4a4

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ a4a4 x y Transformation: Vector in R 6 : t2t2 a 1, a 2, t 1, a 3,

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y Transformation: Vector in R 6 : t2t2 a 1, a 2, t 1, a 3, a 4,

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y t2t2 Transformation: Vector in R 6 : a 1, a 2, t 1, a 3, a 4,

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y Transformation: Vector in R 6 : a 1, a 2, t 1, a 3, a 4, t 2

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y Transformation: Vector in R 6 : a 1, a 2, t 1, a 3, a 4, t 2

v = () T Structure of the Search Space Affine Transformation vs. R 6 On the Complexity of Affine Image Matching f(x,y) =+ x y Transformation: Vector in R 6 : a1a1 a2a2 a3a3 a4a4 t1t1 t2t2 a 1, a 2, t 1, a 3, a 4, t 2  v =  f -1

Structure of the Search Space Hyperplanes On the Complexity of Affine Image Matching X iji‘ : i x 1 + j x 2 + x 3 - i‘  0 Y ijj‘ : i x 4 + j x 5 + x 6 - j‘  0 Let H N be a set of hyperplanes in R 6 which contains for all i, j, i‘, j‘  [-N,N] the hyperplanes: A(H N ) is the set of cells in R 6 defined by H N.

Structure of the Search Space Central Property On the Complexity of Affine Image Matching Lemma 1. For two vectors u, v  R 6 it holds that  u =  v iff if for all i, j, i‘  [-N,N] u and v belong to the same half-subspace according to the partition of R 6 with the hyperplane X iji‘. The same holds for Y ijj‘.

Structure of the Search Space Main Result Theorem 1. Let F be the set of affine transformations, then  N (F)  A(H N ). On the Complexity of Affine Image Matching Corollary 1. |A(H N )| = O(N 18 ).

Section 3 On the Complexity of Affine Image Matching Polynomial Time Algorithm

Efficient Representation Lemma 2. Each convex cell of A(H N ) contains a vector coordinates of which can be encoded by rational numbers of lenght O(log N). Moreover the representatives can be computed efficiently. On the Complexity of Affine Image Matching

Polynomial Time Algorithm The Algorithm On the Complexity of Affine Image Matching Theorem 2. [Edelsbrunner, 1987] Any arrangement of n hyperplanes in R 6 can be constructed in O(n 6 ) time.

Polynomial Time Algorithm The Algorithm On the Complexity of Affine Image Matching Algorithm ImageMatching( ) Input: Images A, B of size (2N+1)  (2N+1) Output:Transformation f = arg min  (f‘(A), B) 1.Construct A(H N ). 2.Set f = Id and  = . 3.For all C  A(H N ) do 4.compute the representative g for C, 5.compute  ‘ =  (g -1 (A), B), 6.if (  ‘ <  ) then set  =  ‘ and f = g Return f. f‘  F

Polynomial Time Algorithm Running Time On the Complexity of Affine Image Matching Lemma 3. The algorithm runs in time O(N 20 ). Theorem 3. The Affine Image Matching Problem can be solved in polynomial time.

Polynomial Time Algorithm Extensions On the Complexity of Affine Image Matching 1.Many subclasses of Affine Transformation 2.Images with more than two Dimensions. 3.Linear Interpolation

Summary 1.The Search Space of discretized inverse affine Transformation has a regular structure. 2.The Affine Image Matching Problem can be solved in polynomial time. On the Complexity of Affine Image Matching

Thank you for your attention! Any Questions? On the Complexity of Affine Image Matching