Debajyoti MondalYang Wang Stephane Durocher Department of Computer Science University of Manitoba

31/05/2013CRV What are Jigsaw Puzzles?

31/05/2013CRV Square Jigsaw Puzzles 24×18 = 432 puzzle pieces

31/05/2013CRV State-of-Art Solvers Pomeranz, Shemesh and Ben-Shahar CVPR 2011 Cho, Avidan and Freeman CVPR 2010 CVPR 2012 Andrew Gallagher Solved puzzles with 432 pieces Average 10% accuracy on 432 piece puzzles Solved puzzles with 3300 pieces Average 94% accuracy on 432 piece puzzles Solved puzzles with 9600 pieces Average 95% accuracy on 432 piece puzzles

31/05/2013CRV Why Solving Jigsaw Puzzles ? Restore Torn Apart Documents content/uploads/2011/04/paperShredding.jpg Fossil Reconstruction Ancient art/document reassembly

31/05/2013CRV Our Robust Jigsaw Solver (Noise and Missing Boundary)

31/05/2013CRV Our Robust Jigsaw Solver (Noise and Missing Boundary)

31/05/2013CRV How to Solve a Puzzle? XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk

31/05/2013CRV Successful Strategies Pomeranz et. al. [CVPR 2011] Sum of Squared Distance (SSD) Gallagher [CVPR 2012] Mahalanobis Gradient Compatibility (MGC) SSD ( x i, x j ) = D LR ( x i, x j ) MGC ( x i, x j ) = f (μ i, G ij )

31/05/2013CRV Our Approach: M+S (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q. MGC SSD M+S 20 images, each with 432 Puzzle Pieces of size 28×28×3

31/05/2013CRV Further Refinements MGC Scoring matrix (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q | MGC(3,1) - MGC(3,2) | < σ Row 3

31/05/2013CRV How to Refine this further? MGC Scoring matrix (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q | MGC(3,1) - MGC(3,2) | < σ Row 3 Greedy choice! No global Agreement!

31/05/2013CRV Selectively Weighted MGC (wMGC) MGC Scoring matrix

31/05/2013CRV Selectively Weighted MGC (wMGC) MGC Scoring matrix A bijection with optimum weight

31/05/2013CRV Selectively Weighted MGC (wMGC) 5 2 MGC Scoring matrix Row 2 wMGC (x i, x j ) = Column 4 (M+S) Score, if ‘Conflict’ MGC Score, otherwise.

31/05/2013CRV Selectively Weighted MGC (wMGC) 5 2 MGC Scoring matrix Row 2 wMGC (x i, x j ) = Column 4 (M+S) Score, if ‘Conflict’ MGC Score, otherwise.

31/05/2013CRV Experimental Results (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q. wMGC (x i, x j ) = (M+S) Score, if ‘Conflict’ MGC Score, otherwise. 20 images, each with 432 Puzzle Pieces of size 28×28×3 MGC SSD M+S wMGC

31/05/2013CRV Gallagher’s Reassembly [CVPR 2012] Scoring Matrix Construct Spanning Tree  Trimming  Filling

31/05/2013CRV Results MIT scene database, 328 images of forest, 308 images of city 81 pieces per puzzle, each piece of size 28×28×3

31/05/2013CRV Future Research  Image Filtering?  How much does it help if we know the image category?  Robust functions for compatibility scoring.

Thank You