1. Debajyoti MondalYang Wang Stephane Durocher Department of Computer Science University of Manitoba

2. 31/05/2013CRV 20132 What are Jigsaw Puzzles?

3. 31/05/2013CRV 20133 Square Jigsaw Puzzles 24×18 = 432 puzzle pieces

4. 31/05/2013CRV 20134 State-of-Art Solvers Pomeranz, Shemesh and Ben-Shahar CVPR 2011 Cho, Avidan and Freeman CVPR 2010 CVPR 2012 Andrew Gallagher Solved puzzles with 432 pieces Average 10% accuracy on 432 piece puzzles Solved puzzles with 3300 pieces Average 94% accuracy on 432 piece puzzles Solved puzzles with 9600 pieces Average 95% accuracy on 432 piece puzzles http://www.cs.bgu.ac.il/faculty/person/dolevp.htmlhttp://www.cs.bgu.ac.il/faculty/person/dolevp.html http://www.cs.bgu.ac.il/faculty/person/shemeshm.html http://www.cs.bgu.ac.il/~ben-shahar/http://www.cs.bgu.ac.il/faculty/person/shemeshm.htmlhttp://www.cs.bgu.ac.il/~ben-shahar/ http://www.eng.tau.ac.il/~avidan/http://www.eng.tau.ac.il/~avidan/ http://people.csail.mit.edu/taegsang/ http://people.csail.mit.edu/billf/http://people.csail.mit.edu/taegsang/http://people.csail.mit.edu/billf/ http://chenlab.ece.cornell.edu/people/Andy/

5. 31/05/2013CRV 20135 Why Solving Jigsaw Puzzles ? Restore Torn Apart Documents http://www.bouldercitysocial.com/wp- content/uploads/2011/04/paperShredding.jpg Fossil Reconstruction http://www.aim.uzh.ch/morpho/wiki//CAP/3-2 Ancient art/document reassembly http://www.edgarlowen.com/n1/b7300.jpg

6. 31/05/2013CRV 20136 Our Robust Jigsaw Solver (Noise and Missing Boundary)

7. 31/05/2013CRV 20137 Our Robust Jigsaw Solver (Noise and Missing Boundary)

8. 31/05/2013CRV 20138 How to Solve a Puzzle? XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk XiXi XjXj XiXi XkXk

9. 31/05/2013CRV 20139 Successful Strategies Pomeranz et. al. [CVPR 2011] Sum of Squared Distance (SSD) Gallagher [CVPR 2012] Mahalanobis Gradient Compatibility (MGC) SSD ( x i, x j ) = D LR ( x i, x j ) MGC ( x i, x j ) = f (μ i, G ij )

10. 31/05/2013CRV 201310 Our Approach: M+S (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q. MGC SSD M+S 20 images, each with 432 Puzzle Pieces of size 28×28×3

11. 31/05/2013CRV 201311 Further Refinements MGC Scoring matrix (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q. 539167248 | MGC(3,1) - MGC(3,2) | < σ Row 3

12. 31/05/2013CRV 201312 How to Refine this further? MGC Scoring matrix (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q. 539167248 | MGC(3,1) - MGC(3,2) | < σ Row 3 Greedy choice! No global Agreement!

13. 31/05/2013CRV 201313 Selectively Weighted MGC (wMGC) MGC Scoring matrix 3 2 2 3

14. 31/05/2013CRV 201314 Selectively Weighted MGC (wMGC) MGC Scoring matrix A bijection with optimum weight

15. 31/05/2013CRV 201315 Selectively Weighted MGC (wMGC) 5 2 MGC Scoring matrix 539167248 Row 2 wMGC (x i, x j ) = Column 4 (M+S) Score, if ‘Conflict’ MGC Score, otherwise.

16. 31/05/2013CRV 201316 Selectively Weighted MGC (wMGC) 5 2 MGC Scoring matrix 539167248 Row 2 wMGC (x i, x j ) = Column 4 (M+S) Score, if ‘Conflict’ MGC Score, otherwise.

17. 31/05/2013CRV 201317 Experimental Results (M+S) Compatibility Score = MGC( x i, x j ) SSD( x i, x j ) 1/q. wMGC (x i, x j ) = (M+S) Score, if ‘Conflict’ MGC Score, otherwise. 20 images, each with 432 Puzzle Pieces of size 28×28×3 MGC SSD M+S wMGC

18. 31/05/2013CRV 201318 Gallagher’s Reassembly [CVPR 2012] Scoring Matrix Construct Spanning Tree  Trimming  Filling

19. 31/05/2013CRV 201319 Results MIT scene database, 328 images of forest, 308 images of city 81 pieces per puzzle, each piece of size 28×28×3

20. 31/05/2013CRV 201320 Future Research  Image Filtering?  How much does it help if we know the image category?  Robust functions for compatibility scoring.

21. Thank You