1. Jigsaw Puzzles with Pieces of Unknown Orientation Andrew C. Gallagher Eastman Kodak Research Laboratories Rochester, New York

2. Outline Introduction Solving Puzzles – Measuring Pairwise Compatibility – Tree-Based Reassembly for Types 1 and 2 – An MRF for Solving Type 3 Puzzles Experiments Conclusion

3. Introduction Solving puzzle assembling the pieces of a jigsaw puzzle into a complete picture

4. Introduction

5. Puzzle type – Type 1: known Rotation, unknown Location – Type 2: Unknown Rotation and Location – Type 3: Unknown Rotation, known Location

6. Solving Puzzles a measure of jigsaw piece compatibility puzzle assembly

7. Solving Puzzles Measuring Pairwise Compatibility – describes the local gradients near the boundary of a puzzle piece – Mahalanobis distance

8. Solving Puzzles Measuring Pairwise Compatibility compute the compatibility D LR (x i, x j ) of a jigsaw piece x j on the right side of piece x i mean distribution: also compute the covariance matrix S iL

9. Solving Puzzles Measuring Pairwise Compatibility compute the gradient from the right side of piece x i to the left side of piece x j then, Mahalanobis distance:

10. Solving Puzzles Measuring Pairwise Compatibility modified the above equations to compute D RL (x j, x i ), then get the symmetric compatibility measure C LR (x i, x j ) store the confidence ratio in the 3D array S(x i, x j, r)

11. Solving Puzzles Evaluation in Puzzle Assembly Similarity performance for types 1 and 2

12. Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – a greedy assembly algorithm inspired by Kruskal’s algorithm for finding a minimal spanning tree – three stages: constrained tree stage Trimming Filling

13. Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – The constrained tree stage nothing prevents the MST from being a graph that results in an assembled puzzle that overlaps onto itself If a collision has occurred then the edge is discarded without merging the forests

14. Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – The constrained tree stage

15. Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – Trimming and Filling

16. Solving Puzzles An MRF for Solving Type 3 Puzzles An natural function to minimize is the total sum of the cost across the boundaries of any two pieces

17. Experiments Four measures – Direct comparison – Neighbor comparison – Largest Component – Perfect Reconstruction

18. Experiments Type 1 Puzzles Type 2 Puzzles

19. Experiments Type 3 Puzzles – orientation accuracy is 97.2% when considering puzzles with 432 pieces each with 28 × 28 pixels Result

20. Experiments Mixed-Bag Puzzles

21. Conclusion a new class of square piece jigsaw puzzles that having pieces with unknown orientations a new measure (MGC) for the compatibility of a potential jigsaw piece matches a tree-based reassembly that greedily merges components a pair-wise MRF where each node represents a jigsaw piece’s orientation