Jigsaw Puzzles with Pieces of Unknown Orientation Andrew C. Gallagher Eastman Kodak Research Laboratories Rochester, New York

Outline Introduction Solving Puzzles – Measuring Pairwise Compatibility – Tree-Based Reassembly for Types 1 and 2 – An MRF for Solving Type 3 Puzzles Experiments Conclusion

Introduction Solving puzzle assembling the pieces of a jigsaw puzzle into a complete picture

Introduction

Puzzle type – Type 1: known Rotation, unknown Location – Type 2: Unknown Rotation and Location – Type 3: Unknown Rotation, known Location

Solving Puzzles a measure of jigsaw piece compatibility puzzle assembly

Solving Puzzles Measuring Pairwise Compatibility – describes the local gradients near the boundary of a puzzle piece – Mahalanobis distance

Solving Puzzles Measuring Pairwise Compatibility compute the compatibility D LR (x i, x j ) of a jigsaw piece x j on the right side of piece x i mean distribution: also compute the covariance matrix S iL

Solving Puzzles Measuring Pairwise Compatibility compute the gradient from the right side of piece x i to the left side of piece x j then, Mahalanobis distance:

Solving Puzzles Measuring Pairwise Compatibility modified the above equations to compute D RL (x j, x i ), then get the symmetric compatibility measure C LR (x i, x j ) store the confidence ratio in the 3D array S(x i, x j, r)

Solving Puzzles Evaluation in Puzzle Assembly Similarity performance for types 1 and 2

Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – a greedy assembly algorithm inspired by Kruskal’s algorithm for finding a minimal spanning tree – three stages: constrained tree stage Trimming Filling

Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – The constrained tree stage nothing prevents the MST from being a graph that results in an assembled puzzle that overlaps onto itself If a collision has occurred then the edge is discarded without merging the forests

Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – The constrained tree stage

Solving Puzzles Tree-Based Reassembly for Types 1 and 2 – Trimming and Filling

Solving Puzzles An MRF for Solving Type 3 Puzzles An natural function to minimize is the total sum of the cost across the boundaries of any two pieces

Experiments Four measures – Direct comparison – Neighbor comparison – Largest Component – Perfect Reconstruction

Experiments Type 1 Puzzles Type 2 Puzzles

Experiments Type 3 Puzzles – orientation accuracy is 97.2% when considering puzzles with 432 pieces each with 28 × 28 pixels Result

Experiments Mixed-Bag Puzzles

Conclusion a new class of square piece jigsaw puzzles that having pieces with unknown orientations a new measure (MGC) for the compatibility of a potential jigsaw piece matches a tree-based reassembly that greedily merges components a pair-wise MRF where each node represents a jigsaw piece’s orientation