Puzzle Solver Sravan Bhagavatula EE 638 Project Stanford ECE

Overview Purpose of Project High Level Implementation Scale Invariant Feature Transform Explanation of Algorithm Results Future Work

Purpose of Project Solving a jigsaw Finding placements Based on locations in original picture

High-level Implementation Needs two inputs ◦ Pieces ◦ Original Image Outputs ◦ Numbered pieces ◦ Original with placements

Scale Invariant Feature Transform Object Recognition technique (David Lowe) Rotation / orientation change was a problem Features obtained similar to neuron responses in inferior temporal cortex (for primate vision) Object Recognition from Local Scale-Invariant Features, D. G. Lowe, International Conference on Computer Vision, Corfu, Greece, Sept

Scale Invariant Feature Transform Keypoint Locations ◦ Defined as extrema of a difference-of- Gaussian function applied in scale space Local Image Description ◦ Robust descriptor to local affine distortion

Scale Invariant Feature Transform Computationally efficient – one second/image order of 1000 features Occlusions Tested very well for rotation / scale changes Chosen for invariance

Explanation of Algorithm P – Image of pieces S – Image of complete picture 1. Find the keypoints in P and S with vl_sift 2. Output a modified P, with piece labels ◦ Use kmeans() to cluster the keypoints in each piece 3. Take a small number of points per cluster ◦ Around 20 – Compare these keypoints with ones in S ◦ 2-norm comparison of the SIFT keypoint descriptors

Explanation of Algorithm – Cont. 5. Find locations in S of matches ◦ These basically count as the location of each piece 6. Classify each region of matches into clusters ◦ I.E., choose a “central point” to designate as the label of the region 7. Output a modified version of S using these cluster labels ◦ One that has the same labels as the one in P, such that similar pieces are in the right locations

Results

Future Work Background of pieces needs to be uniform ◦ Additional step to make the background uniform? Try out orientation, lighting changes Clustering without numPieces Test it on much larger puzzles (~1000 piece, perhaps) ◦ Computation time Solve without the solution image ◦ Much harder, more than just feature matching