2. Puzzle Solver Sravan Bhagavatula EE 638 Project Stanford ECE

3. Overview Purpose of Project High Level Implementation Scale Invariant Feature Transform Explanation of Algorithm Results Future Work

4. Purpose of Project Solving a jigsaw Finding placements Based on locations in original picture

5. High-level Implementation Needs two inputs ◦ Pieces ◦ Original Image Outputs ◦ Numbered pieces ◦ Original with placements

6. Scale Invariant Feature Transform Object Recognition technique (David Lowe) Rotation / orientation change was a problem Features obtained similar to neuron responses in inferior temporal cortex (for primate vision) Object Recognition from Local Scale-Invariant Features, D. G. Lowe, International Conference on Computer Vision, Corfu, Greece, Sept. 1999.

7. Scale Invariant Feature Transform Keypoint Locations ◦ Defined as extrema of a difference-of- Gaussian function applied in scale space Local Image Description ◦ Robust descriptor to local affine distortion

8. Scale Invariant Feature Transform Computationally efficient – one second/image order of 1000 features Occlusions Tested very well for rotation / scale changes Chosen for invariance

9. Explanation of Algorithm P – Image of pieces S – Image of complete picture 1. Find the keypoints in P and S with vl_sift 2. Output a modified P, with piece labels ◦ Use kmeans() to cluster the keypoints in each piece 3. Take a small number of points per cluster ◦ Around 20 – 30. 4. Compare these keypoints with ones in S ◦ 2-norm comparison of the SIFT keypoint descriptors

10. Explanation of Algorithm – Cont. 5. Find locations in S of matches ◦ These basically count as the location of each piece 6. Classify each region of matches into clusters ◦ I.E., choose a “central point” to designate as the label of the region 7. Output a modified version of S using these cluster labels ◦ One that has the same labels as the one in P, such that similar pieces are in the right locations

11. Results

12. Future Work Background of pieces needs to be uniform ◦ Additional step to make the background uniform? Try out orientation, lighting changes Clustering without numPieces Test it on much larger puzzles (~1000 piece, perhaps) ◦ Computation time Solve without the solution image ◦ Much harder, more than just feature matching