1. 04/30/13 Last class: summary, goggles, ices Discrete Structures (CS 173) Derek Hoiem, University of Illinois 1 Image: http://darksideofthecatalogue. wordpress.com/2011/11/22/lig ht-at-the-end-of-the-tunnel-is- glowing-thing-23-12/

2. Final exam Tuesday, May 7, 7-10pm DCL 1320: Students with last names Afridi to Mehta Siebel 1404: Students with last names Melvin to Zmick You should have already contacted us about conflicts. Note there is no specific conflict exam --- most conflicts should be resolved by other classes unless due to 3 tests in one day. http://admin.illinois.edu/policy/code/article3_part2_3-201.html 2

3. What to expect on final Focus on material after midterm 2, but will be stuff from earlier parts of semester also Almost certainly at least one question on each of: – Big-O/Algorithms – Proof by Contradiction – State Diagrams – Induction – Countability (not too complicated) 3

4. Today’s class Summary of concepts learned Fast image retrieval, Google Goggles, and the relevance to discrete structures ICES forms 4

5. What you learned in CS 173 How to model the world How to prove things How to model computational behavior How to think formally and computationally 5

6. How to model the world Logic, propositions, and relations – Used in natural language processing, machine learning, programming languages Sets – Used for data mining, groups (e.g., with social networks), image processing (e.g., sets of pixels), clustering Functions, algorithms – Programming languages, most programming Graphs and trees – Used in search, machine learning, social networks, path planning, menu design State diagrams – Used for design of automated systems, AI planning, map building, robotics 6

7. How to prove things Direct proof Use of cases Indirect proofs such as by contrapositive – Changing into logically equivalent form sometimes makes the proof easier Proof by example or counter-example – Useful to show something exists Induction – Proof for unbounded set of integers Contradiction – Useful to show something can’t exist 7

8. How to model computational behavior Algorithm analysis Recursion trees – Unroll recursive functions to analyze cost at root, internal calls, and leaves Big-O and Big-Theta – Analyze running time independent of implementation details and compute power 8

9. How to think formally and computationally Formal proof methods Number theory – Important for numerical methods P vs. NP – Important classes of algorithms Comparing cardinality of infinite sets – Fundamental implications, such as halting problem 9

10. Google goggles http://www.google.com/mobile/goggles/#text Demo 10

11. How to quickly find images in a large database that match a given image?

12. Basic representation: interest points (also called keypoints) Describe appearance of distinctive image patches 12 Thousands of these per image, each is a 128 dimension vector of numbers

13. Simple idea See how many keypoints are close to keypoints in each other image Lots of Matches Few or No Matches But this will be really, really slow! Like 10 images per second.

14. Slide Slide Credit: Nister 110,000,000 Images in 5.8 Seconds “Scalable Recognition with a Vocabulary Tree”, Nister and Stewenius, CVPR 2006.

15. Slide Slide Credit: Nister

16. Slide Slide Credit: Nister

17. Slide

18. Structure 1: “Visual Words”

19. K-means algorithm Illustration: http://en.wikipedia.org/wiki/K-means_clusteringhttp://en.wikipedia.org/wiki/K-means_clustering 1. Randomly select K centers 2. Assign each point to nearest center 3. Compute new center (mean) for each cluster

20. K-means algorithm 1. Randomly select K centers 2. Assign each point to nearest center 3. Compute new center (mean) for each cluster Back to 2 Demo: http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html http://home.dei.polimi.it/matteucc/Clustering/tutorial_html/AppletKM.html

21. Efficiency from clustering 21

22. Structure 2: trees for nested partitions Following slides by David Nister (CVPR 2006)

35. Much faster processing of query image 35

36. Structure 3: Inverse document file Like a book index: keep a list of all the words (keypoints) and all the pages (images) that contain them. Rank database images based on tf-idf measure. tf-idf: Term Frequency – Inverse Document Frequency # words in document # times word appears in document # documents # documents that contain the word

42. Performance

43. Speedups

44. Slide Slide Credit: Nister 110,000,000 Images in 5.8 Seconds “Scalable Recognition with a Vocabulary Tree”, Nister and Stewenius, CVPR 2006.

45. Summary Clever data structures and efficient algorithms make the difference between 10 images per second and 20 million images per second – Clustering (partitioning) for faster comparison – Trees for faster clustering – Lookup table for faster matching In this class, you learned how to model, analyze, and prove things about discrete structures 45

46. Next steps CS 225: implementing and using data structures such as linked lists, trees, graphs, etc. CS 241/242: experience writing code and structuring programs and dealing with OS CS 373: grammars, finite automata, languages, Turing machines, decidability Research or project experience 46

47. ICES forms Important for course evaluation and feedback Please provide comments about both positive aspects and ways to improve 47

48. Thank you! 48