1. Final Project Mei-Chen Yeh May 15, 2012

2. General In-class presentation – June 12 and June 19, 2012 – 15 minutes, in English 30% of the overall grade In-class individual discussion on May 22 Original plan: Participation 10% Midterm report 20% Projects and presentations 50% project 1: 20% final project: 30% Assignments 20% New plan: Participation 10% Midterm report 25% Projects and presentations 55% project 1: 25% final project 30% Assignments 10%

3. Tasks Your own proposal: implement parts of your ideas in the midterm report Duplicate the image retrieval task on the public benchmarks – INRIA Holidays INRIA Holidays – Kentucky Benchmark (UKB) Kentucky Benchmark (UKB) Landmark classification

4. Image Search: INRIA Holiday 1,491 images of 500 scenes One image per scene is used as query to search within the remaining 1,490 images; 500 queries in total, and 991 corresponding relevant images Evaluation: – the mean Average Precision (mAP) over the 500 queriesmean Average Precision

5. Evaluation Measurement Recall – The fraction of the relevant images which have been retrieved Precision – The fraction of the retrieved images which are relevant All images Relevant |R| Retrieved |A| |Ra||Ra| |Ra|/|R||Ra|/|R| |Ra|/|A||Ra|/|A|

6. Example 10 relevant images. 15 retrieved images Ranking of the images for a given query q – {d 123, d 84, d 56, d 6, d 8, d 9, d 511, d 129, d 187, d 25, d 38, d 48, d 250, d 113, d 3 } Slide credit: Prof. Berlin Chen precision = 100% recall = 10% precision = 67% recall = 20% precision = 50% recall = 30% precision = 40% recall = 40%

7. Summarize the plot into a single number!

8. Average Precision (AP) Average the precisions obtained when a relevant image is retrieved

9. Mean Average Precision (MAP) 1.Find the average precision for each query 2.Compute the mean AP over all queries State-of-the-art* on INRIA: 0.79 *Gorda et al., Leveraging Category-Level Labels For Instance-Level Image Retrieval, CVPR 2012.

10. INRIA Holiday Dataset description – the images themselves – a set of descriptors extracted from these images – a set of descriptors produced, with the same extractor and descriptor, for a distinct dataset (Flickr60K) – two sets of clusters used to quantize the descriptors. These have been obtained from Flickr60K.

11. Kentucky Benchmark 10,200 images of 2,550 objects, all the images are 640x480. Each image is used in turn as query to search within the 10,200 images Evaluation – 4xreall@4 averaged over the 10,200 queries how many of the top-4 returned images are correct – the maximum achievable score is 4 State-of-the-art*: 3.36 *Gorda et al., Leveraging Category-Level Labels For Instance-Level Image Retrieval, CVPR 2012.

12. Landmark Classification Label instances, usually represented by feature vectors, into one of the predefined categories (67 scenes/objects in our case). Evaluation: 4-fold cross validation Yeileu

13. K-fold cross validation Partition the original sample into K subsamples. Of the K subsamples, a single subsample is used for testing, and the remaining K − 1 subsamples are used for training. Repeated K times (the folds), with each of the K subsamples used exactly once as the testing data. The K results from the folds then can be averaged to produce the classification rate.

14. K-fold cross validation …… K Validation

15. Our case: 4-fold 001.jpg 002.jpg 001.jpg 002.jpg 003.jpg 004.jpg 003.jpg 004.jpg 005.jpg 006.jpg 005.jpg 006.jpg 007.jpg 008.jpg 007.jpg 008.jpg Yeileu

16. Landmark classification: Approach Nearest neighbor Support vector machines – Linear – Non-linear (kernel) … Analyze and compare the performances of various design choices

17. Tasks Your own proposal: implement parts of your ideas in the midterm report Duplicate the image retrieval task on the public benchmarks – INRIA Holidays INRIA Holidays – Kentucky Benchmark (UKB) Kentucky Benchmark (UKB) Landmark classification

18. Make a work plan now! Start early No cheating Deliver a working system In-class individual discussion on May 22