1. DOG I : an Annotation System for Images of Dog Breeds Antonis Dimas Pyrros Koletsis Euripides Petrakis Intelligent Systems Laboratory Technical University of Crete (TUC) Chania, Crete, Greece

2. Image Annotation  The task of assigning a name or description to an unknown image  Manual: good quality, but slow, subjective  Automatic: classification problem, relies on associating image analysis features with high level concepts  Difficult to handle all image types  Semantic gap: map features to classes 8/17/2015ICIAR 2012, Aveiro, Portugal2

3. DOG I : http://www.intelligence.tuc.gr/prototypes.php  An automatic image annotation system for images of dog breeds  40 classes (dog breeds)  Descriptions: information in an ontology  Class names, properties, features, textual descriptions (from WordNet, Wikipedia)  Annotations in MPEG7 8/17/2015ICIAR 2012, Aveiro, Portugal3

4. DOG I : System 8/17/2015 ICIAR 2012, Aveiro, Portugal 4 Graphical User Interface (GUI) Feature Extraction Ontology Mapping Image Annotation DOG I Ontology Load Image Select ROI Annotation Method MPEG7 features Color + Texture features Images: 12-dim vectors 40 classes 9 instances/class Class hierarchy Class properties Image Retrieval Select Annotation Mathod Store Annotation in Exif header

5. Select ROI 8/17/2015ICIAR 2012, Aveiro, Portugal5

6. Image Content Analysis  Images of dog breeds are mainly characterized by the spatial distribution of color intensities  A 12-dimension feature vector of Color, Texture, Hybrid feature from LIRE Library  Features are normalized in [0,1]  Not all features are equally important 8/17/2015ICIAR 2012, Aveiro, Portugal6

7. Ontology  40 classes of dog breed organized in IS_A hierarchy  E.g., Dog  Working Group  Saint Bernard  Three separate hierarchies for text, features and visual descriptions  9 instances per class: raw images + a 12- dim feature vector for each image in class 8/17/2015ICIAR 2012, Aveiro, Portugal7

8. DOG I Ontology 8/17/2015ICIAR 2012, Aveiro, Portugal8

9. Image Annotation  The unknown image Q is compared with each one of the 360 images in the ontology  D(Q,I) = Σ i w i d i (Q,I)  Results are ranked by similarity with Q  Weights w i are computed by decision trees  Training set of 3,474 image pairs  8/17/2015ICIAR 2012, Aveiro, Portugal9

10. Annotation Method  Best Match: Select class of most similar instance  Max Occurrence: Select class with more instances in the first 20 answers  Average Retrieval Rank: Select class with instances ranked higher in the first 20 answers  Max Similarity: Select class whose instancing sum-up to max similar score 8/17/2015ICIAR 2012, Aveiro, Portugal10

11. Example Image 8/17/2015ICIAR 2012, Aveiro, Portugal11

12. Annotation Result 8/17/2015ICIAR 2012, Aveiro, Portugal12

13. EXIF Metadata  Descriptive information embedded inside an image  The metadata captured by your camera is called EXIF data..  DOGI stores annotation info with the pictures in the EXIF  Can be useful for image archiving and later retrievals 8/17/2015ICIAR 2012, Aveiro, Portugal13

14. Annotation in MPEG7 8/17/2015ICIAR 2012, Aveiro, Portugal14

15. Evaluation  Average annotation accuracy over 40 queries 8/17/2015ICIAR 2012, Aveiro, Portugal15 Annotation result Max Similarity AVRMax Occurrence Best Match Ranked 1 st 72.5%62.5%65%50% Ranked 2 nd 17.5%22.5%15%10% Ranked 3 rd 5%10% Overall95%92,5%90%

16. Conclusions-Future Work  DOG Ι : An automatic annotation system for dog breeds with good performance  Useful as a tool for many application  Annotation accuracy improves for less categories  Experimenting with more and animal species images categories  More elaborate image classification methods 8/17/2015ICIAR 2012, Aveiro, Portugal16

17. THANK YOU !! 8/17/2015ICIAR 2012, Aveiro, Portugal17