1. MUSCLE/ImageCLEF workshop 2005 Extracting an Ontology of Portrayable Objects from WordNet Atomic Energy Agency of France (CEA) LIC2M (Multilingual Multimedia Knowledge Engineering Laboratory) BP6, 18 Route du Panorama 92265, Fontenay aux Roses, France sveta_zinger@yahoo.com, {milletc,mathieub,grefenstetteg,hedep,moellicp}@zoe.cea.fr S. Zinger, C. Millet, B. Mathieu, G. Grefenstette, P. Hède, P.-A. Moëllic

2. MUSCLE/ImageCLEF workshop 2005 2 Goal: creation of a large-scale image ontology WordNet lexical resourses Image collections acquisition through web- based image mining

3. MUSCLE/ImageCLEF workshop 2005 3 Building a large-scale image ontology for object recognition: list of portrayable objects WordNet lexical resources basis of ontology

4. MUSCLE/ImageCLEF workshop 2005 4 Building a large-scale image ontology for object recognition: visual features semantic filtering clustering classification web-based image mining large-scale visual dictionary

5. MUSCLE/ImageCLEF workshop 2005 5 Pruning approach to WordNet ENTITY has a distinct separate existence (living or nonliving) OBJECT physical object (a tangible a visible entity) simplifying connections selecting branches object  living thing  life  wildlife object  living thing  plant  …  tree  tree of knowledge object  artifact  creation  classic deleted

6. MUSCLE/ImageCLEF workshop 2005 6 Extraction from top-level ontology of portrayable objects ENTITY object living thing natural objectartifactfloater organismcelestial body rock articlecommodity consumer goods 102 nodes in total

7. MUSCLE/ImageCLEF workshop 2005 7 VIKA (Visual Kataloguer) indexing (PIRIA – LIC2M) clustering (shared nearest neighbor) visualisation of clusters web-image search engine (Alltheweb)

8. MUSCLE/ImageCLEF workshop 2005 8 List of portrayable objects (24000 items) queries to the web (e.g. google image) VIKA (Visual Kataloguer)

9. MUSCLE/ImageCLEF workshop 2005 9 Composition of queries: upper node + word identifying portrayable object Example of queries: kino tree red sandalwood tree carib wood tree Japanese pagoda tree palm tree... Japanese pagoda buildings Japanese pagoda tree trees

10. MUSCLE/ImageCLEF workshop 2005 10 VIKA (Visual Kataloguer)

11. MUSCLE/ImageCLEF workshop 2005 11 Web-image search at presentDesired results query « chair»

12. MUSCLE/ImageCLEF workshop 2005 12 Future work: face detection (adaboost learning) – to filter web-image search results semantically (images of objects without people) testing VIKA system performances automatic cluster classification – ignoring irrelevant clusters introducing new connections to the ontology: vision principles (scale), co-occurrence rules

13. MUSCLE/ImageCLEF workshop 2005 13 Future work Vision principles (scale)

14. MUSCLE/ImageCLEF workshop 2005 14 Future work Co-occurrence rules