1. Workshop on Preserving Intellectual Assets: Institutional Repositories and Open Access TEI Thessalonikis, Sindos, September 2006 An introduction to image retrieval Professor Dick Hartley Manchester Metropolitan University

2. Introduction to image retrieval Why is image retrieval important for digital libraries and institutional repositories? Why is image retrieval difficult? What are the approaches to image retrieval?

3. How are we going to achieve this? The bad news –I am going to do some talking –So, you are going to do some listening (I hope!!!!) The good news –You are going to do some work ! (well it is a workshop!!!!)

4. How are we going to achieve this? Day 1 –Why is image retrieval important? –Why is it difficult? –One approach to image retrieval –Practical image retrieval exercise Day 2 –A second approach to image retrieval –Practical exercise in image indexing –Research on image seeking behaviour

5. What do I mean by image retrieval? Digitized images of text Digital images in every conceivable subject from medical imaging, through satellite imagery to art history

6. Why is image retrieval important? Image information is crucial in many contexts Huge quantities of image data is now available in digital form Digital information on every imaginable subject is readily available on the Web Many digital libraries contain digital information; this is pointless unless it can be effectively retrieved

7. So…. Research and practical developments in image retrieval and in understanding of image seeking behaviour have been major areas of development in information retrieval in the last decade

8. Why is it difficult? What is an image about? Look at the following examples……

14. Are you Offended?

15. Why is it difficult? I want a picture of a tower in Greece at night I want a picture of a bridge in Edinburgh during the summer

16. Approaches to image retrieval Content-based image retrieval Concept-based image retrieval

17. Content-Based Image Retrieval Semi-automatic or automatic extraction, indexing and retrieval of images by their visual attributes.

18. Similarity Measures City-Block Distance Minkowsky Distance

19. Retrieval by Colour (Global) Similar Colour (Global)

20. False Positives

21. Retrieval by Colour (Local) Dissimilar Local Colour Colour Histogram Intersection

23. Shape Retrieval

24. Inference

25. Noise

26. Trademark Image Retrieval

27. Device Marks

28. Texture

29. Image Query Paradigms Relational-Based –SQL [Codd, 1970]. –ISQL [Assmann, Venema, and Hohne, 1986], –PROBE [Orenstein, and Manola, 1988], –PSQL [Roussopoulos, Faloutsos, and Sellis, 1988] –Spatial SQL [Egenhofer, 1991]. SELECT city, state, population, location FROM cities ON us-map WHERE location within (4+4, 11+9) AND population > 450,000 Tabular-based data model –Query By Example (QBE) [Zloof, 1977]. Aggregate by Example [Klug, 1981] Generalised Query by Example [Jacobs and Walczak, 1983] Office by Example [Whang et al. 1987] Time by Example [Tansel et al., 1989] Natural Forms Query Language (NFQL) [Embley, 1989]. Query by Pictorial Example (QPE) [Chang and Fu, 1980]. PICQUERY [Joseph and Cardenas, 1988].

30. Query by Visual Example

31. QBIC Niblack et al., 1993. Lee et al., 1994. Bird et al., 1996. Bird et al., 1999. “One of the key challenges that remains in making the technology pervasive and useful is the design of the user interface.” [Flickner et al., 1997.]

32. Query by Image

33. Query by Icon

34. Query by Paint

35. Query by Sketch Query by Visual Example

36. Beyond Query by Visual Example 2D/3D Visualization –ImageVIBE [Cinque et al., 1998]. –3DVIBE [Santini and Jain, 1997]. Virtual Reality –Query by Photograph [Assfalg et al., 2000]. Taxonomy Human Perception of Images –Burford et al. [2002].

38. CBIR Summarized CBIR permits retrieval by image attributes –Colour, shape, texture (or a combination)

39. CBIR Advantages No metadata necessary Possible to “index” a huge volume of material and rapidly Does not depend on interpretation of meaning

40. CBIR disadvantages “Semantic gap” between what users want and what CBIR systems can achieve No sensible means by which queries can be presented to a CBIR system

41. CBIR uses Restricted areas such as trade mark searching

42. Time for a break, then you can experiment with an operational CBIR system