Image Search Presented by: Samantha Mahindrakar Diti Gandhi

Introduction Definition The process of retrieving and displaying relevant images based on user’s queries from a database Contributing Factors for Image Search Increase in the availability and demand of digital images Decreases in costs for storage capacity and processing

Text-Based Image Retrieval (TBIR) Uses text descriptions to retrieve relevant images Time, location, events, objects Advantage – based on currently in use text retrieval system Disadvantage – Inconsistency based on the variation of human interpretation Google Image Search, AltaVista Image Search Google Image Search

Content-Based Image Retrieval (CBIR) Extracts images based on image content. Level 1: Retrieval by primitive features such as color, texture, shape and spatial location. Level 2: Retrieval of objects of given type. Example: find the picture of the flower. Level 3: Retrieval of abstract attributes that involves high level reasoning. Example: ‘find picture of a baby smiling’.

Color Color Characteristics of images Selection of Color Space is important in CBIR Examples  RGB, CMY Averaging colors for regions Histograms Vectors

Texture Visual Pattern Uniform intensity region of a simple shape that is repeated Two Types of Analysis Structural  texture elements are used for determining shapes and placement within a image Statistical  used for fine texture

Shape Involves detecting the border and boundary of objects. This is done by edge detection algorithms. They measure area, circularity, shape signature, curvature, eccentricity to determine the shape. Different objects such as flowers, brain tumors etc require different algorithms. Hence shape preprocessing algorithms are application dependent.

Spatial Location This feature is used for region classification. Spatial location are defined as ‘upper, bottom, top’ according to the location of the region in the image. For example sea and sky could have similar color and texture features, but their spatial locations are different.

Querying Simple Visual Feature Query: The user specifies values for feature. Simple Visual Feature Query Feature combination query: Specify values for combination of different features. Localized feature query: The user indicates feature values and locations by placing regions on the canvas. Localized feature query Query by example: The system provides a set of random images. The user selects an image closest to their query. The system then finds similar images. Query by example Concept queries: User can specify concept such as laughing as a query. Concept queries

Semantic Gap Semantic gap is the difference between human perception of a concept and how it can be represented using machine level language. Sky can represented as light blue (color), upper (spatial), uniform (texture). The focus is now shifted from designing low- level image features to reducing the semantic gap between the visual features and richness of human semantics.

Reducing semantic gap Different techniques are used to reduce the semantic gaps Object ontology Machine learning ALIPR Relevant feedback Semantic template WWW image retrieval

IA and Image search The problem that is inherent for information retrieval also applies to image retrieval i.e. semantic gap. Evaluation Metrics Recall  Percentage of all Relevant Images in the Search Database which are Retrieved Precision  Percentage of Retrieved Pictures that are Relevant to the Query

Documents Query Hits Representation Function Representation Function Query RepresentationDocument Representation Comparison Function Index Relation to the architecture of Information Retrieval Systems

Future Implications Continue to bridge the semantic gap Descriptive Story Text to Image Retrieval News agencies and Educational Story Illustrations Connecting Emotion to Images for Retrieval Using Image Retrieval to Preserve Art Image Retrieval for People with Disability Combine Sound with Search Medical Applications Improved Diagnosis

Reference Sources A Survey of content-based image retrieval with high- level semantics A Survey of content-based image retrieval with high- level semantics Techniques and Systems for Image and Video Retrieval Techniques and Systems for Image and Video Retrieval Real-Time Computerized Annotation of Images (ALIPR) Real-Time Computerized Annotation of Images SIMPLIcity: Semantics-Sensitive Integrated matching for Picture Libraries SIMPLIcity: Semantics-Sensitive Integrated matching for Picture Libraries Image Retrieval: Ideas, Influences, and Trends of the New Age Image Retrieval: Ideas, Influences, and Trends of the New Age Content-based image retrieval: a comparison between query by example and image browsing map approaches Content-based image retrieval: a comparison between query by example and image browsing map approaches

Thank You !! Questions are welcome….