MMDB-9 J. Teuhola Standardization: MPEG-7 “Multimedia Content Description Interface” Standard for describing multimedia content (metadata). Goal: Efficient searching, browsing and filtering of audiovisual material: still images, graphics, 3D models, audio, speech, video, multimedia presentations. Feature extraction and the search engine are not included in the standard. Descriptions are based on XML. Overview: Feature extraction Standard description Search engine Scope of MPEG-7

MMDB-9 J. Teuhola MPEG-7 concepts Feature: Distinctive characteristic of a given MM object Descriptor: Defines the syntax and semantics of feature representation; instantiation = descriptor value Description scheme: Structure and semantics between components (which can be descriptors or description schemes) Description definition language (DDL): Allows the creation of new descriptors and description schemes Description instance: Description scheme + set of descriptor values that describe the data. Descriptions have coded representations. Description Definition Language Descriptors Description schemes Descriptions Definition structuringinstantiation DefinitionTags

MMDB-9 J. Teuhola MPEG-7: example descriptors Visual:  Basic structures and layout (2D, 3D, time)  Color (color space, dominant color, quantization, layout, …)  Texture (edge histogram, homogenous texture, …)  Shape (region-based shape, contour-based shape, …)  Motion (camera motion, motion activity, motion trajectory, …)  Localization (region locator, spatio-temporal locator)  Face recognition Audio:  Basic (low-level) features of audio signals (spectrum etc.)  High-level description tools, e.g. sound recognition and indexing, instrumental timbre, spoken content, audio signature, melodic description

MMDB-9 J. Teuhola MPEG-7: potential application areas Digital libraries (retrieval from archives of text, images, speech); Education (finding teaching material, preparing virtual courses) Journalism (searching archives by voice, face, etc.) Broadcast indusry (audiovisual archives) Entertainment business (video-on-demand, games) Culture (contents of museums, art galleries); Police investigations (surveillance, face recognition) Geographical information systems (spatial databases, cartography, natural resources management) Medicine (patient information; telemedicine)