MULTIMEDIA DATA MODELS AND AUTHORING

DATA MODEL REQUIREMENTS Extensible (new data types can be added) Able to represent basic media types Flexible (items can be specified, queried, and search) Allow efficient storage and search

A general multimedia data model Spatial Temporal Composite Object layer Text Image Graphics Audio Video Media type layer …. …. …. …. Raw Compressed Grayscale Color JPEG JBIG DPCM Media format layer A general multimedia data model

MULTIMEDIA DATA MODEL Object layer Consists of media items with specified and temporal relationships. Example: slide show consisting of a number of images and accompanying audio. Media Type Layer Contains text, graphics, image, audio and video . Features or attributes are specified.

Media Format Layer Specifies the media formats in which data is stored. Has many possible formats. Example : Images can be in raw bit or compressed formats.

EXAMPLE DATA MODELS VIMSYS Data Model Domain Knowledge Domain Events Domain Objects Image Objects Image Representation Domain Knowledge Domain Independent

The VIMSYS model manage visual information (images and video) Consist four layers: Image representation layer Image object layer Domain object layer Domain event layer contains image data and any transformation that results in an alternate image representation. only can handle queries that request pixel-based information. provides the raw data for upper layers to extract and define higher level features.

Image Object Layer Has two sublayers : - Segmentation sublayer condenses information about an entire image or video into spatial or temporal clusters of summarized properties. relies heavily on image and video segmentation techniques. - Features sublayer Efficient to compute, organizable in a data structure and amenable to a numeric distance computation to produce a ranking score.

Domain Object Layer User-defined entity representing a physical entity or a concept that derived from one or more features in the lower layers. Domain Event Layer Defines event that can be queried by the user based on motion (velocity of object movement), spatial and/or temporal relationships between objects and so on.

MULTIMEDIA VIDEO MODEL Episode Scene Shot Frame A general multimedia video model

A general video model : Consist of 4 level. Frame, Shot, Scene, Episode. Frame : Individual image in a video. Shot : Set of frames recorded in a single shooting. Scene : Set of shots with common semantics. Episode : Set of scenes in a specific sequence.