1. Essence Principles
Essence is the physical representation of content in different forms and formats. It carries the actual message or meaning. Essence can be produced, altered, stored, exchanged, transmitted or broadcast.
Essence in the Context of CMS
Different forms & formats
Media types
Various encoding formats
Hi-res vs. browse
Enhanced essence processing
CMS has to be format agnostic but nevertheless support advanced content processing
Additional literature for Essence Principles
W. Effelsberg, R. Steinmetz: “Video Compression Techniques” dpunkt Verlag, Heidelberg, 1998

2. Forms of Essence Basic Elements
Classified according to timing characteristics
Continuous
Consecutive, time dependent information units
Related to presentation time (i.e. 25 F/sec PAL F/sec NTSC)
Video, audio, animation, etc.
Discrete
No inherent timing requirements
Text, images, graphics, etc.
Structured Essence Formats
Combining different elements
Basic & structured
Relationships through
Links
Timing constraints
Management requires
Maintenance of object integrity
Relationship consistency
Handling of external links

3. Encoding & Compression Basics
Compression: Reduction of Bit Rate
Reduction due to:
Exploiting redundancies in bit stream
Skipping information that humans sense are not perceptible to
Lossless compression
Decompressed data stream is identical to compressed data stream
Lossy compressions
Decompressed data stream approximation of original due to missing or approximated data
Basic compression techniques
Entropy coding (lossless, e.g. Run-Length Coding)
Source coding (lossy)
Takes properties of human senses into account
Hybrid coding A B C D E ! 5
Original Data
Compressed Data

4. The Representation of Content

5. Proxy Types Metadata Multiple Media Elements Textual Proxies
Written documentation in plain text
Documents
Abstracts, scripts, production sheets, etc.
Contracts, legal documents
Database Proxies
Structured content representation
Data model
Elements & attributes
Multimedia Proxies
Browse copies
Low-resolution audio & video copies
Audiovisual abstracts
Keyframes
Automatically extracted and compiled
Combinations of media types
Storyboards (text, images, time-codes)
Multiple Media Elements

6. Metadata Definition Description Aspects Metadata Types
“Data about data“
Description of content
Representation of content
Description Aspects
Viewpoints
User view
Application view
System view
Workflow steps
Covers entire workflow from content creation to archiving
Content exchange
Inner-organisational
Inter-organisational
Metadata Types
Content related metadata
Material related metadata
Location related metadata

7. Workflows and Content Description
Acquisition and Ingest
Creation information
Automatically retrieved
GPS positioning, date & time, recording equipment, format, etc.
Acquisition
Video & audio source & compression format
Production, Broadcast and Archiving
Post-production and editing
Timecode, subtitles, commissioning, picture editor
Programme as Broadcast
Date & time of broadcast, channel, statistical information, re-broadcasts
Cataloguing
Cataloguer, Cataloguing date, topical classification, programme affiliation, abstract

8. Object related Metadata
Concept
Describes content object as a single entity
Content identifiers
Content description
Title, creators, directors, etc.
Involved instances
Programmes, editorial offices, etc.
Representation in the System
Database
Data model capturing all relevant aspects
Database Management System for persistent storage and search
Structured files
Index by fulltext engines
Stored in file systems

9. Segment Related Metadata
Segments in Content Objects
Segmentation can be
Temporal (time-codes)
Spatial (region co-ordinates)
Advantages
Cataloguing flexibility
Describe segments at appropriate level of detail
Target search operations
Temporal Segmentation
Segments associated with time-line
Logical or physical time-line
Text associated with segments
Discrete media elements are associated with time concepts
Log Sheets

10. Segment related Metadata: Stratification
Keywords Spotted
Closed Captions
Copyright
Image Content
Timecode
Key-frames
Browse
HiRes
Strata
Textual description with time relationship
Different concepts represented by different strata
Replacing traditional log sheets

11. Content Hierarchies Collection of Programmes Programme Programme Item
Grouping of programmes that are part of the same products
E.g. TV series, news programme, current affairs magazine
Programme
Logically related chunks of programme items
E.g. single broadcast of news programme, current affairs magazine
Programme Item
Single entity part of a larger unit
E.g. news story, interview, performance
Shot
Sequence of frames between transition

12. Content Hierarchies Collection of Programmes
Grouping of programmes that are part of the same products
E.g. TV series, news programme, current affairs magazine
Programme
Logically related chunks of programme items
E.g. single broadcast of news programme, current affairs magazine
Programme Item
Single entity part of a larger unit
E.g. news story, interview, performance
Shot
Sequence of frames between transition

13. Search and Query for Metadata
Requirements
Unified search interface
User groups
experienced vs. non-experienced
Search over multiple databases and information systems
Query Types
Fulltext Query
Natural language search
Performed on indexed files
Query for Labels
Structured queries for specific concepts
Names, places, dates
Mapped onto different information systems
Query for Segment
Segment search only
E.g. “ person = X present at place=Y and keyword= A, B, C“
Native Query
Database search
No federated search possible

14. Inter-Organisational Metadata Exchange
Proprietary Data Model
P/META, SMEF
MPEG-7
XML
Serialize/Pack
Encode
references
Transmit
SDTI MOS
CORBA
. . .
SMPTE Metadata Dictionary
references
P/META, SMEF
MPEG-7
Proprietary Data Model
Decode
Unpack/Deserialize
XML

15. SMEF and P/META Standard Media Exchange Framework (SMEF)
Developed by the BBC Media Data Group
Exchange data model for the BBC
No BBC wide applicable data model
Existing systems keep their data models
New systems have to be SMEF compliant
Covers complete broadcast production chain
Plan, draft and commissioning
Content creation
Promos and programme scheduling
Playout and transmission
Media management and archive
Probably the most extensive data model today
P/META
Working group within the European Broadcasting Union (EBU)
Goal: EBU standard exchange data model („EURO-SMEF“)
SMEF is one contribution
We pre-process the video material in order to support quick navigation and meaningful overviews over the content to be documented. The pre-processing automatically breaks the video into individual pieces of camerawork (shots) . In order to avoid misdetections, we need to analyse picture contents and camera movements.
For each shot we try to find a representative set of keyframes that gives a comprehensive overview of the shot.
We also try to combine the elementary shots back together into scenes or other units of documentation, automatically detecting repeated segments, like dialogue shots or interviews, for example.
The result of this analysis is a detailed, storyboard-like overview. You will have the opportunity to see some results in the afternoon session demos

16. SMEF Basics Core Concept
Entities: “ Some thing an organisation Maintains“
Entities are characterised by attributes
Dynamic attributes
Reference data (e.g. ISO country codes)
Relationships represent associations between entities
Defined by business rules derived from business processes
Entity-relationships diagrams to illustrate the data model
Media object at the centre of the data model
Entity Structure
Editorial Objects
EDITRORIAL_OBJECT_VERSION ( title, sub-title, creation date & time)
Subtypes:
PROGRAMME_OBJECT_VERSION
MUSIC_SPEECH_SOUND_ITEM_OBJECT
OTHER_ITEM_OBJECT
EDITORIAL_OBJECT_VERSION_WORK
Record of creation and development process
EDITORIAL_OBJECT_VERSION_INSTANCE
Temporary or permanent instance of work
Editorial Object Groups
EDITORIAL_OBJECT_GROUP (e.g. series, programmes strands)
ACQUISTION_BLOCK (e.g. CD)
EDITORIAL_OBJECT_ASSOCIATION (e.g. playlists)

17. Entity Structure (cont.)
Media Object Entity
MEDIA_OBJECT (action start date & time, capture start date & time, etc.)
Single media type (e.g. Audio clip, video segment, stills, etc.)
General and editorial metadata
Subtypes:
AUDIO_CLIP, SHOT
GRAPHIC, STILL, DATA (e.g. captions, web sites, text, etc.),
MEDIA_OBJECT_GROUP
Non-hierarchical grouping of events (e.g. different shots of one event on the football pitch)
MEDIA_OBJECT_GROUP_TYPE
Action, sensor perspective, sound
UNIQUE_MATERIAL_ENTITY (compression ratio, creation date and time)
Specifies media related subtypes and coding standards
Other Entities
Documentation and Management
STORAGE, STORAGE_TYPE
EDITORIAL_DESCRIPTION_SCHEME, EDITORIAL_DESCRIPTION_TERM
Individuals
PERSON, ORGANISATION
ROLE
Production related entities
Transmission related entities

18. SMEF Data Model: Contract Negotiation
PERSON
Primary Key
PER_Identifier [PK 1]
Non-Key Attributes
PER_First_Name PER_Middle_Name PER_Last_Name PER_Salutation_Form PER_Suffix_Name PER_Description PER_Presenter_Read_Rate
negotiates
negotiated by
RIGHT
CONTRACT_LINE
CONTRACT
Primary Key
contained in
Primary Key
contained in
Primary Key
RIG_Number [PK 1]
CNL_Line_Identifier [PK 1]
CON_Number [PK 1]
contains
contains
Non-Key Attributes
Non-Key Attributes
Non-Key Attributes
RIG_Jurisdiction_Desc RIG_Start_Date RIG_End_Date RIG_Condition_Desc
CNL_Amount CNL_Delivery_Date CNL_Delivery_Time
CON_Effective_Date

19. SMPTE Metadata Dictionary
The Standard
Society of Motion Pictures and Television Engineers, “SMPTE Metadata Dictionary, RP210.2 (including RP210.1) Merged Version, post trail publication of RP210.2”, White Plains, NY; USA, December 2001.
Objectives
Capturing and exchanging metadata
Does not specify a data model, cataloguing convention or description scheme
Structure
Distinct classes
Share common characteristics or attributes
Hierarchical organisation

20. SMPTE Metadata Dictionary
Class 1 Identification & Location
Unambiguous ID
SMPTE UMID, GUID, ISO, unique IPR ID, etc.
Class 2 Administration
Administrative or business information
Including
Rights information, usage restrictions, encryption information, financial
Class 3 Interpretation
Indexing and cataloguing
For administering , searching and retrieving of content
Textual description
Content & narrative descriptions, keywords, scripts, genre, etc.
Low-level technical descriptors
Colour histograms, texture maps, object shapes, etc.
Class 4 Parametric
Technical characteristics of camera or sensor systems
Technical characteristics of essence & metadata
Creation information, encoding characteristics, time-code characteristics

21. SMPTE Metadata Dictionary (cont.)
Class 5 Process
Processing and processing results of essence lifecycle
E.g. EDL, audit trail, list of compression and de-compression steps, format changes, etc.
Class 6 Relational
Relationships between objects
Content objects
Combinations of essence, objects and metadata
Examples of relationships
Essence-to-essence, metadata-to-metadata, object-to-object, Metadata-to-object
Generic, relatives, related production material, etc.
Class 7 Spatio-Temporal
Temporal and spatial aspects of
Content, originating camera, sensor or system
Geo-spatial information
Time information
Date, time-code, synchronisation marks, motion vectors, etc.

22. Organisational and Experimental Classes
Class 14 Organisationally Registered for Public Use
Elements registered by organisations or individuals
Managed by SMPTE Registration Authority
Metadata published for general use
Intended for exchange of metadata between organisations
Class 15 Organisationally Registered as Private
Metadata elements are not published
Tags referring to the element are identified
Reserved for organisational use
Class 16 Experimental
Metadata defined for research
Systems with
Limited access
Experimental environments
Does not have to conform to metadata dictionary definitions

23. Metadata Dictionary Element Structure
SMPTE Universal Label
Combination of attributes, sets and data types
Label structure
2* 16 Byte numerical layout
Element types give the format of valid values

24. SMPTE Universal Label Objective
To unambiguously identify and locate content
Single frame granularity
Associated with a content object for its entire lifetime
Usage within storage and streaming applications
Locally generated but globally unique
Acquisition
UMID Structure
32 Byte basic UMID
Universal label identifier
Length descriptor
Identifier of copy or version of an object
Material number identification
32 optional signature UMID
Time/ Date – Time of creation
Spatial co-ordinates (camera location)
Country code
Organisation code (of producing organisation)
Use code (name of producer)