Description of some multimedia ontologies Rapha ë l Troncy Thursday 1 st of December, 2005

Multimedia Ontologies Multimedia ontologies can be related to many works, standards, communities … Focus here on MPEG-7... as a whole ISO Standard composed of 11 parts but it is still growing … DDL = XML Schema + small extensions Descriptors and Description Schemes  1182 elements, 417 attributes, 377 complex types in the schema (MDS + Visual + Audio parts)  40 Classification Schemes

Agenda DSTC's Proposal Jane Hunter & Suzanne Little (ABC model) TUC's Proposal Chrisa Tsinaraki (DELOS) INA's Proposal Antoine Isaac & Raphaël Troncy DMAG/MTG's Proposal Roberto Garcia & Oscar Celma

DSTC's Proposal (summary) 1st proposal chronologically MPEG-7 ontology developed in RDFS, then converted into DAML+OIL and now OWL Contains 60 classes and 40 properties OWL Full ontology Cover the upper level of the MDS segments and decomposition agent, role, place, time and instrument basic visual descriptors Available at:

TUC's Proposal Software Engineering Framework aiming to facilitate Knowledge-based MPEG-7 Multimedia Application Development Being developed in TUC/MUSIC Consists of: An Ontological Infrastructure Methodologies for Interoperability between MPEG-7 and OWL An underlying MPEG-7 Repository Tools implementing the DS-MIRF approach

DS-MIRF Ontological Infrastructure

An Upper Ontology that fully captures the MPEG-7 MDS and a part of the MPEG-7 Visual Being extended to capture the MPEG-21 DIA Architecture Transformation Rules to transform OWL Ontologies and OWL/RDF metadata into MPEG-7 documents Implemented in the GraphOnto Ontology and Metadata Editor

TUC's Ontology Definition Methodology Manual definition based on the following rules: MPEG-7 Simple Datatype Representation: Integration of the XML Schema simple datatypes (rdfs:Datatype) MPEG-7 Complex Type Representation: Homonym OWL Classes  Attributes & Simple Type Elements: Datatype Properties  Complex Type Elements: Object Properties  Subclassing:  Simple Types: Datatype property with rdf:ID “type_nameContent” – where type_name is the type of the supertype  Complex Types: rdfs:subclassOf  Constraints: OWL Restrictions MPEG-7 Classification Scheme Representation: Individuals of the MDS type “ClassificationSchemeType”

TUC's Proposal (summary) Cover the full standard Contains 420 classes and 175 properties for the MDS part OWL DL Cover also the CSs The 40 CSs have been translated into simple OWL hierarchy Available at: zip zip

INA's Proposal Different uses of AV Document Descriptions Archival and description of documents from a cultural heritage point of view: INA Exchanging program identification and characterization for interactive TV: TV-Anytime Diffusion of program information (news agencies): ProgramGuideML Storing and sharing AV content descriptions (automatic extraction results): MPEG-7 standard

Can we find an AV core ontology? There are many common needs amongst observed applications Characterization of programs and sequences Decomposition of programs and sequences Ability to introduce description of the activities that constitute the context of AV documents (roles of people involved, ways production and broadcast are achieved) These concepts are close to a "neutral" archival viewpoint

C.O. content Concerning AV objects: distinction sequence/program decomposition and qualification of those objects link to external world themes and entities (content description) Underlying use patterns for elicited categories

Example: Upper-level categorization of sequences

Example: Upper-level categorization of program

INA's Proposal (summary) Cover the MDS upper part, and partially the audio and visual parts Contains 610 classes OWL DL Cover also (partially) the CSs adapted to TV Anytime (and ProgramGuideML) adapted to INA's needs Linked to the DOLCE ontology Available at:

DMAG/MTG's Proposal Reuse MPEG-7 implicit semantics. XML  Semantic Web: XSD2OWL: schema to ontology. XML2RDF: XML instance data to RDF instances. Tested as integration base for: MusicBrainz, Simac, Music Vocabulary, etc.

XMLSchema to OWL Mappings complexType//element complexType|group|attributeGroup element|attribute XML Schema owl:intersectionOf owl:unionOf owl:maxCardinality owl:minCardinality rdfs:subClassOf owl:Restriction owl:Class rdfs:range rdfs:subPropertyOf rdf:Property owl:DatatypeProperty owl:ObjectProperty OWL

DMAG/MTG's Proposal (summary) Cover the whole MPEG-7 + the upper level of the Classification Schemes Contains 2372 classes and 975 properties OWL Full Available at:

Questions ?

Towards an harmonization of these multimedia ontologies Rapha ë l Troncy Thursday 1 st of December, 2005

Preliminary results Current state: Proposals are made individually, for specific research project The OWL files contains some errors  Corrected versions available for DSTC and DMAG/MTG ontologies Different conceptualizations have been obtained … even if they start from the same MPEG-7 schema No MPEG-7 endorsement

(Personal) Idea: Three step process Aligning pair-wise ontologies Visual part of aceMedia vs DMAG vs TUC MDS part of Hunter vs DMAG vs TUC vs INA CSs part of DMAG vs TUC vs INA Merging and fusing Re-engineering (manually) ! Current proposals are based on automatic translation rules from XSD to OWL (except INA) Does NOT go beyond what XML Schema can express

Open Issues OWL DL vs OWL Full Real modularization of MPEG-7 upper part, visual, audio, CSs requires breaking some existing relationships ! Do we have to capture all MPEG-7 ? people, agent, role, place are not multimedia- specific Are there more multimedia ontologies currently developed ? Proton: Boemie: re-use the aceMedia ontologies (MDO, VDO)

Questions ?