Digital Preservation: The Multimedia Standards way Mario Döller Assistant Professor University of Passau, Germany 1st International Digital Preservation Interoperability Framework (DPIF) Symposium

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Agenda Heterogeneity in Digital Preservation and Multimedia Retrieval in general Selected solutions based on Multimedia Standards oMPEG Query Format oJPEG JPSearch Conclusion

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Digital Preservation (DP) Efforts National Programs (US, EU, Australia, etc.) oNDIIPP, RAMA, ORION, CASPAR, PLANETS, PADI, PANIC, … Digital Preservation Europe (DPE) oimprove collaboration and synergies between existing preservation initiatives across Europe Developed Metadata Formats oGeneral: MPEG-7, Dublin Core, … oDP: VRA (Visual Resources Association) Core 4.0, CIDOC Conceptual Reference Model (CRM, (ISO 21127:2006) ), museumdat, …

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Multimedia Retrieval Current Situation

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Example for heterogeneous Image Annotation MPEG-7 Mario Döller Dublin Core … Alps Mario Döller …

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Current Standardization Efforts MPEG – Query Format (ISO/IEC SC29 WG11) (ISO/IEC ) JPEG - JPSearch (ISO/IEC SC29 WG1) (ISO/IEC 24800) W3C – Media Annotations Working Group (

MPEG – Query Format (ISO/IEC SC29 WG11) (ISO/IEC )

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch The MPEG Query Format (MPQF) International Standard since end of 2008 standardizes messages from and to multimedia repositories and provides extended functionalities for service discovery, service selection and service capability description. General Concepts obases on XML and is defined by an XML Schema odecoupled from any other metadata standard (also MPEG-7) osupport for any XML based MM metadata description ointegration of limited XQuery functionality oMPQF divided into 3 main categories Management Input Query Format Output Query Format

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Scenario

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Concepts Query I MPQF supports: oSynchronous/Asynch ronous mode oTimeout functionality MPQF combines: oExact matches oFuzzy requests How to query MMRS satisfactorily? Query Design

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Concepts Query - Condition assign preferenceValue and thresholdValue to every condition assign scoringFunction to every „Boolean Operator“ (AND, OR, XOR) (recommended to follow t-norm, t- conorm rules) result in rank and confidence evaluation for every item

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Examples Management I request: give me all available MMRS! request: give me all available MMRS fitting to my desired requirements

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Examples Query I Browsing Query QueryByFree Text

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Examples Assume: DB contains images annotated with the Corine Land Cover specification combined with MPEG-7. Example images show industrial or commercial units (121) in the area of Sines/Portugal [European ]

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Example Spatial Query … <SpatialRelation relationType="urn:...:SpatialRelationCS:2008:south" sourceResource="ID1"> … Give me all satellite images that show an industry unit in the south of something else! ID1

JPEG - JPSearch (ISO/IEC SC29 WG1) (ISO/IEC 24800) current status

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch JPSearch Objectives provide a standard for interoperability for image search and retrieval systems oby defining the interfaces and protocols for data exchange between them provide an abstract framework and flexible search architecture that allows: oadding, updating or querying metadata of images and image collections ofederated search across different systems othe integration of best-of-breed independent search components, provided by different companies

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch JPSearch Overall Structure

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 2: Registration, Identification and Management of Metadata Schema (1) Schema identification: oidentified by a URI and XML Namespace Schema and Transformation Rules management oCentral authority hosted by JPEG Create a single core schema Definition of Transformation Rules operform semantic, structural and syntactic mapping rules between XML-encoded metadata descriptions from different formats

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Workflow of a JPSearch request

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Query Transformation

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch AIR: ARCHITECTURE FOR INTEROPERABLE RETRIEVAL ON DISTRIBUTED AND HETEROGENEOUS MULTIMEDIA REPOSITORIES

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Summary - Outlook Introduced MPEG Query Format and the JPSearch approach for improving interoperability during multimedia retrieval Future Work oEstablish the standards for cultural heritage projects?

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Questions ?

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 3: JPSearch Query Format (FCD) Derived from MPEG Query Format Only minor changes (namespaces, etc.) Restrictions: oNo TemporalQueryType oonly image domain allowed Modifications: oQueryByROI oQueryByMedia oMIME-Type

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 4: JPSearch File Format (CD) Extension of JPEG-1/JPEG2000 file format Fully compatible to JPEG-1/JPEG2000 and provides additional functionality carrying associated metadata within a file Overall structure of JPEG-1-compliant version of JPSearch file format.

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 5: Data Interchange Format between Image Repositories (CD) should be able to perform synchronization and consolidation/aggregation of repositories Synchronization of: oMeta part: to identify the content of data part oData part: image, collection of images, metadata, ontology or URI Relys on image collection format of ISO/IEC (Photo player MAF), MP4

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Workflow of a JPSearch system (1) 1.JPQF query receives 2.Metadata based on Core Schema is transformed to the metadata of the N native schemas 3.Additional Metadata is transformed to N native Schemas (if possible else the information is discarded) 4.N times Query transformation (optional) 5.Forward N queries to N native systems where the interpreter transforms it to the native query language and executes it 6.Transforms N individual result sets to core schema. 7.Aggregate N result sets to 1 result set 8.Forward final result set to user

Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch AIR: Planned search concepts