1. Digital Preservation: The Multimedia Standards way Mario Döller Assistant Professor University of Passau, Germany 1st International Digital Preservation Interoperability Framework (DPIF) Symposium

2. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Agenda Heterogeneity in Digital Preservation and Multimedia Retrieval in general Selected solutions based on Multimedia Standards oMPEG Query Format oJPEG JPSearch Conclusion

3. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Digital Preservation (DP) Efforts National Programs (US, EU, Australia, etc.) oNDIIPP, RAMA, ORION, CASPAR, PLANETS, PADI, PANIC, … Digital Preservation Europe (DPE) oimprove collaboration and synergies between existing preservation initiatives across Europe Developed Metadata Formats oGeneral: MPEG-7, Dublin Core, … oDP: VRA (Visual Resources Association) Core 4.0, CIDOC Conceptual Reference Model (CRM, (ISO 21127:2006) ), museumdat, …

4. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Multimedia Retrieval Current Situation

5. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Example for heterogeneous Image Annotation MPEG-7 Mario Döller Dublin Core … Alps Mario Döller …

6. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Current Standardization Efforts MPEG – Query Format (ISO/IEC SC29 WG11) (ISO/IEC 15938-12) JPEG - JPSearch (ISO/IEC SC29 WG1) (ISO/IEC 24800) W3C – Media Annotations Working Group (http://www.w3.org/2008/WebVideo/Annotations/)

7. MPEG – Query Format (ISO/IEC SC29 WG11) (ISO/IEC 15938-12)

8. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch The MPEG Query Format (MPQF) International Standard since end of 2008 standardizes messages from and to multimedia repositories and provides extended functionalities for service discovery, service selection and service capability description. General Concepts obases on XML and is defined by an XML Schema odecoupled from any other metadata standard (also MPEG-7) osupport for any XML based MM metadata description ointegration of limited XQuery functionality oMPQF divided into 3 main categories Management Input Query Format Output Query Format

9. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Scenario

10. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Concepts Query I MPQF supports: oSynchronous/Asynch ronous mode oTimeout functionality MPQF combines: oExact matches oFuzzy requests How to query MMRS satisfactorily? Query Design

11. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Concepts Query - Condition assign preferenceValue and thresholdValue to every condition assign scoringFunction to every „Boolean Operator“ (AND, OR, XOR) (recommended to follow t-norm, t- conorm rules) result in rank and confidence evaluation for every item

12. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Examples Management I request: give me all available MMRS! request: give me all available MMRS fitting to my desired requirements

13. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Examples Query I Browsing Query QueryByFree Text

14. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Examples Assume: DB contains images annotated with the Corine Land Cover specification combined with MPEG-7. Example images show industrial or commercial units (121) in the area of Sines/Portugal [European ]

15. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch MPQF Example Spatial Query … <SpatialRelation relationType="urn:...:SpatialRelationCS:2008:south" sourceResource="ID1"> … Give me all satellite images that show an industry unit in the south of something else! ID1

16. JPEG - JPSearch (ISO/IEC SC29 WG1) (ISO/IEC 24800) current status

17. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch JPSearch Objectives provide a standard for interoperability for image search and retrieval systems oby defining the interfaces and protocols for data exchange between them provide an abstract framework and flexible search architecture that allows: oadding, updating or querying metadata of images and image collections ofederated search across different systems othe integration of best-of-breed independent search components, provided by different companies

18. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch JPSearch Overall Structure

19. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 2: Registration, Identification and Management of Metadata Schema (1) Schema identification: oidentified by a URI and XML Namespace Schema and Transformation Rules management oCentral authority hosted by JPEG Create a single core schema Definition of Transformation Rules operform semantic, structural and syntactic mapping rules between XML-encoded metadata descriptions from different formats

20. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Workflow of a JPSearch request

21. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Query Transformation

22. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch AIR: ARCHITECTURE FOR INTEROPERABLE RETRIEVAL ON DISTRIBUTED AND HETEROGENEOUS MULTIMEDIA REPOSITORIES

23. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Summary - Outlook Introduced MPEG Query Format and the JPSearch approach for improving interoperability during multimedia retrieval Future Work oEstablish the standards for cultural heritage projects?

24. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Questions ?

25. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 3: JPSearch Query Format (FCD) Derived from MPEG Query Format Only minor changes (namespaces, etc.) Restrictions: oNo TemporalQueryType oonly image domain allowed Modifications: oQueryByROI oQueryByMedia oMIME-Type

26. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 4: JPSearch File Format (CD) Extension of JPEG-1/JPEG2000 file format Fully compatible to JPEG-1/JPEG2000 and provides additional functionality carrying associated metadata within a file Overall structure of JPEG-1-compliant version of JPSearch file format.

27. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Part 5: Data Interchange Format between Image Repositories (CD) should be able to perform synchronization and consolidation/aggregation of repositories Synchronization of: oMeta part: to identify the content of data part oData part: image, collection of images, metadata, ontology or URI Relys on image collection format of ISO/IEC 23000-3 (Photo player MAF), MP4

28. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch Workflow of a JPSearch system (1) 1.JPQF query receives 2.Metadata based on Core Schema is transformed to the metadata of the N native schemas 3.Additional Metadata is transformed to N native Schemas (if possible else the information is discarded) 4.N times Query transformation (optional) 5.Forward N queries to N native systems where the interpreter transforms it to the native query language and executes it 6.Transforms N individual result sets to core schema. 7.Aggregate N result sets to 1 result set 8.Forward final result set to user

29. Universität Passau, Lehrstuhl für Verteilte Informationssysteme Prof. Dr. Harald Kosch AIR: Planned search concepts