SQL/MM Part 8: Metadata Registry (MDR) ISO/IEC JTC 1/SC 32 Plenary Jeju, Korea, June 22, 2009 Dongwon Jeong Dept. of Informatics & Statistics, Kunsan National Univ. | Working Group 4 – SQL Multimedia & Application Packages 32N1893

Contents  Background  Motivation  Introduction to SQL/MM MDR  Concluding Remark  Q/A 2 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

3  Background  Motivation  Introduction to SQL/MM MDR  Concluding Remark

4 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary Metadata  For ISO/IEC 11179, metadata is defined to be data that defines and describes other data. This means that metadata are data, and data become metadata when they are used in this way. The set of circumstances, purposes, or perspectives for which some data are used as metadata is called the context. So, metadata are data about data in some context. personName Name: personName Definition: a name to identify human Data type: String Max size: 20 Min size: Dongwon Jeong DATA METADATA

5 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary Purposes of ISO/IEC – Metadata Registry  The purposes of ISO/IEC – Metadata Registry are to promote the following: Standard description of data Common understanding of data across organizational elements and between organizations Re-use and standardization of data over time, space, and applications Harmonization and standardization of data within an organization and across organizations Management of the components of data Re-use of the components of data

6 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary High-level metamodel of ISO/IEC  High-level overview of the central regions of the metamodel

7 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary High-level metamodel of ISO/IEC 11179: Example  High-level overview of the central regions of the metamodel Country name Country name code Countries ISO Alpha Code

MDR Implementations  There are various MDR Implementations  Ref.: 8 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary OrganizationRegistryURL Australian Institute of Health and Welfare Metadata Online Registry (METeOR) Canadian Institute for Health Information CIHI Data Dictionaryhttp://secure.cihi.ca/ddexternal/welc ome.do UK Cancer GridCancer Grid Metadata Registryhttps:// :8443/exist/re st/db/mdr/web/homepage.xquery US National Cancer InstituteCancer Data Standards Repository (caDSR) tructure/cacore_overview/cadsr US Environmental Protection AgencyEnvironmental Data Registryhttp:// US Health organizations (multiple)US Health Information Knowledgebase (USHIK) U.S. Department of Homeland Security (DHS) and U.S. Department of Justice (DOJ) US National Information Exchange Model (NIEM) US Department of JusticeGlobal Justice XML Data Model (GJXDM) id=43

9  Background  Motivation  Introduction to SQL/MM MDR  Concluding Remarks

Issues 10 Various MDR systems (instances) Get metadata information from any registries? Various MDR systems (instances) Get metadata information from any registries? Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary Designing a database schema using metadata of metadata registries Developing an integrated metadata registry systems Making a metadata search system based on various metadata registries Providing a standardized inference for users to use a registry

Issues (cont.) 11 Various MDR systems (instances)? Get metadata information from any registries in a consistent/standardized manner? Various MDR systems (instances)? Get metadata information from any registries in a consistent/standardized manner? Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary Designing a database schema using metadata of metadata registries Developing an integrated metadata registry systems Making a metadata search system based on various metadata registries Providing a standardized inference for users to use my registry How can we achieve this goal? (obtain metadata from MDRs) A Consistent/Standard access method is required

Issues (cont.) 12 Various MDR systems (instances)? Get metadata information from any registries in a consistent/standardized manner? Various MDR systems (instances)? Get metadata information from any registries in a consistent/standardized manner? Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary Designing a database schema using metadata of metadata registries Developing an integrated metadata registry systems Making a metadata search system based on various metadata registries Providing a standardized inference for users to use my registry No Standardized Access Method !!! (ISO/IEC does not provide) No Standardized Access Method !!! (ISO/IEC does not provide)

Issues (cont.) 13 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary No method to build valid MDRs strictly following the standard (No mandatory attributes; Misuse) MDR systems have been developed with different access methods each other (Heterogeneity between MDR management systems) Difficult to realize an integrated metadata registry system (Integration of relevant and similar MDRs) Inconsistency Issue between MDRs (Incompatibility issue of metadata) Need to harmonize various MDRs (MDR Integration: Tightly or Loosely Coupled) Need to get metadata from one or more MDRs (for building a database according to predefined DEs)

Key Issues to be Resolved 14 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary What Issues should we resolve?

Key Issues to be Resolved (cont.) 15 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary How to guarantee valid metadata registry building How to get metadata from various registries - How to define registry structure & insert, delete, and modify metadata - DDL and DML (except retrieval language) in SQL - How to retrieve metadata from registries in a consistent way - Retrieval language in SQL

Possible Approaches 16 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary How to resolve the issues?

Possible Approaches (cont.)  Interface Mapping between MDR Management Systems Required for implementation of one-to-one adaptor  High cost Not guarantee metadata registries to be valid  Standardized Binding API-based Approach ISO/IEC (Program level) Every system should implement functions (bodies) for the binding APIs  High cost Not guarantee metadata registries to be valid  Direct DB Access Users should know the schema structures of MDRs  Security problem Not guarantee valid metadata registries  Standardized Query Language-oriented Approach: Our Approach SQL/MM Standards Approach – User-defined type and Routines Can guarantee valid metadata registries to be built 17 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

18  Background  Motivation  Introduction to SQL/MM MDR  Concluding Remarks

19 Discussion History SC32 Joint Study Periods Meetings (WG2,3,4), Clearwater, FL, February 2007 SC32 Plenary and WG Meetings, New York, USA, July 2007 First presentation on this standard item Simple Introduction to SQL/MDR (Idea description) Request for a study period on SQL/MM MDR Documents: 32N1656-WG4N0061-jfk0023-studyrequest-sql_mdr-jfk.pdf & jfk0024-wg4-presentation-sql-mdr.ppt Presentation title: SQL/MDR: Query Language for Sharing and Exchanging of Metadata between MDRs - Proposal for a Study Period in WG4 – WG 4 Meeting, Jagsthausen, Germany, November 2007 A tough draft has been provided Documents: str015 wg4-sqlmm-mdr.doc Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

20 Discussion History (cont.) SC32 Plenary and WG Meetings, Sydney, Australia, May 2008 November 15, 2008: Approved Proposed NWIP Resolutions: SC321737a SC 32 approves the following subdivisions and SC32 instructs its secretariat to conduct a 60-day letter ballot for Project , and accept the project subdivision if at least four National Bodies support the subdivision. SC32N1809: Five NBs approved SC 32/WG 4 Interim Meeting, Portugal, November 2008 Project NumberTitleProject EditorRationale Doc # SQL Multimedia and Application Packages – Part 8: MDR Dongwon Jeong WG4N0070 (rationale 3) Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary In addition, Open Forums and Workshops

Purpose  The purpose of this International Standard is to define multimedia and application specific types and their associated routines for retrieval of metadata using the user-defined features in ISO/IEC 9075  ISO/IEC SQL/MM is structured as a multi-part standard, and at present, it consists of the following parts: Part 1: Framework Part 2: Full-Text Part 3: Spatial Part 5: Still Image Part 6 : Data Mining Part 7: History Part 8: MDR 21 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

Scope  This Part of ISO/IEC introduces the metadata registry part of ISO/IEC 13249, gives the references necessary for this part of ISO/IEC 13249, defines terms, notations, conventions, and definitions specific to this part of ISO/IEC 13249, defines concepts specific to this part of ISO/IEC 13249, defines metadata registry user-defined types and associated routines, covers specifications for retrieval of metadata of registries does not covers specifications for insertion, deletion, modification of metadata including creation of schema structures for registries /** According to the comment of the WG 2 experts, the Workshop on ROR, On-Demand Model Selection and SMMP, September 2008, Wuhan, China */ 22 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

Overview  Registries have been used for systematic management of metadata describing data A variety of registry frameworks has been developed for applications fields, and many registries have been built for management of metadata.  For valid and exact registry exchanging and sharing, the following problems should be resolved: Inconsistent access method Difficulty of registry management system development  This part therefore aims to provide a standard retrieval method  This part enables to obtain metadata in a consistent way from metadata registries, and facilitates usage of the standard. 23 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

Overview (cont.)  This part includes the specifications for management of metadata registries, and the specifications are defined in the same way with SQL packages such as SQL/MM Spatial, SQL/MM Mining, SQL/MM Still Image, and so on  This part covers the definition of operational architectures and processes for the consistent retrieval that can support transparency regardless of registries with different database structures  This part specifies user-defined features for the ISO/IEC standard A user can define an MDR schema using the user-defined types provided in this part This standard can lead the valid MDR creation because the user-defined types reflect all information of the ISO/IEC standard The user-defined types can also enable management and access of metadata. 24 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

Conceptual Model 25 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary User-Defined Types Metadata Definition Retrieve Metadata from Metadata Registries Registry Storage Model (Tables/Views) Registry Storage Model (Tables/Views) Metadata Retrieval Routines (Methods, Functions) Metadata Update association Scope call

Definition of User-Defined Types and Routines  MDR User-Defined Types An MDR user-defined type is mapped to its corresponding MDR class An example of MDR classes = {data element, conceptual domain,...}  Type Definition Form The MDR_ type is an abstraction for classes of metamodel of ISO/IEC For example, the user-defined type for the class ‘Data_Element’ of the ISO/IEC metamodel is defined as follows: 26 MDR_ type Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary MDR_Data_Element

Definition of User-Defined Types and Routines (cont.)  Create a UDT Statement CREATE TYPE MDR_Data_Element AS ( ) STATIC METHOD MDR_administered_item_administration_record() RETURNS TABLE( ), STATIC METHOD MDR_data_element() RETURNS TABLE( ), METHOD MDR_data_element_name() RETURNS 27 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

Definition of User-Defined Types and Routines (cont.)  Definition of the Static Method MDR_data_element() Obtain all data of data element CREATE STATIC METHOD MDR_data_element() RETURNS TABLE( ) FOR MDR_data_element BEGIN RETURN TABLE( SELECT FROM user_de_table); END 28 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary user_de_table is the table name that is defined by a user. i.e., BEGIN-END part should be implemented by users. user_de_table is the table name that is defined by a user. i.e., BEGIN-END part should be implemented by users.

Definition of User-Defined Types and Routines (cont.)  Output Parameters of the Static Method MDR_data_element() data_element_administration_record MDR_Administration_Record, representation_class_qualifier CHARACTER VARYING(30), data_element_precision Integer  The attribute data_element_administration_record is the structured type MDR_Administration_Record 29 12th Annual Open Forum for Metadata Registries (OFMR2009)

Retrieve Metadata  Select Statement SELECT DE.data_element_name, DE.status FROM TABLE (MDR_Data_Element::MDR_data_element()) AS DE WHERE DE.data_element_administration_record.administrative_status() = 'Recorded’; 30 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary In the SELECT clause, data_element_name and status are defined based on attribute names in the metamodel of ISO/IEC 11179

Concluding Remark  Contributions Ease of use Familiarity to users Consistent and Standardized access method – Independent query description on physical structures (table structures) – Low modeling cost (decreases query description time) – Simplicity of query statement for accessing distributed registries – Low complexity for distributed query composition  In its application aspect, this standard contributes as follows. can be used Unified Access Method to Various Registries provides Unified query and Unified query result form can be used for developing distributed registries management system can be used as a communication protocol (method) for developing semi- automatic tools: Registries Mapping and Integrating tools can do easy data Integration (among databases following their own registry) 31 Tutorial of SQL/MM Part 8 - Metadata Registry (MDR), ISO/IEC JTC 1/SC 32 Plenary

Dongwon Jeong Information Sciences & Technology Laboratory, Informatics & Statistics Department, Kunsan National University Q/AQ/A 32