MMI: Defining & Doing Marine Metadata Interoperability John Graybeal & Luis Bermudez Marine Metadata Interoperability Project Monterey Bay Aquarium Research Institute 10 October, 2005

Marine Metadata Interoparability Initiative 1 What is data interoperability?  The science use case I want to evaluate a condition, model, sensor, or measurement in a given region and time Please find and acquire the related data (items) given my criteria: region, time, other… Give all this data back to me the way I want it Using my terms, units, and reference frames Classically, in rows (time) and columns (items) I want to iterate on the search/access results And presto! There it is!

Marine Metadata Interoparability Initiative 2 Interoperability: The metadata challenges Wait—what happened to the data challenges? We’re going to ignore this for today, because At one level, it’s easier then metadata (sort of) A lot of teams and services work on it Metadata describes the data (carries it along?) It isn’t our problem. We’re doing Marine Metadata So let’s consider the metadata interoperability challenges in our use case.

Marine Metadata Interoparability Initiative 3 What’s needed for marine metadata interoperability? We need community agreement in several areas. The way to transfer the metadata: Transport Protocol What data to transfer: Content Standard What the contents mean: Vocabulary Transport Protocol Content Standard Vocabulary

Marine Metadata Interoparability Initiative 4 We have some of these— so where’s the problem?

Marine Metadata Interoparability Initiative 5 We have some of these— so where’s the problem? (1) EML ISO DCMI FGDC MarineXML GML ADL NetCDF ASCII ContentContent ProtocolProtocol ESML OPeNDAP REST SOAPZ39.50 DFDL THREDDS LAS WxS HTTP HDF TIF JPEG MetadataData Coards/CF

Marine Metadata Interoparability Initiative 6 We have some of these— so where’s the problem? (2) Search for sea water temp data Water Temperature sea_water_ temperature TEMP BODC GCMD CF Only matches

Marine Metadata Interoparability Initiative 7 Hurdles in marine metadata Not recognizing the fundamental problems Common goals not appreciated Little reuse of previous work No way to work together Discover collaborators Actually collaborate Dissimilar approaches and understandings (technologists, scientists, educators, leaders) ”Not invented here" attitudes

Marine Metadata Interoparability Initiative 8 How do we go forward?

Marine Metadata Interoparability Initiative 9 We need a community. And we need to create agreements.

Marine Metadata Interoparability Initiative 10 Building a digital community: How to create agreement First: Create a community Discover common ground Identify technical gaps and pursue them Identify issues Recommend workable approaches to proceed Resolve issues Finalize common understandings Iterate

Marine Metadata Interoparability Initiative 11 Building a digital community —around metadata Find people who see the problems and want to work together to fix them Get funding to support basic infrastructure, distributed resources, technical leadership

Marine Metadata Interoparability Initiative 12 Building a digital community —around metadata Create a community center

Marine Metadata Interoparability Initiative 13 Discovering common ground Use community center as a starting point. One place with all the information References Reviews, comments News, events Content provider A meeting place for people, projects, and organizations.

Marine Metadata Interoparability Initiative 14 Identifying technical gaps— Vocabularies Few formal marine vocabularies exist There was no agreed format or location Most weren’t comprehensive or detailed There was no way to connect them There was no architecture to nurture them Like children, they need encouragement to grow healthy and strong, and to play well with others Water Temperature sea_water_ temperature TEMP BODC GCMD CF

Marine Metadata Interoparability Initiative 15 Controlled Vocabulary Formally managed, community-specific terminology that collectively represents a specialized vocabulary for a community. Benefits: Helps to avoid misspellings and avoid the use of arbitrary words that cause inconsistencies. Helps to solve semantic incompatibilities among distributed systems.

Marine Metadata Interoparability Initiative 16 Controlled Vocabulary Two types: Parameter Discovery Vocabulary (PDV) Used to guide the user to find data Examples: GCMD, BODC Discovery, AGU Index Terms Parameter Usage Vocabulary (PUV) Used to tag the data collected Examples: U.S. JGOFS, BODC, CF These distinctions are similar to, but not the same as, “semantic” and “syntactic”

Marine Metadata Interoparability Initiative 17 Semantic Interoperability Distributed and heterogeneous systems are able to solve issues related to controlled vocabulary.

Marine Metadata Interoparability Initiative 18 Vocabularies: Discovery and harmonization DTD CommaSeparatedValues HTML TabSeparatedValues RelationalDatabase XML/XSD RDF OWL

Marine Metadata Interoparability Initiative 19 ~ 60 ontologies are available at

Marine Metadata Interoparability Initiative 20 Vocabularies: Harmonization with Voc2OWL

Marine Metadata Interoparability Initiative 21 Classes and individuals Parameter beam1_velocity wind_speed beam2_echo_intensity Class Looks like a Real world objects Individuals beam1_velocity wind_speed beam2_echo_intensity

Marine Metadata Interoparability Initiative 22 Properties

Marine Metadata Interoparability Initiative 23 Minimum set of properties

Marine Metadata Interoparability Initiative 24 Vocabularies: Adding details and structure How to maintain hierarchies? Many vocabularies are hierarchical, like habitats or taxonomies Voc2OWL handles this important and common semantic information; VINE is still evolving How to encourage adding definitions? Virtually all of the 60 vocabularies lack definitions Mapping workshop participants felt the pain Should this be a goal of our tools? Or the vocabulary owners? Perhaps both?

Marine Metadata Interoparability Initiative 25 Relating Vocabularies: Mapping tool: Got VINE? Vocabulary INtegration Environment

Marine Metadata Interoparability Initiative 26 Alignment and mapping Main relations: Same as (owl:sameAs) Inverse, symmetric and transitive Narrower Than (map:narrowerThan) Transitive, Inverse of map:broaderThan) Broader Than (map:broaderThan) Transitive, Inverse of map:narrowerThan)

Marine Metadata Interoparability Initiative 27 Relating Vocabularies: Mapping Workshop Results

Marine Metadata Interoparability Initiative 28

Marine Metadata Interoparability Initiative 29 Relating Vocabularies: The community’s role

Marine Metadata Interoparability Initiative 30 Using Vocabularies: Services

Marine Metadata Interoparability Initiative 31 The complete solution: Semantic mediation Water Temperature sea_water_ temperature TEMP BODC GCMD CF Search for sea water temp data Finds all the data sets!

Marine Metadata Interoparability Initiative 32 Identifying issues Given we are a focal point, this is not a problem. Issues come to MMI is a forum 3 biggies: guidance, recommendations, standards We are expected to solve the issues Featured in ORION and IOOS cyberinfrastructure efforts Fortunately, goals and needs overlap greatly We aim to empower the community to solve its issues

Marine Metadata Interoparability Initiative 33 The complete solution: Semantic mediation architecture Semantic Mediation Services Repositories Clients ?

Marine Metadata Interoparability Initiative 34 Workable approaches Part 1: Demonstrations “Show us some useful results of this technology.” This can take three forms: A. Just a quick example of what it can do. B. Something that reflects its interoperability and some system interactions. C. A real demonstration of the capabilities, scaled up with multiple partners.

Marine Metadata Interoparability Initiative 35 Quick Example: Mapped Terms

Marine Metadata Interoparability Initiative 36 Demonstrating Systems: Semantic Interoperability

Marine Metadata Interoparability Initiative 37

Marine Metadata Interoparability Initiative 38 Scaling up to the real world: MMI Demonstration 1) Vocabulary Harmonization 2) Vocabulary Mapping 3) Vocabulary Services 4) Access to Data

Marine Metadata Interoparability Initiative 39 Workable approaches Part 2: Processes toward consensus The mapping workshop: successful for most Carefully considered all aspects of workshop Domains and teams (targeted audience) Training and roles Goals and processes Now, making it a process model for others Can this process be extended? Vocabulary mapping scales to a community. What about content standards? (crosswalks) Transport protocols? (web services?)

Marine Metadata Interoparability Initiative 40 Interoperability: Is the science use case nearly achieved? “Find and acquire the related data, given my constraints.” Some rough solutions sketched out & demo’d Many agreements still needed Data discovery, data exchange Metadata (MMI): transport protocols, content standards, and vocabularies Specialized content standards: units, quality control, process descriptions One can see a path through the woods (toward a light at the end of the tunnel…)

Marine Metadata Interoparability Initiative 41 Wrapup: The MMI Project This is our one-year anniversary talk! NSF starter funding, with SURA(ONR) in-kind support, NOAA CSC bridge funds International contributions and support Main deliverables: web site, and a community Goal for future: Solve the metadata problem At least, creating effective processes to improve it Asking GEON, and its contributors, to work with us Identifying strategies and partners for operations

Marine Metadata Interoparability Initiative 42 MMI as of Aug 2005 ~ 200 members. > 600 documents available. First place in Google search for "marine metadata”. 8 major presentations in first year (e.g. DMAC, NEPTUNE DMAS, OBIS). 10 virtual tutorials given related to marine ontologies, tools and web services. One workshop “Advancing Domain Vocabularies” August 2005, Boulder, CO. Tools: VINE, VOC2OWL, Ontology Web Services and application to query distributed repositories.

Marine Metadata Interoparability Initiative 43 MMI Popularity Approaching 200 members

Marine Metadata Interoparability Initiative 44 Our Guides Roy Lowry, BODC Robert Arko, LDEO Julie Bosch, NOAA Ben Domenico, Unidata Karen Stocks, SDSC Steve Hankin, NOAA - Ocean.US/DMAC Mark Musen, Stanford Univ Michael Parke, Univ of Hawaii Lola Olsen, NASA Goddard Bob Weller, WHOI Dawn Wright, Oregon State University Steering Committee Executive Committee John Graybeal, MBARI. (PI) Philip Bogden, SURA/SCOOP Stephen Miller, SIO. Francisco Chavez, MBARI. Stephanie Watson, Texas A&M

Marine Metadata Interoparability Initiative 45 MMI: Your Handy Reference Guide MMI: Help Line: Post an event: Post an item: Get info re site: Workshop: Ontologies: Term Search: Demo: