Integrating Distributed Data Systems Using Ontologies, Web Services and Standards: An MMI Case Study John Graybeal, Luis Bermudez, Kevin Gomes, Michael Godin December 6, 2005

Marine Metadata Interoperability Initiative 1 The data systems MBARI-developed data systems Shore Side Data System (SSDS) Adaptive Ocean Sampling Network (AOSN) Pre-existing systems SSDS in development for 5 years AOSN built for 2003 campaign Different but overlapping users Different but overlapping data

Marine Metadata Interoperability Initiative 2 SSDS

Marine Metadata Interoperability Initiative 3 AOSN

Marine Metadata Interoperability Initiative 4 The problem SSDS AOSN get me Data for Variable ocean_temperature (C) get me Data for Parameter temperature_1 (deg C)

Marine Metadata Interoperability Initiative 5 This is (in part) a metadata problem

Marine Metadata Interoperability Initiative 6 The Marine Metadata Initiative NSF starter funding, with SURA(ONR) in-kind support, NOAA CSC bridge funds International contributions and support Main deliverables: web site, and a community Goal for future: Solve the metadata problem At least, create effective processes to improve it Asking world-wide contributors to work with us Identifying strategies and partners for operations

Marine Metadata Interoperability Initiative 7 MMI as of Dec ~ 200 members > 800 documents available First place in Google search for "marine metadata” 11 major presentations in first year IOOS DMAC, OBIS, Oceans, GEON, OOSTech, AGU, Quartod 10 virtual tutorials given related to marine ontologies, tools and web services One workshop “Advancing Domain Vocabularies” (August 2005, Boulder, CO) Tools: VINE, VOC2OWL, Ontology Web Services, Tethys, Tethys-Axis Most are available at Sourceforge

Marine Metadata Interoperability Initiative 8 Data life cycle Data Source Data Provider Data User gets processes: formats/archives publishes gets processes: uses/analyzes sends gets processes: formats/archives sends

Marine Metadata Interoperability Initiative 9 The answer: Community Agreements A way to map different vocabularies A set of agreed services to Use the mappings Find the data of interest Get the data Tethys MMI Interoperability Demonstration

Marine Metadata Interoperability Initiative 10 First: Understanding each other vocabularies via vocabulary mappings

Marine Metadata Interoperability Initiative 11 Tethys Framework Data Producer Mappings 1.Convert the parameters, data sources, units and flags used in their system to an ontology (tool Voc2OWL changes ASCII to OWL). 2.Map the terms used in the system to the following widely used standard vocabularies for discovery (GCMD) and for usage (CF), which are available as ontologies at MMI. 3.Register your maps with the Tethys catalog.

Marine Metadata Interoperability Initiative 12 Conversion Harmonize: Convert dictionary of parameters, sources, units and flags to the Web Ontology Language (OWL), that is a common format to express vocabularies/ontologies (recommended by W3C) DTD CommaSeparatedValues HTML TabSeparatedValues RelationalDatabase XML/XSD RDF OWL

Marine Metadata Interoperability Initiative 13 Achieve fast conversions VOC2OWL can help you ! - More about IN41B-07 Thursday at 09:30

Marine Metadata Interoperability Initiative 14 Doing semantics by mapping terms

Marine Metadata Interoperability Initiative 15 Doing semantic mapping fast VINE can help you !

Marine Metadata Interoperability Initiative 16 We’ve tried this! A Domain Mapping Workshop

Marine Metadata Interoperability Initiative 17 Now, we use the mappings

Marine Metadata Interoperability Initiative 18 Tethys Framework Semantic Mediation View 1.Data producer submits mappings to semantic mediator as an ontology file. 2.User asks the semantic mediator for terms related to the concept of interest 3.Semantic mediator looks up mappings Responds with related terms used by other vocabularies (= data producers) Other mediators may provide different relationships

Marine Metadata Interoperability Initiative 19 Tethys Portal: User Interface

Marine Metadata Interoperability Initiative 20 Finding the related data

Marine Metadata Interoperability Initiative 21 Tethys: SOAP based WSDL files describe the available methods

Marine Metadata Interoperability Initiative 22 Tethys Portal - Service Broker Knows about the Data Producers Type of terms the data producer is using End point of the SOAP Web Service

Marine Metadata Interoperability Initiative 23 Tethys Portal: User Interface

Marine Metadata Interoperability Initiative 24 Tethys: Methods getMetadata arguments: parameter, source, start, end, downLimit, upLimit, northLimit, westLimit, southLimit, eastLimit, verticalPositive returns: Dublin Core Metadata in RDF/XML note: An ID of the data is returned as URIDataSet

Marine Metadata Interoperability Initiative 25 Getting the related data

Marine Metadata Interoperability Initiative 26 Tethys: Methods getMetadata arguments: parameter, source, start, end, downLimit, upLimit, northLimit, westLimit, southLimit, eastLimit, verticalPositive returns: Dublin Core Metadata in RDF/XML note: An ID of the data is returned as URIDataSet getASCII arguments: URIDataSet, start, end, downLimit, upLimit, northLimit, westLimit, southLimit, eastLimit, verticalPositive returns: An ASCII file (1 header, tab separated)

Marine Metadata Interoperability Initiative 27 Invoking MMI Ontology Web Services to get sameAS and narrowerThan terms for: Ocean Temperature. Found Temperature_8 in the SSDS ontology. Found sea_surface_temperature in the AOSN onto. Looking for SSDS in the Tethys catalogue. Found SSDS SOAP Web Service endpoint at: Invoking the method getMetadata with the parameter: Temperature_8. Found 4 data sets, metadata added to the portal cart. Looking for AOSN in the Tethys Catalogue. … Tethys Portal in Action

Marine Metadata Interoperability Initiative 28 Results (based on Dublin Core Metadata) Variable Sea Surface Temperature in data set nps_aircraft (download plot) Autonomous Ocean Sampling Network 2003 Field Campaign data (AOSN-II). By Data Source/Observation Data/Aircraft/Sea Surface … Creator is: Naval Postgraduate School (NPS), Monterey CA Time range is: from: to: Coordinates are: north: east: south: west: ID: urn:aosn.mbari.org.recordVariable.id: Tethys Portal Search Results

Marine Metadata Interoperability Initiative 29 Tethys Portal Gets the Data Invokes the method getASCII with the URIDataset urn:aosn.mbari.org.recordVariable.id:243483, possibly refining the query in time and space. ASCII data is returned: timevertical (m down)latitude (degrees_north) longitude (degrees_east)sea_surface_temperature (C) T20:34: T20:36: T20:37: T20:38: ……

Marine Metadata Interoperability Initiative 30 Tethys Framework: Status Mapping processes and tools tested in workshop and with SSDS and AOSN. The methods (interfaces) were defined and iterated using SSDS and AOSN systems. Ontologies and mappings to date are available via Onrtology Web Services. We worked hard to make this easy for you. Helper methods (JAVA API) to create metadata were developed. Tethys-AXIS created to quickly deploy web services in Tomcat. Documentation… a lot of documentation

Marine Metadata Interoperability Initiative 31 Conclusions Interoperability requires semantic solutions Fastest (best?) answer mixes web services and semantic mappings Domain-specific mappings are inevitable (so, go ahead and start doing them) Later, full ontologies can be integrated Initial prototype software available now You can participate in the demo at the link below Many data providers plan to work with the demo

Marine Metadata Interoperability Initiative 32 Our Guides Roy Lowry, BODC Robert Arko, LDEO Julie Bosch, NOAA Ben Domenico, Unidata Karen Stocks, SDSC Steve Hankin, NOAA - Ocean.US/DMAC Mark Musen, Stanford Univ Michael Parke, Univ of Hawaii Lola Olsen, NASA Goddard Bob Weller, WHOI Dawn Wright, Oregon State University Steering Committee Executive Committee John Graybeal, MBARI. (PI) Philip Bogden, SURA/SCOOP Stephen Miller, SIO. Francisco Chavez, MBARI. Stephanie Watson, Texas A&M

Marine Metadata Interoperability Initiative 33 MMI: Your Handy Reference Guide MMI: Help Line: Post an event: Post an item: Get info re site: Workshop: Ontologies: Term Search: Tethys:

Marine Metadata Interoperability Initiative 34 Thank you Luis Bermudez

Marine Metadata Interoperability Initiative 35 Tethys Framework Portal View 1.Receive data producer registrations:Receive SOAP endpoint of the data producer Ontology (mappings) of the data producer’s terms Types of terms (e.g., “CF”) understood by the data producer 2.Receive semantic mediator registrationsReceive SOAP endpoint where you can query the mappings (SeRQL, an RDF query language) Different mediators may provide different content 3.Receive user requests for (meta)data, invoke semantic mediators to learn relevant terms, query data producers using their accepted terms, return (meta)dataReceiveinvokequeryreturn