Biological and Chemical Oceanography Data Management Office slide 1 of 21 Interoperability ~ An Introduction Cyndy Chandler Biological and Chemical Oceanography Data Management Office (BCO-DMO) Woods Hole Oceanographic Institution 26 July 2008 OOS Interoperability Planning Workshop National Space Science and Technology Center University of Alabama at Huntsville, AL USA

Biological and Chemical Oceanography Data Management Office slide 2 of 21 Discussion Points What do we mean by “interoperability”? Why does it matter? How do we achieve interoperability?  What are some of the expected challenges and effective strategies?

Biological and Chemical Oceanography Data Management Office slide 3 of 21

Biological and Chemical Oceanography Data Management Office slide 4 of 21 What do we mean by “interoperability”? the ability to exchange data and information between two or more systems (separated by a recognized boundary) with minimal loss of information  clients of interoperable systems may be machines syntactic interoperability  file formats  data structures  exchange protocols semantic interoperability  term definitions (controlled vocabularies)

Biological and Chemical Oceanography Data Management Office slide 5 of 21 Interoperability... a wee bit more from MMI a client should not be required to possess in depth understanding of the data in order to access them interoperable systems should be designed to support machine access (not just people clients) interoperable metadata systems should be designed to support automated, accurate, lossless, machine-to- machine exchange of information

Biological and Chemical Oceanography Data Management Office slide 6 of 21 Why does interoperability matter? improved open access to public data  enabling data mining, cataloging systems, portals, mash-ups ecosystem researchers are asking global questions model developers need access to data from a variety of domains (biology, chemistry, etc and social and economic data too)  we must rise to the challenge of designing interoperable data systems that are up to these tasks

Biological and Chemical Oceanography Data Management Office slide 7 of 21 How do we achieve interoperability? What are some of the expected challenges? varied funding sources and governing agencies projects with different priorities and leadership distributed data systems large volume, heterogeneous data expectation of near real-time availability

Biological and Chemical Oceanography Data Management Office slide 8 of 21 adopt standards use controlled vocabularies publish metadata databases use Semantic Web technologies build community What are some of the effective strategies? *

Biological and Chemical Oceanography Data Management Office slide 9 of 21 This is going to be difficult... best solutions will be found through community recognition of local implementations Baker & Chandler, in press. publication expected in DSR II before end of Title: Enabling Long-Term Oceanographic Research: Changing Data Practices, Information Management Strategies and Informatics online: final draft complete: 9-JUL-2008http://dx.doi.org/ /j.dsr

Biological and Chemical Oceanography Data Management Office slide 10 of 21 The data … CMarZ GEOTRACES Iron Synthesis NACP OCB US GLOBEC US JGOFS US SOLAS controlled vocabularies community building metadata standards *

Biological and Chemical Oceanography Data Management Office slide 11 of 21 Ocean Observatory Data … expecting large volume, heterogeneous data to be made available in near real time in the past, large volume data sets tended to be more homogeneous (e.g. fewer columns); while data sets with many columns tended to be smaller in size HomogeneousHeterogeneous Sensor dataManual observations PhysicalBiology, Chemistry Data from ocean observatory sensor arrays are/will be a continuous stream of large volume, heterogeneous data.

Biological and Chemical Oceanography Data Management Office slide 12 of 21 Ocean Observatory Data … attempting to reduce this challenge by  identifying 20 IOOS core variables  recognizing the importance of metadata  identifying common use cases  adopting common standards (OGC and SWE)  fostering community-wide involvement and communications  sponsoring mysterious, after-dark gatherings in Huntsville, AL providing little more information beyond street address and suggested footwear... *

Biological and Chemical Oceanography Data Management Office slide 13 of 21 Semantic Web Semantic Web technologies offer one solution set next four slides are courtesy of Peter Fox (HAO/ESSL/NCAR) and are from an April 2008 presentation

Biological and Chemical Oceanography Data Management Office slide 14 of 21 Semantic Web Basics The triple: {subject-predicate-object} Interferometer is-a optical instrument Optical instrument has focal length An ontology is a representation of this knowledge W3C is the primary (but not sole) governing organization for languages, specifications, best practices, etc.  RDF - Resource Description Framework  OWL Ontology Web Language (OWL 1.1 on the way) Encode the knowledge in triples, in a triple-store, software is built to traverse the semantic network, it can be queried or reasoned upon Put semantics between/ in your interfaces, i.e. between layers and components in your architecture, i.e. between ‘users’ and ‘information’ to mediate the exchange (P. Fox, 2008)

Biological and Chemical Oceanography Data Management Office slide 15 of 21 (P.Fox, 2008) … … VO Portal Web Serv. VO API DB 2 DB 3 DB n DB 1 Semantic mediation layer - VSTO - low level Semantic mediation layer - mid-upper-level Education, clearinghouses, other services, disciplines, etc. Metadata, schema, data Query, access and use of data Semantic query, hypothesis and inference Semantic interoperability Added value Mediation Layer Ontology - capturing concepts of Parameters, Instruments, Date/Time, Data Product (and associated classes, properties) and Service Classes Maps queries to underlying data Generates access requests for metadata, data Allows queries, reasoning, analysis, new hypothesis generation, testing, explanation, etc. *

Biological and Chemical Oceanography Data Management Office slide 16 of 21 Semantic Web Methodology and Technology Development Process Establish and improve a well-defined methodology vision for Semantic Technology based application development Leverage controlled vocabularies, etc. Use Case Small Team, mixed skills Analysis Adopt Technology Approach Leverage Technology Infrastructure Rapid Prototype Open World: Evolve, Iterate, Redesign, Redeploy Use Tools Science/Expert Review & Iteration Develop model/ontology (P. Fox, 2008)

Biological and Chemical Oceanography Data Management Office slide 17 of 21 (P. Fox, 2008) The Information Era: Interoperability managing and accessing large data sets higher space/time resolution capabilities rapid response requirements data assimilation into models crossing disciplinary boundaries. Modern information and communications technologies are creating an “interoperable” information era in which ready access to data and information can be truly universal. Open access to data and services enables us to meet the new challenges of understanding the Earth and its space environment as a complex system:

Biological and Chemical Oceanography Data Management Office slide 18 of 21 Semantic Web technologies being used in some of Peter Fox’s informatics projects: Semantically-Enabled Science Data Integration (SESDI): Semantic Knowledge Integration Framework (SKIF): Semantic Web for Earth and Environmental Terminology (SWEET): SWEET is a JPL project to provide a common semantic framework for a variety of Earth science initiatives.

Biological and Chemical Oceanography Data Management Office slide 19 of 21 more challenges... reality check: insufficient metadata … researcher aboard R/V Oceanus, August 2008, talking about metadata for shipboard sampling Data originators recognize that recording metadata is a good idea, and they will even record metadata ~ if we make it easy for them to do so. * "I feel like I'm so busy with all the other little things in a cruise that I don't have time to worry about making up log sheets. But if someone puts an already made log sheet in front of me, I'll use it." "When I see projects with good metadata, it just makes my heart go pitter patter."

Biological and Chemical Oceanography Data Management Office slide 20 of 21 more challenges... reality check: heterogeneous data sets … The BCO-DMO database includes many data sets in which depth and temperature are reported and more than 25 different definitions of each.

Biological and Chemical Oceanography Data Management Office slide 21 of 21 * Community building Share knowledge gained doing implementation at the local level – participate in workshops and publish Fall 2008 AGU Meeting in San Francisco, CA 8-12 December; Earth and Space Science Informatics IN12: Strategies for Improved Marine and Synergistic Data Access and Interoperability IN19: From Data to Synthesis: Next-Generation Science Applications Abstract Submission Deadline: 10 September 2359 UT Thank you...