IPT + Darwin Core OBIS XML Schema OBIS Database Schema Explained Mike Flavell OBIS Data Manager OBIS Nodes Training Course, Oostende, Belgium, 6 May 2014

Overview Summary / Background Darwin Core Terms that Map to the OBIS Schema IPT Server Issues

Summary / Background Why do we need Darwin Core? Datasets originate from different surveys, different methodologies, different fields We need to exchange datasets in a common format, quickly with appropriate metadata. Data Provider => OBIS Nodes => iOBIS …

What is Darwin Core? The Darwin Core is a body of standards. Includes a glossary of terms (properties, elements, fields, columns, attributes, or concepts) Intended to facilitate the sharing of information about biological diversity by providing reference definitions, examples, and commentaries. Primarily based on taxa, their occurrence in nature as documented by observations, specimens, samples, and related information. –See for more informationhttp://rs.tdwg.org/dwc/

History of Darwin Core The name "Darwin Core" was first used in 1998 by Biology Implementers Group (ZBIG) while commenting on the profile's conceptual similarity with Dublin Core. Later expressed as an XML Schema document for use by the Distributed Generic Information Retrieval (DiGIR) protocol. A TDWG task group was created to revise the Darwin Core, and a ratified metadata standard was officially released on 9 October 2009.

History of Darwin Core Darwin Core has had numerous previous versions in production usage. NameNamespaceNumber of termsDate Issued Darwin Core 1.0Not Applicable Darwin Core 1.2 (Classic) ual/darwin/2003/ /09/01 Darwin Core 1.21 (MaNIS/HerpNet/ORNIS /FishNet2) ual/darwin/2003/ /03/03 Darwin Core OBIS /07/05 Darwin Core 1.4 (Draft Standard) /07/05 Darwin Core Terms (properties) /10/09

History of Darwin Core What is OBIS’s use of Darwin Core? –Previously: DiGIR + OBIS XML Schema For Data Transfer –Now: IPT For XML Transfer Darwin Core was Ratified by “TDWG” on 9 October 2009 that version is often referred to a ratified Darwin Core

Biodiversity Information Standards (TDWG) Also known as the Taxonomic Databases Working Group Not for profit scientific and educational association that is affiliated with the International Union of Biological Sciences. Formed to establish international collaboration among biological database projects. Promoted the wider and more effective dissemination of information about the World's heritage of biological organisms for the benefit of the world at large. Focuses on the development of standards for the exchange of biological/biodiversity data.

Technologies for Data Exchange

In the past there was DiGIR… Data From Provider DiGIR Server iOBIS Database Record based XML Request/Respo nse using the OBIS XML Schema ? Schema Based on Darwin Core 1.2 Relatively Slow and Complex to Obtain Data DiGIR Designed for Distributed Database Queries The OBIS Database Schema was originally desgined for data provided via DiGIR in the OBIS XML Schema format.

Now there is IPT… Data From Provider IPT Server iOBIS Database Dataset Based XML Request/Response using ratified Darwin Core and Extensions OBIS Node Managers & Helpers XML Schema is based on ratified Darwin Core and Extensions Relatively Fast and Simple to Obtain Data iOBIS needs to map from ratified Darwin Core and Extensions to it’s Database Schema that was designed for DiGIR based data

What is IPT? An application that runs within Apache Tomcat Server Hosting / publishing Darwin Core Archive Files Darwin Core Archive = Biodiversity Data and Metadata packaged as a ZIP File Developed by GBIF

IPT Server Take a look at

Mapping Data From Providers Data From Provider IPT Server iOBIS Database Dataset Based XML Request/Response using ratified Darwin Core and Extensions OBIS Node Manager s & Helpers Datasets from Providers must be “mapped” to ratified Darwin Core for it to be uploaded onto an IPT Server iOBIS will only consume selected Darwin Core Terms during its harvesting Process. OBIS Node Mangers must understand the Providers Dataset and what is it possible to populate iOBIS with using Darwin Core

Need for a Resource to Help Mapping & know what iOBIS can handle i.e. Schema Mapping Mapping Data From Providers Please take a copy of rDwC(IPT) to iOBIS Field Mapping – concise version.xslx to your desktop / file system. Inspect it for current terms that can be mapped to iOBIS

Mapping Issues Fields that were mapped in DiGIR that do not have a Darwin Core Term Date Fields => eventDate Need for Enhancement to use “dynamic Properties” or “measurement or Fact” Start/End Longitude/Latitude & Date/Time => eventDate?

Issues / Problems with Mapping from rDwC to the OBIS Schema. Mapping Data From Providers Please take a copy of rDwC(IPT) to iOBIS Field Mapping – detailed version.xslx to your desktop / file system.