A Provisional Observational Data Standard to Facilitate Data Sharing and Aggregation Lynn Kutner, Bruce Stein, and Donna Reynolds TDWG Annual Meeting, St. Louis, Missouri, USA October 2006

Topics  Context and definitions  Development of the standard  challenges and goals  process  requirements  Overview of the standard  conceptual data model  required and core fields

Who We Are:  A nonprofit organization providing the scientific information and technology to guide effective conservation action  A network of 80 natural heritage programs in the United States, Canada, and Latin America, supported by NatureServe’s staff of science, information technology, and conservation professionals. What We Do:  Collect, manage, and disseminate detailed local information on species and ecosystems, and develop products, tools, and conservation services to help meet conservation needs at all geographic levels NatureServe

Data collection and recording Data management and reporting Conservation expertise and analysis Information access and interoperability Decision support Scientific standards and methods Conservation Decisions Standards: The Foundation NatureServe is a cosponsor of the TDWG Observational Data Subgroup.

Newest team member: Sam, with mom Lynn

Context and Definitions

Characteristics of an Observation  Fundamental unit of scientific inventory  Can be documented by various types of evidence, including voucher specimens, photographs, recordings, or sight records  Ancillary data may include abundance, distribution, reproductive status, phenology ecological associations, environmental conditions, and much more  Can be linked to other observations by common factors such as time, place, and protocol – the framework for monitoring

What is an Observation? An observation characterizes evidence for the presence or absence of an organism or set of organisms through a data collection event at a location.

What is Monitoring? Monitoring is an activity that results in a set of observational data gathered during repeated visits over a period of time using consistent or comparable methodology, for the purpose of detecting and documenting changes and trends.

Development of the Standard

Primary Challenge The Observational Data Standard must encompasses data sets that differ widely in content and quality  discipline-specific data  historical data  positive and negative (absence) data  data on single species as well as ecological communities

Goal and Purpose  Goal: Collaboratively identify a core set of concepts that are broadly applicable regardless of data or survey type  Purpose: Facilitate data sharing and aggregation within the conservation community, including data discovery through global search portals.

Process  Project began in mid-2005 with funding from Gordon and Betty Moore Foundation  Comparison of attributes among 16 observation databases to identify breadth and commonalities  Observation Working Group—multi-institutional composition; formed in March 2005 and held two-day workshop in June 2005  Identified and debated key issues  Drafted set of core concepts and requirements  Production of drafts for two rounds of review (April and June 2006), with subsequent incorporation of comments  Version 1.0 presented September 2006

Core Observation Concepts  What – attributes that identify the observed element  Where - location attributes  When - observation / collection event date / time  Who - observer / collector of the information  How - survey methods / effort / protocol  Evidence – documentation attributes  Biology - associated biological attributes  Environment - associated environmental attributes  Data management attributes – data sensitivity, quality control, etc.

Requirements The Observational Data Standard shall:  Focus on “core” attributes, including a subset of required fields.  Incorporate and be compatible with existing standards.  Apply to data from a variety of survey protocols.  Accommodate monitoring data.  Allow documentation of locations where a species was not observed (absence data).  Accommodate species and community data.  Provide ability to add to core attributes for specific groups of users (extensibility).  Allow users to record a list of specify on a plot.  Include attributes indicating data quality and validation.  Be independent of a particular software implementation.

Content of the Observational Data Standard

Major Entities  Observation – Contains the required information that defines an observation and optional biological and environmental attributes  Observation Grouping - A set of observations grouped according to some common criteria  Protocol – The plan or procedure used to collect data; may link to internal or external references.  Survey - Coordinated effort to gather information; multiple observations  Project - May link related surveys  Search Area – May link to single observation or survey  External Documentation – Pointer to information that is managed in a different system

Major Entity Relationships

Observation Cardinalities  An Observation may be related to zero or one Survey.  An Observation may be related to zero or one Protocol.  An Observation may be related to zero or one GIS Shape.  An Observation may be related to zero, one, or many External Documentations.  An Observation may be related to zero or one Element List.  An Observation may be related to zero, one, or many Observation Grouping.  An Observation may be related to zero or one Search Area.

Minimum Requirements for an Observation Record  Formal name (scientific name) Calypte anna  Where found (GIS shape, coordinates, or text) New Mexico: Eddy County, 3 mi S of Artesia along Rt. 285  Date (to year(s), at least) 30 November 1978  Observer(s) / collector(s), or “Unknown” D.J. Davis

Ranking of Attributes  Required – cannot be null, although data may be incomplete  Core – important information but not required (may not be available or applicable)  Priority Core – a subset of core attributes that should always be filled in if data are available  Additional – supplementary data

Identification Attributes

Observation Grouping  Name or label for observation grouping  Criteria - the common characteristic or other criteria used to group the observations (same element, same location, or any other criteria)  Owner - the person who created the observation grouping  Monitoring comments – changes over time and trends within the grouping

Species List  For community plot data  List of the species found and data such as stratum and abundance (e.g., percent cover or number)  Do not have to make an observation record for each species  May make an observation record for each species

Documentation  Every entity can link to internal or external reference  Link to materials via IDs in another data system  Specimens  Protocol descriptions  Project details

Next Steps  Standard will be utilized in current and future software development projects at NatureServe:  Kestrel Project – a web-enabled implementation of observations database being developed for Parks Canada; includes proof of concept of discipline- specific extensions  Handheld field data collection tool - funding received from NSF  Development of XML schema and mapping to existing TDWG standards

Questions / Discussion