Exchanging Environmental Data for the Gulf of Maine 2007 Exchange Network Collaboration Grant Prep webinar for development meeting October 28th-29th, 2008 at GoMOOS in Portland, ME

Purpose of Webinar Provide more information/refresher on grant tasks Prepare attendees for Oct th mtg at GoMOOS Provide a heads up on information needed from data providers prior to meeting

Review of Grant Tasks *Tasks highlighted in blue will be basis for agenda items at meeting. Maybe add web application development.

Node client setup – Info needed prior to meeting about dataset(s) List of fields, their sizes and types Frequency at which data is collected Data collection method - manually, automatic, both... Data storage (database, Excel, paper...) QA/QC process (How long does it take before data is ready to be released/updated?) Is historical data revised over time? Will node data submission process be done manually or automatically? How much legacy data will be submitted? (Number of records in data set) A survey will be sent out via for data providers to fill out and return prior to meeting on Oct 28-29, 2008.

Data Submission Create XML from Partner Data Submit to Node via HTTPS (Firewall friendly) GoMOOS to assist with node client setup

Data Exchange Template Development Will use: - Exchange Network (EN) data standards - Water Quality Exchange (WQX) 2.0 data exchange template - any other standards/templates/schemas to help build template (NBII, JGOF, Environment Canada, IOOS, modeling, others?) - can add things to template not covered by data standards - consider different types of data from partners: physical/chemical, biological, spatial, modeling, buoy etc. - do have the option of using the WQX as is

Data Standards Basics in Relation to the WQX Template Broken down into these data blocks: Organization – name of contact/owner/source of the data, address, phone, etc. Project – name of sampling program, purpose for obtaining the data (project description) etc. Monitoring Location – lat/long/elevation of where the data was obtained, type of location (ocean, pipe, river, etc.), how located, etc. Activity – date collected, by whom, depth, equipment, sample collection method info, etc. Analysis and results – data result including analytical method, lab info, detection limits etc. These and other exchange network data standards are available at:

Monitoring Location Excerpt from Data Standard

Monitoring Location Excerpt from WQX Template We will be creating a data exchange template like this.

Given this, our mission is to: Look at the data standards and the WQX data exchange template and decide what we want for data elements Look at other applicable standards/schemas/templates See if can fit in data not designed as part of WQX such as modeling and spatial coverages (if desired) Use the WQX as a starting point –Add data elements from data standards –Subtract data elements –Create data elements not in data standards if needed –Determine domain lists, required data, optional data etc. –WQX 2.0 data exchange template available at:

Developing the data exchange template Things to consider: 1) What data elements are needed for the intended general purpose web application? 2) What elements could be populated easily and by most partners for the particular type of data? – Do not want a data element that will be rarely populated or is difficult to obtain. 3) A smaller schema is more likely to be used by more partners than an extensive one. – but there has to be enough info to make the retrieval useful – need a balance 4) How will your data will fit into the template (mapping)? 5) How do we deal with changes to the template or addition of new data sets?

Mapping your data to the template Can you supply data which meets: –domain list values –required elements –size/type requirements –metadata requirements

Creating the XML file Will you need assistance building an XML file? Do you have the software to do it or do you need a tool?

What to send when? Building a trading partner agreement - When do you expect to send your data to GoMOOS node? - when it is updated - on a specific frequency (e.g. every Wed at 2:00AM) - What data will you send? - specific data set(s) - do you have the authority to send data? - Will be developing trading partner agreements that specify these terms.

Schema Development Meeting October 28-29, 2008 at GoMOOS office in Portland, ME Goals: –Understand node and node client setup –Build the schema Evaluate the WQX schema, make modifications if necessary to accommodate partner datasets Review/crosswalk to other schemas –Develop trading partner agreements – Web application discussion and development (if time allows) What do partners need to make this meeting the most useful?

In advance of the meeting Provide examples of your data and descriptions of data elements via survey Review data standards and WQX 2.0 and think about changes you would suggest Think about how your data might fit into the schema See if your org can develop an XML file Think about what data you will send and when Respond to meeting notice request if plan on attending Will send follow-up regarding these items.

Proposed Schedule for Grant

Resources Exchange Network Data Standards: listing.htm WQX 2.0 Data Exchange Template: water/wqx.htm water/wqx.htm Trading Partner Agreement Example tpa.htm tpa.htm

Questions?