OTN Data Warehouse Overview: Data Workflow Data Warehouse and Integration Data Access OTN Canada Symposium, Data Management Workshop, June 2, 2011 (SUB 224)

OTN Data Workflow Diagram OTN HQ Plone Folder Instrument Metadata OTN Postgresql Database Project Metadata Collaborator Plone Folders Collection Metadata Google Earth The OTN Data Workflow include: spreadsheet templates for capture of collection metadata, PLONE project folders for by collaborator and by mission organization of data and metadata files (further described in How to Submit Collection Metadata), PostgreSQL database schemas and tables for integrating data and metadata, GoogleEarth maps for visualizing the database contents and comma separated (CSV) outputs files data extractions from the database. To date, the primary visualization focus has been on receiver data, tagging data products are currently under development. Instrument Logs OTN Project Plone Data Links Geoserver WMS, WFS WMS/KML CSV Products WFS/CSV

OTN Data Warehouse and Integration Major processing steps include: Field collection spreadsheets and VUE export files are converted to CSV files and loaded “as is” to raw database tables (for specific collaborator schema); Raw database tables are loaded into processed (mostly type casting) tables and then views (for specific collaborator schema). Processed tables/views are loaded into OBIS like tables (for specific collaborator schema). Then collaborator schemas are rolled up into OBIS Schema views. Additional OBIS schema tables/views for metadata and internal use. OBIS schema views are then used to create GEOSERVER schema views for use in creating Geoserver Layers. The Geoserver Layers (Store otn) are then created using the OTN Postgresql database GEOSERVER Schema views. Only two of the Geoserver Layers (resources_points, moorings) may be accessed without a valid Geoserver username/password. Individual collaborator’s data are kept in their own password protected PostgreSQL schema.

OTN Data Access Though the Geoserver Layers were created as Proof of Concept and are under considerable development there are URLs on the OTN Project Plone site which are using these layers. OTN Project Plone Data folder includes publically accessible specially constructed KMZ/KML. URLs for some of the OTN Geoserver Layers have been posted on the OTN Project Plone Data folder under CSV product links (OTN Site Members only accessible). All but resources_points include filter for collaborator/resource (e.g. Halifax Line filter where collectioncode=’HFX’). Only resources_points and moorings are available without Geoserver username/password

Ocean Tracking Network Portal Members Site: http://www.marinebiodiversity.ca/OTN Most data access currently through Members’ Site. Access to OTN Data Sheets, OTN Regional and Project Metadata, OTN Collaborator Metadata and Data Repository, OTN videos, and more...

OTN Project Plone Data PUBLIC – just click on Data tab and the Publically available items are displayed, for example the GE KMZ.

OTN Project Plone CSV Products Under Construction There are also site members’s only access, requiring login, for example to see the CSV products. Need to login to see more including CSV Products.

OTN Project Plone CSV Products Under Construction Others are being developed as part of Metadata Catalogue Protype that Bob will be demoing shortly.