1. OTN Data Warehouse Overview: Data Workflow
Data Warehouse and Integration
Data Access
OTN Canada Symposium, Data Management Workshop, June 2, 2011 (SUB 224)

2. OTN Data Workflow Diagram
OTN HQ Plone Folder
Instrument Metadata
OTN Postgresql Database
Project Metadata
Collaborator Plone Folders
Collection Metadata
Google Earth
The OTN Data Workflow include: spreadsheet templates for capture of collection metadata, PLONE project folders for by collaborator and by mission organization of data and metadata files (further described in How to Submit Collection Metadata), PostgreSQL database schemas and tables for integrating data and metadata, GoogleEarth maps for visualizing the database contents and comma separated (CSV) outputs files data extractions from the database.
To date, the primary visualization focus has been on receiver data, tagging data products are currently under development.
Instrument Logs
OTN Project Plone Data Links
Geoserver
WMS, WFS
WMS/KML
CSV Products
WFS/CSV

3. OTN Data Warehouse and Integration
Major processing steps include:
Field collection spreadsheets and VUE export files are converted to CSV files and loaded “as is” to raw database tables (for specific collaborator schema);
Raw database tables are loaded into processed (mostly type casting) tables and then views (for specific collaborator schema).
Processed tables/views are loaded into OBIS like tables (for specific collaborator schema).
Then collaborator schemas are rolled up into OBIS Schema views.
Additional OBIS schema tables/views for metadata and internal use.
OBIS schema views are then used to create GEOSERVER schema views for use in creating Geoserver Layers.
The Geoserver Layers (Store otn) are then created using the OTN Postgresql database GEOSERVER Schema views.
Only two of the Geoserver Layers (resources_points, moorings) may be accessed without a valid Geoserver username/password.
Individual collaborator’s data are kept in their own password protected PostgreSQL schema.

4. OTN Data Access
Though the Geoserver Layers were created as Proof of Concept and are under considerable development there are URLs on the OTN Project Plone site which are using these layers.
OTN Project Plone Data folder includes publically accessible specially constructed KMZ/KML.
URLs for some of the OTN Geoserver Layers have been posted on the OTN Project Plone Data folder under CSV product links (OTN Site Members only accessible). All but resources_points include filter for collaborator/resource (e.g. Halifax Line filter where collectioncode=’HFX’). Only resources_points and moorings are available without Geoserver username/password

5. Ocean Tracking Network Portal
Members Site:
Most data access currently through Members’ Site.
Access to OTN Data Sheets, OTN Regional and Project Metadata,
OTN Collaborator Metadata and Data Repository, OTN videos, and more...

6. OTN Project Plone Data
PUBLIC – just click on Data tab and the Publically available items are displayed, for example the GE KMZ.

7. OTN Project Plone CSV Products
Under Construction
There are also site members’s only access, requiring login, for example to see the CSV products.
Need to login to see more including CSV Products.

8. OTN Project Plone CSV Products
Under Construction
Others are being developed as part of Metadata Catalogue Protype that Bob will be demoing shortly.