Remote Instrumentation and Sensor Networks on the Great Barrier Reef CIMA based sensor networks… Ian Atkinson

New Technologies… Sensor Networks: will enable us to monitor our environment more closely than ever before. Grid computing: will enable the distribution of data collected to researchers around the globe in real time, as well as storing the massive amounts of historical data in online repositories. The challenge: Streamlining the transfer of data from sensors to researchers and managers.

Major Projects 1.ReefGrid 2.Remote Instrument Monitoring

RoadNet: SIO, SDSC, UCSD

Frank Vernon, SIO

Rutgers Cool Room: Costal Observatories

ReefGrid ReefGrid is a pilot project to develop the technologies to allow real time sensor network monitoring of the Great Barrier Reef (GBR). NCRIS-Integrated Marine Observation System GBR monitoring program Encompasses the entire e-Research data flow Sensors Data Aggregatio n Data Transpo rt Data QA/QC Data Ingestion and Storage Data Dissemination Foundation for QOOS: Queensland Oceans Observing System

Davies Davies Reef is the location for the initial deployment of a sensor network on the GBR Davies Reef is ~100km East of Townsville. Data has to be sent 80km to AIMS.

GBR Network? Local Communications Networks

Davies AIMS’ Weather Station at Davies Reeg is being used as a base for the sensor network. A hybrid power system with wind and solar energy and battery back up powers the system.

Davies The Sensors A range of sensor technologies is being used. The most important two are 1-Wire Sensors and Ambient Systems’ Wireless uNodes These technologies provide low cost sensors that can be deployed in large numbers covering a wide area. Data Aggregatio n Data Transpo rt Data QA/Q C Data Ingestion and Storage Data Disseminatio n Sensors

Costal Radars (CODAR) HF ocean surface radar to monitor surface currents, wind directions and wave heights Capricorn and Bunker Group of reefs and islands in the southern part of the GBR 100 x100 km, 3km pixel April 2006 deployment Mal Heron, JCU

Davies Data Aggregation Data Aggregatio n Data Transpor t Data QA/Q C Data Ingestion and Storage Data Disseminatio n Sensors Software tools that allow us to organise many different types of sensing devices for simple reference, access & probing. System is scalable so that many sensors can be added. Individual Sensors Data Aggregation Data Sinks

Sensor Gateway: Use Case To add / remove / re-configure a sensor –Make the change at the hardware level –Update the configuration database Sink Source Root Sensor –Push the new configuration file to the appropriate sink See

Davies Data Transport Sensors Data Aggregatio n Data Transpo rt Data QA/Q C Data Ingestion and Storage Data Disseminatio n Two aspects of Data Transport: 1.the physical connection to sensor networks many 10’s of km off the coast 2.the protocols for interacting with the sensor network (retrieving data and interrogating the system).

Davies Data Transport Sensors Data Aggregatio n Data Transpor t Data QA/Q C Data Ingestion and Storage Data Disseminatio n microwave link to provide data communications between the sensor network at Davies Reef and the mainland. "evaporation duct" just above the ocean, allows microwaves to propagate around the curved surface of the Earth. removes the need for a line-of-site connection between the two ends of the microwave link.

JAINIS Davies Data Transport Sensors Data Aggregatio n Data Transpo rt Data QA/Q C Data Ingestion and Storage Data Disseminatio n The protocols for communications with the Grid are tied to an overall dataflow that includes QA/QC and ingestion of data into storage facilities. JAINIS (J CU A nd IN diana University I nstrument S ervices ). CIMA provides a consistent and reusable framework for including shared instrument resources distributed Grids. Several enhancements have been made to the CIMA implementation

JAINIS Davies JAINIS Sensors Data Aggregatio n Data Transpo rt Data QA/Q C Data Ingestion and Storage Data Disseminatio n JAINIS has a structure that has an Instrument Representative that allows the instrument (or data sink in ReefGrid) to be represented as a web service. This web service can be accessed by a Data Manager to control the flow of data through QA/QC and preprocessing to a data repository. The Web Services can also be used to stream live data and to provide interactive control over the sensor network.

Davies JAINIS CIMA Instrument Representative Data Manager CIMA SOAP Interface GridSphere Portal SRB Data Repository (data and metadata) Kepler mediated Experimental logic SRB MCAT PGL Applications/ Processing Data Sinks/ Sensors CIMA core code from IU JCU/DART reengineered CIMA code SDSC Components Workflow & computational Pipelines, Automatic metatdata generation Live Data Feeds, Instrument Control

Davies Data Manager Data Aggregatio n Data Transpo rt Data QA/Q C Data Ingestion and Storage Data Disseminatio n Sensors JAINIS uses Kepler as its Data Manager. Kepler provides a graphical user interface allowing the user to define workflows. Metadata can be automatically generated in Kepler based on input data from the Instrument Representative

Davies Data Manager Data Aggregatio n Data Transpo rt Data QA/Q C Data Ingestion and Storage Data Disseminatio n Sensors QA/QC routines are performed in Kepler. Preprocessing can also be performed in Kepler (e.g. converting units of a measurement) Kepler can controls the location where data is stored (creating file names and directories, metadata)

Davies Data Ingestion and Storage Data Aggregatio n Data Transpor t Data QA/Q C Data Ingestion and Storage Data Disseminatio n Sensors In the JAINIS implementation, the data is stored using SRB (Storage Resource Broker). Heterogenous storage devices act as single logical storage Metadata Catalogue Service (MCAT) –Stores metadata associated with objects –Metadata is vital for finding data in large data stores, as well as describing how the data was collected, what QA/QC and what pre-processing has been done

Personal Grid Library: PGL Implementation of a web-based digital library (Gridsphere portlets) Provides an interface to templated metadata manipulation –Define object classes (Images, reports, datasets...) –Define appropriate metadata tags –Define metadata display template Uses SRB to manage the digital objects and the metadata Exists by placing a metadata template file in a directory - used by interface to render library

PGL: Interface and metadata/annotations

Davies Data Ingestion and Storage Data Aggregatio n Data Transpor t Data QA/Q C Data Ingestion and Storage Data Disseminatio n Sensors Integrating SRB with content management systems (plone).

JCU e-Research and Sensor Network Collaborations ARC Research Network on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP). ARC Research Network for Materials and Molecular Sciences (MMSN). Key partner of Queensland Cyber-Infrastructure Foundation (QCIF). Australian Partnership for Advanced Computing (APAC). Ambient Systems and University of Twenté (Netherlands) Indiana University (USA). San Diego Supercomputing Centre. EM Solutions (Brisbane).

Sensor Data Flows Summary ––Wireless data links Sensor Networks Temp, salinity, Light, turbidity, etc. Data ‘Sinks’ Internet Microwave links - via humidity ducting SRB MCAT Data Repository Disk/Tape Storage (multiple locations) Data Sinks enable Simple sensor configuration Interface Scalability / ease of maintenance Model for ‘plug and play’ sensor configuration Security Automatically update backend data structures Developed by DART for Davies Reef SN Platform Network & sensor monitoring AODC-JF & other portals Data Manager CIMA SOAP Interface Kepler mediated Experimental logic Workflow & computational pipelines, Automatic metatdata generation Local and remote Grid computing Compute resources Custom portals / Research codes Sensor Devices Inexpensive, commodity inspired Ad hoc network Above and below water wireless network & wired Many partners involved Great Barrier Reef - Ocean Observing System

Registry and directory server Data Acquisition Code(s) Register Notify Web Services interface Request/ Response Events/ Streaming Locate Analysis Codes Storage Instrument Semantics, Metadata Microwave link

DIMSIM: Distributed Integrated Multi-Sensor and Instrument Middleware