1 1 Observatory Middleware Framework (OMF): Enterprise Service Bus for Environmental Observatories Duane R Edgington MBARI (Monterey Bay Aquarium Research Institute) Moss Landing, California, USA Duane R Edgington MBARI (Monterey Bay Aquarium Research Institute) Moss Landing, California, USA INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

Contributors Randal Butler Terry Fleury Von Welch Duane Edgington Kevin Gomes John Graybeal Bob Herlien Scripps Institution of Oceanography (Scripps) Randal Butler Terry Fleury Von Welch Duane Edgington Kevin Gomes John Graybeal Bob Herlien Scripps Institution of Oceanography (Scripps) 2 Observatory Middleware Framework (OMF): Enterprise Service Bus for Environmental Observatories INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

3 Agenda today Demonstrate a USA activity to prototype cyberinfrastructure (CI) in support of earth observatories INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

Ocean Observatories Initiative USA National Science Foundation next- generation, decadal-scale program. 10 year, 330M$ US observing and interacting with the oceans. The Initiative has four basic elements Regional: cluster of experimental sites Global: widely spaced, deep- ocean, full-ocean water-column moorings, Coastal: focused on the variable, steep gradients of the near-shore Cyberinfrastructure: an integrative data management system. USA National Science Foundation next- generation, decadal-scale program. 10 year, 330M$ US observing and interacting with the oceans. The Initiative has four basic elements Regional: cluster of experimental sites Global: widely spaced, deep- ocean, full-ocean water-column moorings, Coastal: focused on the variable, steep gradients of the near-shore Cyberinfrastructure: an integrative data management system. 4 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

5 Ocean Observing Initiative CyberInfrastructure –Start with experiences and systems developed for single ocean observatories (isolated Moored or Cabled to shore). –Designing for: Collaborative, web access Utilizing Service Oriented Architecture Incorporating Enterprise Service Bus technology Grid enabled –Requirements: Multiple, diverse platforms (cabled to shore, moored surface expressions, solar powered, diesel powered, associated autonomous mobile platforms) Wide geographic distribution Thousands of instruments Distributed data Interactive control (near) real-time data acquisition Event response –Start with experiences and systems developed for single ocean observatories (isolated Moored or Cabled to shore). –Designing for: Collaborative, web access Utilizing Service Oriented Architecture Incorporating Enterprise Service Bus technology Grid enabled –Requirements: Multiple, diverse platforms (cabled to shore, moored surface expressions, solar powered, diesel powered, associated autonomous mobile platforms) Wide geographic distribution Thousands of instruments Distributed data Interactive control (near) real-time data acquisition Event response INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

Regional Scale Node The Regional Scale Nodes will enable new approaches to studies of Ocean and Earth processes that, for example: regulate global climate store anthropogenic carbon form many mineral deposits support major fish stocks govern rainfall patterns on land threaten coastlines with storms, tsunamis, harmful algal blooms, and oxygen depletion deliver sediment and pollutants from land to the oceans create and recycle Earth’s crust, and harbor novel microbial life forms within subseafloor volcanic environments. The Regional Scale Nodes will enable new approaches to studies of Ocean and Earth processes that, for example: regulate global climate store anthropogenic carbon form many mineral deposits support major fish stocks govern rainfall patterns on land threaten coastlines with storms, tsunamis, harmful algal blooms, and oxygen depletion deliver sediment and pollutants from land to the oceans create and recycle Earth’s crust, and harbor novel microbial life forms within subseafloor volcanic environments. 6 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

Global and Coastal nodes 7 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

8 Facilitating direct & immediate interaction with Ocean Connecting & coordinating operations between Observatories and with Science & Education Providing scientists with the capability to observe and respond to emergent conditions in the ocean Facilitating direct & immediate interaction with Ocean Connecting & coordinating operations between Observatories and with Science & Education Providing scientists with the capability to observe and respond to emergent conditions in the ocean INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

9 OOI CI Scenarios (Use Cases) The project's architecture will be configured for certain basic scenarios, including: –Monitor and control a single observatory, or multiple observatories; –Detect and respond to an event; –Fuse data from an observatory with a pre-existing ocean model; –Design field experiments; –Create 'virtual' observatories by combining components distributed among multiple physical observatories. The project's architecture will be configured for certain basic scenarios, including: –Monitor and control a single observatory, or multiple observatories; –Detect and respond to an event; –Fuse data from an observatory with a pre-existing ocean model; –Design field experiments; –Create 'virtual' observatories by combining components distributed among multiple physical observatories. INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

10 OOI CI Architecture Features Leverage the integrative principles of modern, service-oriented computer architecture Adopt Enterprise Service Bus, to integrate sensors, storage, scientific laboratories and computing Enable Grid Computing to integrate data with advanced ocean models and visualization Reuse the same software over many scales -- coastal, regional and global Leverage the integrative principles of modern, service-oriented computer architecture Adopt Enterprise Service Bus, to integrate sensors, storage, scientific laboratories and computing Enable Grid Computing to integrate data with advanced ocean models and visualization Reuse the same software over many scales -- coastal, regional and global INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

11 Ocean Observatory Schematic TeraGrid Open Science Grid INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

Observatory Middleware Framework (OMF) Researching alternative approaches  extend beyond a single physical observatory  support multi-domain research  integrate existing sensor and instrument networks with a common instrument proxy  support a set of security (authentication and authorization) capabilities critical for community- owned observatories Researching alternative approaches  extend beyond a single physical observatory  support multi-domain research  integrate existing sensor and instrument networks with a common instrument proxy  support a set of security (authentication and authorization) capabilities critical for community- owned observatories 12 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

OMF Schematic 13 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

14 Example of a single cabled to shore observatory: Monterey Accelerated Research System (MARS) INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

15 Example of Moored Observatory: Monterey Ocean Observing System Surface expression with meteorological instruments and power generator Cable to sub-surface instruments (temperature, currents, bio-optical, etc.) Vertical profiler on cable Benthic node with extension cables to instruments on bottom Surface expression with meteorological instruments and power generator Cable to sub-surface instruments (temperature, currents, bio-optical, etc.) Vertical profiler on cable Benthic node with extension cables to instruments on bottom Benthic network Optical/power cable Instrument node INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

ROADnet sensor map 16 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

17 Enterprise Service Bus (ESB) –Addresses requirements for Cross-Cutting Functions (security, policy enforcement) –Connects Multiple Platforms Enterprise Service Bus (ESB) –Addresses requirements for Cross-Cutting Functions (security, policy enforcement) –Connects Multiple Platforms Enterprise Service Bus (ESB) OMF Prototype –Focus on high risk elements (e.g. applying policy on ESB, common instrument interface, common network interface) OMF Prototype –Focus on high risk elements (e.g. applying policy on ESB, common instrument interface, common network interface) INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

Enterprise Service Bus 18 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

OMF 19 Security Hydrophone INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 Focus on high risk elements (e.g. applying security & policy on ESB, common instrument interface, common network interface

Use case 20 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 o IP Instrument Proxy o Translates OGC SPS local protocol o SP Security Proxy o Verifies incoming messages o Signs outgoing messages o Authorization Service Unit o Validates incoming messages o Authorizes messages o Signs messages OGC SPS Native/Legacy

Demonstration 21 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

1. A researcher uses a web portal to send a request to remotely modify the data collection process from a specific instrument in the offshore instrument network. 22 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

2. The Security Proxy signs the outgoing modification request and passes it through to the Enterprise Service Bus (ESB) via the Message Broker. 23 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

3. ActiveMQ, serving as a Message Broker, delivers the message to the Enterprise Service Bus 24 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

4. The Authorization Service Unit verifies the message signature, applies policy, authorizes the message, and resigns it with its own key. The Enterprise Service Bus then routes the message to its intended destination, in this case, the networked instrument. 25 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

5. ActiveMQ, serving as a Message Broker, delivers the message to the networked instrument. 26 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

6. The Security Proxy verifies incoming messages to ensure that the Authorization Service Unit in the Enterprise Service Bus has processed them. 27 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

7. The Instrument Proxy converts the message (as needed) to the syntax and commands specific to the instrument for which it is intended. 28 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

8. After reaching the deployed instrument network, the message is relayed to the intended instrument. 29 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

9. The instrument then sends a confirmation or other response, which is returned to the researcher via the same logical route as used by the original request. The message destination has a unique identity in OMF, as encoded in the original request and authenticated by the Security Proxy. 30 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legac y

10. The response is returned to the researcher by the web portal. Additional diagnostic information, accumulated as the communication passes through the OMF and instrument network, is also made available to the user and system operators as appropriate given their respective authorizations. 31 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009 OGC SPS Native/Legacy

What is new about all of this? Support of multiple instrument interfaces through Instrument Proxy End-to-end standard security mechanism Implement message-based-system through off-the-shelf Enterprise Service Bus Support highly distributed system though Federated Enterprise Service Bus Support of multiple instrument interfaces through Instrument Proxy End-to-end standard security mechanism Implement message-based-system through off-the-shelf Enterprise Service Bus Support highly distributed system though Federated Enterprise Service Bus 32 INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

33 Partners in the OOI Cyberinfrastructure project include OMF Partners Univ California San Diego o Scripps Institution of Oceanography o Calif Inst for Telecommunications and Information Tech (CalIt2) o San Diego Supercomputer Center o National Center for Microscopy and Imaging Research Academic Partners o NASA Jet Propulsion Laboratory o Massachusetts Inst of Tech, Center for Ocean Engineering o Monterey Bay Aquarium Research Institute o North Carolina State University, Dept. Computer Science o Rutgers University, Coastal Ocean Observatory Lab o University of Chicago, Globus (Ian Foster) o Univ Southen California, Information Sciences Institute o Univ Illinois UC, National Center for Supercomputing Applications o Woods Hole Oceanographic Institution Corporate Partners o Raytheon o Triad Project Management Univ California San Diego o Scripps Institution of Oceanography o Calif Inst for Telecommunications and Information Tech (CalIt2) o San Diego Supercomputer Center o National Center for Microscopy and Imaging Research Academic Partners o NASA Jet Propulsion Laboratory o Massachusetts Inst of Tech, Center for Ocean Engineering o Monterey Bay Aquarium Research Institute o North Carolina State University, Dept. Computer Science o Rutgers University, Coastal Ocean Observatory Lab o University of Chicago, Globus (Ian Foster) o Univ Southen California, Information Sciences Institute o Univ Illinois UC, National Center for Supercomputing Applications o Woods Hole Oceanographic Institution Corporate Partners o Raytheon o Triad Project Management INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009

34 Questions? INGRID 2009, Alghero, Sardinia, Italy April 1-3, 2009