OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative Instrument and Platform Agent Architecture (IPAA) Steve Foley Life Cycle Architecture Review La Jolla, CA

OOI CI LCA REVIEW August 2010 Outline IPAA background Architectural Overview Release 1 Product Description Use Case Overview Technology Challenges and Achievements Progress in Elaboration Phase (from LCO to LCA) LCA Accomplished Use Cases Demonstrated Use Cases

OOI CI LCA REVIEW August 2010 IPAA Subsystem Overview Provides interface between instruments and the rest of CI Configures and commands instruments Collects data from instruments and publishes into CI Intended to be more production of drivers than development of CI core concepts after initial data plumbing is complete

OOI CI LCA REVIEW August 2010 Architectural Overview 2 sides to the IPAA black box: AMQP protocol to talk to CI’s registries, services, agents, UIs, etc. RS-232/Ethernet with instrument-specific text or binary protocols to communicate with instrument. Between the 2 sides are the agent and driver that provide access control, locking, translation, buffering across slow links, instrument/driver specific logic

OOI CI LCA REVIEW August 2010 Architectural Overview

OOI CI LCA REVIEW August 2010 Instrument Agent Interface: CI Side Generic common interface for system-wide interaction Extensible at lower levels for instrument-specific work Based on getting and setting parameters and executing commands Announces capabilities and uses registries for discovery Potentially capable of transactions and sequences

OOI CI LCA REVIEW August 2010 Instrument Agent Interface Execute commands (CI and Instrument) Get/Set parameters (CI and Instrument) Get status (CI and Instrument) Get capabilities (CI and Instrument) Get translator function (CI) Get/Set CI lifecycle state (CI) Get registry entries (CI)

OOI CI LCA REVIEW August 2010 Test Instrument: SBE49 SeaBird Electronics SBE49 CTD has been our simple, text-based test instrument for now, easy to mock up and test. Some commands: DS – Status & params Polled sampling PUMPON / PUMPOFF TS – Take single sample Autonomous sampling START / STOP NAVG=x – Samples to average Settings BAUD=x OUTPUTFORMAT=x …various coefficients to set

OOI CI LCA REVIEW August 2010 Instrument Driver Interface: Instrument Side S>navg=16# set a few parameters S>autorun=y S>pumpdelay=30 S>ds# display some settings SBE 49 FastCAT V 1.3a SERIAL NO number of scans to average = 16 pressure sensor = strain gauge, range = minimum cond freq = 3273, pump delay = 30 sec start sampling on power up = yes output format = raw HEX temperature advance = seconds celltm alpha = 0.03 celltm tau = 7.0 real-time temperature and conductivity correction enabled, not applied to raw data S>outputformat=2# Raw Tmp, Cond, Pres, Sal S>ts# Test single sample , , , S>start# Start streaming , , , , , , , , , …

OOI CI LCA REVIEW August 2010 Instrument Driver Interface: Instrument Side Takes Get/Set/Execute based calls Translates to/from instrument protocol Maintains necessary protocol state/mode with instrument Needs to know what commands are valid SBE49 has very little state to track, other instruments will have more

OOI CI LCA REVIEW August 2010 Release 1 Product Description Use Case Overview Responsible for: Command An Instrument (#18) Supporting: Hello Instrument (#2) Direct Instrument Access (#19) Command a Resource (#20) Version a Resource (#24) Assess Access Policy (#31)

OOI CI LCA REVIEW August 2010 Command an Instrument Use Case Initial use case is to command an instrument User determines commands to apply to an instrument Instrument Agent has already been created and registered Commands are issued to Instrument Agent Commands have been redirected to Instrument Driver specific to the instrument Commands are applied to the instrument by the driver Responses are passed back to the user Does not handle permissions or direct access

OOI CI LCA REVIEW August 2010 Status of Progress Elaboration period has been focused on infrastructure for process management, data flow, and proper interaction with registries End-to-end exchanges of messages demonstrate complete data pathway and interface sanity “Command An Instrument” use case was addressed and can be demonstrated.

OOI CI LCA REVIEW August 2010 Technology and Interface Choices AMQP between Agent and Driver May be changed during construction if performance or situation requires it AMQP between Agent and CI registries Currently TCP with text protocol to serial device (would be through a serial-to-Ethernet device)

OOI CI LCA REVIEW August 2010 Technology Challenges and Achievements Agent registration  discoverable Instrument Agents Separation of agent and driver  decoupled processing Simulated device  testing of data plumbing Protocol interaction with device  science data flows Start data flowing from instrument, acquiring by driver, and publishing to subscribers  complete data path coverage

OOI CI LCA REVIEW August 2010 Challenges – Registration Registries did not initially exist, but have been developed with the Data Management team, and their contents defined Instruments are the first working agents, so plenty of work needed to be done to organize registries Sequence is a bit involved, but easy to use at higher levels

OOI CI LCA REVIEW August 2010 Challenges – Registration

OOI CI LCA REVIEW August 2010 Challenges – Separation of Agent/Driver Having the driver and agent in the same process was problematic for both CI interactions and instrument interactions happening simultaneously Agent and driver communicate via AMQP messages Separate agent and driver allows for independent development, testing, profiling, maintenance, and improvement as needed Messaging layer is added complexity

OOI CI LCA REVIEW August 2010 Challenges – Simulation Did not initially have instruments Have multiple developers Need simple, repeatable, controllable, far end for commands/queries We developed a simulator that handles very simple requests from our code. Allows for a quick guess at compatibility before a test instrument is ready. Definitely not a substitute for a real instrument

OOI CI LCA REVIEW August 2010 Challenges – Interacting with devices All instruments behave a little differently Initial simulator should fit protocol spec…but no guarantees First cut of interaction with a sensor is to a mocked up SeaBird Electronics SBE-49 Allows for commanding an instrument and flowing data Rest of instrument agent flows data through completed data plumbing

OOI CI LCA REVIEW August 2010 Achievement – Putting It All Together Initial setup and registration

OOI CI LCA REVIEW August 2010 Challenges – Interaction and instrument command

OOI CI LCA REVIEW August 2010 Challenges – Interaction and data publishing

OOI CI LCA REVIEW August 2010 Early Work Completed - RSN While not on the Release 1 agenda, advance investigations were made into RSN interfacing to keep pace with the rest of the project. Initial plans are for the interface to be Python (AMQP?) based, SNMP may also be an option if needed Interfaces are being specified in detail Design seems to be inline with our future plans for platform management RSN high-level integration design CG embedded device (Technologic Systems TS-7370) testing and performance profiling

OOI CI LCA REVIEW August 2010 Early Work Completed - CGO Work has been done to plan integration with the CGO platforms. On-board controller will be Linux based embedded ARM processor (Technologic Systems TS-7370) Python capable, early versions of code function without modifications. AMQP library is slow, needs tuning More use case work needs to be done

OOI CI LCA REVIEW August 2010 One more Elaboration Iteration IPAA has one more iteration to go before IPAA moves on to construction. This iteration will finish up some plumbing: Support for direct access mode Streamline data flow Keep up with changing CI interfaces

OOI CI LCA REVIEW August 2010 Plan for Construction Integration More integration with RSN, possible prototyping Specify interfaces with CGO team, continue testing on TS-7370 Write preliminary acquisition drivers for CGO platforms? Driver production Add drivers ADCP driver (binary format) Add Q330 data logger driver (via 3 rd party Antelope sw)

OOI CI LCA REVIEW August 2010 Significant Risks Performance on CGO embedded device Profiling indicates that OOI logic in python is fine for the anticipated load, but AMQP library is not. Other libraries exist and may be worth testing. If Python is too slow, we may port of some code to C Common data format is still unspecified Simulators are only so good…will need real devices soon Some instruments are more difficult/involved than others Staffing – We need the people to write the drivers

OOI CI LCA REVIEW August 2010 Addressing the Key Issues Thanks ! Questions ?