1. OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger Life Cycle Architecture Review La Jolla, CA

2. OOI CI LCA REVIEW August 2010 Agenda Subsystem Purpose Release 1 Product Description Use Case Overview Architectural Overview Status of Progress Use Cases Address, Use Cases Demonstrated Technology Challenges and Achievements Plan for Construction Significant Risks

3. OOI CI LCA REVIEW August 2010 Common Operating Infrastructure: Purpose Provide the integration & communication environment for all the other subsystem services

4. OOI CI LCA REVIEW August 2010 Release 1 Product Description Use Case Overview IDTitleDescription UC.R1.01Hello UserUser gets an ID and logs in UC.R1.05Synchronize State DataSynchronize state in distributed data store UC.R1.10Define InteractionDescribe pattern of interaction between actors UC.R1.11Define New ServiceAdd new service to system capabilities UC.R1.14Use Service AnywhereMessages go to services wherever they are UC.R1.20Command A ResourceSend typical commands to specific resource UC.R1.28Operate SystemConfigure system and respond to requests UC.R1.29Monitor SystemAnticipate issues using monitoring tools UC.R1.30Troubleshoot SystemDiagnose issues using logs, feeds, tools UC.R1.31Assert Access PolicyDefine access policy for given resource UC.R1.32Conduct NegotiationNegotiate agreement (or not) between agents UC.R1.33Enroll in an OrgEnter as a member into an Organization (Org) UC.R1.34Share an OrgResource Collaborate with Org member offering a resource UC.R1.35 Share Affiliated Orgs' Resources Orgs form a contract to allow resource sharing UC.R1.36Create an OrgCreate an Organization (Org) with defined characteristics

5. OOI CI LCA REVIEW August 2010 Architectural Overview Capability Container Messaging, Common Message Format Service Framework Distributed State Management, Common Object Model Resource Management Security and Identity Management Presentation Framework

6. OOI CI LCA REVIEW August 2010 COI Services Service Framework Resource Management Presentation Framework Capability Container Exchange Identity Management Governance Framework Distributed State Managemt 2940-00001 OV2 CI

7. OOI CI LCA REVIEW August 2010 Capability Container

8. OOI CI LCA REVIEW August 2010 Scope of Release 1 Provide a basic “capability container” for service and agent integration in Python and Java Secure reliable asynchronous messaging Distributed state management Resource registration and basic resource life cycle management Governed resource sharing and access Basic user interface support

9. OOI CI LCA REVIEW August 2010 Status of Progress IDTitleDescription UC.R1.01Hello UserUser gets an ID and logs in UC.R1.05Synchronize State DataSynchronize state in distributed data store UC.R1.10Define InteractionDescribe pattern of interaction between actors UC.R1.11Define New ServiceAdd new service to system capabilities UC.R1.14Use Service AnywhereMessages go to services wherever they are UC.R1.20Command A ResourceSend typical commands to specific resource UC.R1.28Operate SystemConfigure system and respond to requests UC.R1.29Monitor SystemAnticipate issues using monitoring tools UC.R1.30Troubleshoot SystemDiagnose issues using logs, feeds, tools UC.R1.31Assert Access PolicyDefine access policy for given resource UC.R1.32Conduct NegotiationNegotiate agreement (or not) between agents UC.R1.33Enroll in an OrgEnter as a member into an Organization (Org) UC.R1.34Share an OrgResource Collaborate with Org member offering a resource UC.R1.35 Share Affiliated Orgs' Resources Orgs form a contract to allow resource sharing UC.R1.36Create an OrgCreate an Organization (Org) with defined characteristics

10. OOI CI LCA REVIEW August 2010 Technology Challenges and Achievements Key Achievements Robust messaging: AMQP / RabbitMQ Definition of a process abstraction Distributed data store and structured object transport Resource Registries Presentation UI Framework Technology Challenges

11. OOI CI LCA REVIEW August 2010 COI Technology List Capability Container Python, Twisted, txAMQP Java, Spring Open Telecom Platform (OTP) architecture FIPA ACL Message Format (standard headers) in Common Format Messaging RabbitMQ AMQP broker (with federation extensions) Policy and Governance Jess Rules engine Identity Management CILogon Internet2 Security infrastructure Distributed State Management Cassandra Key Value Store GIT compatible Content Addressable Storage (CAS) architecture Presentation Framework Grails platform (Java/Groovy)

12. OOI CI LCA REVIEW August 2010 Capability Container and Messaging Capability Container

13. OOI CI LCA REVIEW August 2010 Capability Container and Messaging Use Cases: TBD PD Deploy a service in a capability container Enroll in an Exchange Space Send a request message to a service Access to infrastructure services

14. OOI CI LCA REVIEW August 2010 Secure Reliable Messaging Capability Container Capability Container Capability Container Capability Container Capability Container Capability (Service) Capability (Service) Capability (Service) Capability (Service) Capability (Service)

15. OOI CI LCA REVIEW August 2010 The “Exchange” Applications communicate through Exchange Spaces Exchange Spaces are namespaces of “communicators” Applications need to enroll in Exchange Spaces Governance applies within Exchange Spaces

16. OOI CI LCA REVIEW August 2010 Exchange Space and Points Exchange Points are the message routing and queuing resources of Exchange Spaces Communicators play the role of Producer, Consumer, and Distributer (Broker)

17. OOI CI LCA REVIEW August 2010 Messaging Abstraction Capability Container Capability Container Capability Container Send Message From: “name2” To: “name4” In: Exchg-Space1 Action: “invoke-service”

18. OOI CI LCA REVIEW August 2010 Common Message Format Capability Container Provides message handling through interceptors After a service sends a message, before it receives a message Message signing and validation Policy enforcement Governance tracking Provides a common message format for all CI messages Based in ACL FIPA message structure Content, encoding, ontology

19. OOI CI LCA REVIEW August 2010 Distributed State Management Use Case: UC.R1.05 Synchronize State Data Cassandra Backend Content Addressable Store Structured object abstraction Structure definition and identification Transport Persistence

20. OOI CI LCA REVIEW August 2010 Content Addressable Store

21. OOI CI LCA REVIEW August 2010 Resource Management

22. OOI CI LCA REVIEW August 2010 CI Resources “CI governed” Resource Standard and user metadata attributes (in OOI convention) References to other resources Categories Information resource Physical (stateful) resource Taskable resource

23. OOI CI LCA REVIEW August 2010 Resource Management Services Resource Agent Resource Registry Resource (external) 2940-00005 OV2 COI

24. OOI CI LCA REVIEW August 2010 Services and Resources as Agents Resource Agent Resource Agent Proxy Agent Capability Container Capability Container Capability Container Capability Container

25. OOI CI LCA REVIEW August 2010 Scenario TBD: Refer to PD use case An instrument, a physical resource, is represented by an agent to the system and its users Users request control of the instrument Capabilities are projected into another domain of authority by a proxy agent

26. OOI CI LCA REVIEW August 2010 Security and Identity Management

27. OOI CI LCA REVIEW August 2010 Scenario: UC.R1.1 Hello User User is member of organization acting as an identity provider. User performs one-time registration with COI. User then routinely authenticates with COI using identity asserted by their home organization. After authentication, can participate in Secure Messaging: enrolling in exchange spaces and performing operations managed by Governance.

28. OOI CI LCA REVIEW August 2010 IdM Architecture Overview InCommon, the IdM federation of choice for U.S. higher ed. today. CILogon builds on InCommon to support thick clients. Security messaging leverages XML Security Messaging

29. OOI CI LCA REVIEW August 2010 IdM Secure Messaging Overview

30. OOI CI LCA REVIEW August 2010 UC.R1.02: Access Service Anywhere A service gets deployed on a capability container Initialization: service enrolls as “communicator” into an Exchange Space A user application looks up the service and sends a service request message Look up the service in the service registry Enroll in necessary exchange spaces/points Send a message via the exchange

31. OOI CI LCA REVIEW August 2010 Plan for Construction General End-to end integration with all subsystems Separation the COI "ION-Core" from other COI services and release as module Integrate IdM and governance/agents into the container pervasively Provide a UI development platform Provide a local test, integration test and operational execution platform Stabilize core, container and service interfaces Iteration R1C1. Theme: Full software integration and refactoring for production release Support and achieve the integration into one code base (and branch) for all subsystems and implementation project Refactor and define the ION-Core packages and their interfaces to non-core services Work with CEI on system bootstrapping, process execution and scalability issues Pervasive use of Resource Agents Integration of Identity Management in the Web UI presentation framework Initial policy enforcement applied to resource access. Integration test harness Define initial Java Container Environment Make Presentation Platform available for integration of subsystem specific UIs Scoping for R1C2. Theme: Functional completeness and end-to-end integration with subsystems Integrate governance concepts (Org/facility) Complete work on resource agents with governance concepts Interaction management: conversations interaction types Enable subsystems to contribute fully capable UIs Scoping for R1C3. Theme: Robustness and governance Develop governance concepts into federated facility Finalize the UI work together with UX team Performance test core container and services Automatic Integration and system test

32. OOI CI LCA REVIEW August 2010 Significant Risks High Maturity and complexity of Governance concepts and technologies Maturity and complexity of interaction management concepts and technologies Medium Integration of Identity Management Integration of Policy enforcement Robustness of capability container implementation Scalability of current core infrastructure implementation Low (not much because all other subsystems depend on COI)

33. OOI CI LCA REVIEW August 2010 Thanks ! Questions ?

34. OOI CI LCA REVIEW August 2010 Capability Container Components (1)

35. OOI CI LCA REVIEW August 2010 Capability Container Components (2)

36. OOI CI LCA REVIEW August 2010 Resource Agent Services

37. OOI CI LCA REVIEW August 2010 Policy and Governance Services

38. OOI CI LCA REVIEW August 2010 Exchange Space Exchange Space is comprised of –Distributed Application Facility (DAF) –Distributed IPC Facility (DIF)

39. OOI CI LCA REVIEW August 2010 Back-End Infrastructure

40. OOI CI LCA REVIEW August 2010 Exchange Points and the DIF

41. OOI CI LCA REVIEW August 2010 Message Brokers over DIF

42. OOI CI LCA REVIEW August 2010 Messaging Service Interfaces

43. OOI CI LCA REVIEW August 2010 Risk Mitigation Development Out of the box: RabbitMQ AMQP message broker Python: flexible and powerful applications Twisted: distributed application framework txAMQP: messaging library Integration Message abstraction for services Intercepting message handler (in and out) Policy and governance integration (via agents) Development console

44. OOI CI LCA REVIEW August 2010 Enrolling in an Exchange Space 2940-00061 OV6 COI

45. OOI CI LCA REVIEW August 2010 Send a message 2940-00063 OV6 COI

46. OOI CI LCA REVIEW August 2010 Receive a message 2940-00062 OV6 COI