1. Ocean Observatories Initiative OOI Cyberinfrastructure Common Operating Infrastructure Subsystem Michael Meisinger, Munindar Singh, Von Welch OOI Cyberinfrastructure Life Cycle Objectives Milestone Review, Release 1 San Diego, CA February 23-25, 2010

2. OOI CI LCO Review, Feb 2010 2 Overview COI Subsystem Overview Capability Container and Messaging Resource Management Resource Governance & Background Federating Facilities Security and Identity Management Putting it all Together

3. OOI CI LCO Review, Feb 2010 3 Common Operating Infrastructure Is the integration & communication environment for all the other subsystem services

4. OOI CI LCO Review, Feb 2010 4 COI Services Service Framework Resource Management Presentation Framework Capability Container Exchange Identity Management Governance Framework Distributed State Managemt 2940-00001 OV2 CI

5. OOI CI LCO Review, Feb 2010 5 Scope of Release 1 Provide a basic “capability container” for infrastructure and application service integration Secure reliable asynchronous messaging Governed resource sharing and access Support federation of facilities Monitoring service interactions for compliance Resource registration and basic resource life cycle management Basic user and external interface support Support for multiple programming languages

6. OOI CI LCO Review, Feb 2010 6 Outlook: COI Release 2 Advanced resource management –Life cycle services –User resource activation Advanced capability container with full federated facility support Interaction specification and enactment Embedded capability containers Advanced system operations and monitoring Out of scope for release 1

7. OOI CI LCO Review, Feb 2010 7 Risks High –Capability Container integration –Common message format –Governance framework –Secure messaging –Service integration platform Medium –Authentication, Policy enforcement –Resource registry –Distributed state framework –Service registry Low –(not much because all other subsystems depend on COI) –User interface platform Iteration 1 and 2 (Inception period) prototyping activities

8. OOI CI LCO Review, Feb 2010 8 Capability Container and Messaging Use Cases –Deploy a service in a capability container –Enroll in an Exchange Space –Send a request message to a service –Access to infrastructure services

9. OOI CI LCO Review, Feb 2010 9 Capability Container

10. OOI CI LCO Review, Feb 2010 10 Secure Reliable Messaging Capability Container Capability Container Capability Container Capability Container Capability Container Capability (Service) Capability (Service) Capability (Service) Capability (Service) Capability (Service)

11. OOI CI LCO Review, Feb 2010 11 The “Exchange” Applications communicate through Exchange Spaces Exchange Spaces are namespaces of “communicators” Applications need to enroll in Exchange Spaces Governance applies within Exchange Spaces

12. OOI CI LCO Review, Feb 2010 12 Exchange Space and Points Exchange Points are the message routing and queuing resources of Exchange Spaces Communicators play the role of Producer, Consumer, and Distributer (Broker)

13. OOI CI LCO Review, Feb 2010 13 Messaging Abstraction Capability Container Capability Container Capability Container Send Message From: “name2” To: “name4” In: Exchg-Space1 Action: “invoke-service”

14. OOI CI LCO Review, Feb 2010 14 Common Message Format Capability Container –Provides message handling through interceptors After a service sends a message, before it receives a message Message signing and validation Policy enforcement Governance tracking –Provides a common message format for all CI messages Based in ACL FIPA message structure Content, encoding, ontology

15. OOI CI LCO Review, Feb 2010 15 Risk Mitigation Development Out of the box: –RabbitMQ AMQP message broker –Python: flexible and powerful applications –Twisted: distributed application framework –txAMQP: messaging library Integration –Message abstraction for services –Intercepting message handler (in and out) –Policy and governance integration (via agents) –Development console

16. OOI CI LCO Review, Feb 2010 16 CI Resources “CI governed” Resource Standard and user metadata attributes (in OOI convention) References to other resources Categories –Information resource –Physical (stateful) resource –Taskable resource

17. OOI CI LCO Review, Feb 2010 17 Resource Management Services Resource Agent Resource Registry Resource (external) 2940-00005 OV2 COI

18. OOI CI LCO Review, Feb 2010 18 Services and Resources as Agents Resource Agent Resource Agent Proxy Agent Capability Container Capability Container Capability Container Capability Container

19. OOI CI LCO Review, Feb 2010 19 Scenario An instrument, a physical resource, is represented by an agent to the system and its users Users request control of the instrument Capabilities are projected into another domain of authority by a proxy agent

20. OOI CI LCO Review, Feb 2010 20 Resource Governance

21. OOI CI LCO Review, Feb 2010 21 Motivating Governance Administering collaborations –Based on framing normative relationships among peers –Abstracting away from low-level details OOI, broadly: many stakeholders; many resources; longevity of decades Exchange spaces, narrowly: abstractions for communicating; assembly of multiple topologies for messaging; analogous to traditional enterprise integration patterns

22. OOI CI LCO Review, Feb 2010 22 Elements of a Service Engagement Enactment: doing the domain work – what the end user cares most about Administration: captured via contracts –Partnerships –Rules of encounter Identity Enforcement

23. OOI CI LCO Review, Feb 2010 23 What is Governance? Broadly, administering service engagements IT Governance: How IT resources are administered SOA Governance: How services are created, deployed, removed, … Currently, governance is manual –Low productivity –Poor scalability for fine-grained, real time governance decisions –Hidden, implicit considerations yield low confidence and poor maintainability

24. OOI CI LCO Review, Feb 2010 24 Why Governance? Stakeholders using resources to best serve individual and collective needs –Share resources in a controlled manner –Configure and reconfigure dynamically –Enable unanticipated uses for resources –Respect human organizational needs In a nutshell, stakeholders administer themselves

25. OOI CI LCO Review, Feb 2010 25 Separation of Concerns Protocol: specifying the interactions among autonomous parties Policy: specifying the decision making of each autonomous party as it participates in various protocols Behavior: specifying the implementation that realizes the interactions

26. OOI CI LCO Review, Feb 2010 26 Principles of Governance: 1 Vividness of Modeling –Grounded in applications; modeled entities are real Autonomy of Participants –Stating rules of encounter; omitting policies from specifications Centrality of Organizations –Modeling communities, facilities, the OOI; specifying rules of encounter; monitoring contracts; sanctioning violators

27. OOI CI LCO Review, Feb 2010 27 Principles of Governance: 2 Minimality of Operational Specifications –Leaving restrictions unstated except where essential to correctness Institutional Actions –Creation and manipulation of commitments; granting or denying powers, authorizations; effecting sanctions –Separation of concerns from those of operational interactions Reification of Representations –Explicit: hence, inspectable, sharable, and manipulable

28. OOI CI LCO Review, Feb 2010 28

29. OOI CI LCO Review, Feb 2010 29 Exchange Space Use Case OOI CI LCO Review, Feb 2010 29

30. OOI CI LCO Review, Feb 2010 30 Messaging View of Enrollment OOI CI LCO Review, Feb 2010 30

31. OOI CI LCO Review, Feb 2010 31 Community Affiliation Use Case OOI CI LCO Review, Feb 2010 31

32. OOI CI LCO Review, Feb 2010 32 Combined Scenario, Schematically OOI CI LCO Review, Feb 2010 32

33. OOI CI LCO Review, Feb 2010 33 Risk Mitigation Development Out of the box: –RabbitMQ AMQP message broker –Python: flexible and powerful applications –Twisted: distributed application framework –txAMQP: messaging library Integration –Message abstraction for services –Intercepting message handler (in and out) –Policy and governance integration (via agents) –Development console

34. OOI CI LCO Review, Feb 2010 34 Security and Identity Management

35. OOI CI LCO Review, Feb 2010 35 Secure Messaging and Identity Management Identity Management (IdM) is the management and communication of user identities and attributes for use by Governance, Audit and other systems. –Federated IdM is the use of user information from one organization in another organization. Secure Messaging encapsulates message authentication, integrity and confidentiality.

36. OOI CI LCO Review, Feb 2010 36 Scenario User is member of organization acting as an identity provider. User performs one-time registration with COI. User then routinely authenticates with COI using identity asserted by their home organization. After authentication, can participate in Secure Messaging: enrolling in exchange spaces and performing operations managed by Governance.

37. OOI CI LCO Review, Feb 2010 37 Architecture Goals Leverage user identities from their home organization (identity provider). Allow for multi-homing of users and migration of users between organizations. Allow for technology changes by providing for abstraction layer between technology at user’s home institution and COI. Allow for trade-offs on ease-of-use versus strength of security. Allow for both thin (web browser) and thick (command-line) clients.

38. OOI CI LCO Review, Feb 2010 38 Architecture Overview

39. OOI CI LCO Review, Feb 2010 39 Architecture Overview

40. OOI CI LCO Review, Feb 2010 40 Technology Overview Utilize InCommon as the IdM federation of choice for U.S. higher ed. today. CILogon builds on InCommon to support thick clients. –Expect to be needed for next few years. Security messaging leverages XML Security Messaging, conceptually at least.

41. OOI CI LCO Review, Feb 2010 41 Putting it all together A service gets deployed on a capability container –Initialization: service enrolls as “communicator” into an Exchange Space A user application looks up the service and sends a service request message –Look up the service in the service registry –Enroll in necessary exchange spaces/points –Send a message via the exchange

42. OOI CI LCO Review, Feb 2010 42 Enrolling in an Exchange Space 2940-00061 OV6 COI

43. OOI CI LCO Review, Feb 2010 43 Send a message 2940-00063 OV6 COI

44. OOI CI LCO Review, Feb 2010 44 Receive a message 2940-00062 OV6 COI

45. OOI CI LCO Review, Feb 2010 45 COI Technology List Messaging –RabbitMQ AMQP broker (with federation extensions) –Distributed IPC Facility Implementation Capability Container –Python, Twisted, txAMQP –Java, Spring –Open Telecom Platform (OTP) style service deployment –FIPA ACL Message Format (standard headers), DM Common Format Policy and Governance –Rules engine (Jess/Pyke) Identity Management –CIlongon –Internet2 Security infrastructure Resource Management –Redis Attribute Store (with DM enhancements) Presentation Framework –Portal framework (such as Django, Drupal)

46. OOI CI LCO Review, Feb 2010 46 Elaboration Plan Elaboration Iteration 1 –Secure messaging (using IdM technologies) –Policy enforcement for resource/service requests –Integration of DM metadata model in resource registry –Distributed service state coordination via the AttributeStore Elaboration Iteration 2 –User registration with external identities –Policy definition and enforcement –Demonstrate federated facilities –Integrated basic capability container, ready for use by –Demonstrate integration with CEI provisioning and DM distribution, storage and inventory –Initial web user interface framework

47. OOI CI LCO Review, Feb 2010 47 Thanks!

48. OOI CI LCO Review, Feb 2010 48 Capability Container Components (1)

49. OOI CI LCO Review, Feb 2010 49 Capability Container Components (2)

50. OOI CI LCO Review, Feb 2010 50 Resource Agent Services

51. OOI CI LCO Review, Feb 2010 51 Policy and Governance Services

52. OOI CI LCO Review, Feb 2010 52 Exchange Space Exchange Space is comprised of –Distributed Application Facility (DAF) –Distributed IPC Facility (DIF)

53. OOI CI LCO Review, Feb 2010 53 Back-End Infrastructure

54. OOI CI LCO Review, Feb 2010 54 Exchange Points and the DIF

55. OOI CI LCO Review, Feb 2010 55 Message Brokers over DIF

56. OOI CI LCO Review, Feb 2010 56 Messaging Service Interfaces

57. OOI CI LCO Review, Feb 2010 57 Extra Slides

58. OOI CI LCO Review, Feb 2010 58 Registration Service

59. OOI CI LCO Review, Feb 2010 59 Authentication (thick client)

60. OOI CI LCO Review, Feb 2010 60 Authentication (thin client)

61. OOI CI LCO Review, Feb 2010 61 Secure Messaging Data Model OOI CI LCO Review, Feb 2010 61