1. Ian Foster Computation Institute Argonne National Lab & University of Chicago A Sociology of the Grid? Carl Kesselman Information Sciences Institute, University of Southern California

2. 2 Abstract The term virtual organization (VO), when used to denote a dynamic collection of individuals, institutions, and resources united by some common interest or task [4], has emerged as a popular, and presumably useful, organizing principle in distributed systems. It is common to see systems being deployed to support one or more VOs, policies may be expressed in terms of VOs, and services are required to support the creation and evolution of VOs. Unfortunately, the popularity of the term has led to a lack of clarity in its meaning: at the limit, a VO could variously denote a multi-decadal scientific collaboration, a commercial outsourcing relationship, a weblog, or an email exchange between two individuals. Yet presumably these different scenarios vary greatly in their requirements for IT infrastructure support, security, reliability, performance, cost, and so on, and may benefit from different technical solutions. This lack of clarity hinders both communication and the identification of required tools. Thus, we seek in this talk to clarify the VO concept and its implications for distributed system implementation--to define a "sociology of the grid.

3. 3 “With the establishment of large scale multidisciplinary production Grid infrastructures such as the EGEE, OSG, DEISA, TeraGrid, or NAREGI, the concept of Virtual Organizations (VO) has been constantly refined and efficient management of VOs and their policies is becoming one of the central topics for these infrastructures.” Why We Are Here

4. 4 “The Anatomy of the Grid,” 2001 The … problem that underlies the Grid concept is coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations. The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem-solving and resource -brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization (VO).

5. 5 Examples “The application service providers, storage service providers, cycle providers, and consultants engaged by a car manufacturer to perform scenario evaluation during planning for a new factory” “Members of an industrial consortium bidding on a new aircraft” “A crisis management team and the databases and simulation systems that they use to plan a response to an emergency situation” “Members of a large, international, multiyear high-energy physics collaboration”

6. 6 From the Organizational Behavior and Management Community “[A] group of people who interact through interdependent tasks guided by common purpose [that] works across space, time, and organizational boundaries with links strengthened by webs of communication technologies” — Lipnack & Stamps, 1997 l Yes—but adding cyber-infrastructure: u People  computational agents & services u Communication technologies  IT infrastructure Collaboration based on rich data & computing capabilities

7. 7 NSF Workshops on Building Effective Virtual Organizations [Search “BEVO 2008”]

8. 8 Two Perspectives l Technology used to enhance collaboration (Computer Supported Collaborative Work) l Collaboration used to enhance technology

9. 9 What is an Organization? l A organization has an identity and a purpose, which it seeks to fulfill within its environment l The organization’s purpose influences its participants, structure, activities, and deliverables, whether products or services l The organization’s performance can be evaluated with respect to various metrics Is a virtual organization any different?

10. 10 IdentityLegal aspects. Credentials. PurposeAnything legal … EnvironmentAvailable service & resource providers. Legal &organizational constraints ParticipantsIdentity-based or attribute-based. People, services, resources, sensors. StructureCentralized, decentralized, … ActivitiesBusiness processes. Workflows. DeliverablesData products. Services. Instrument operations. … PerformanceThroughput, responsiveness, growth, happiness, security, …

11. 11 “I can’t solve this problem alone—I need to involve my buddies Erwin and Miron” It looks like you’re creating a VO Get help with creating the VO Just create the VO without help “Where am I going to eat tonight?” From: Ian To: Erwin, Miron Subject: Help me find a restaurant

12. 12 IdentityBoston-Diner-VO. Informal, so no legal status. PurposeFind a good restaurant in Boston. Avoid eating with loud sports fans. EnvironmentNews feeds. Satellite data feeds. Analytic services. EGEE, OSG, TeraGrid. ParticipantsErwin, Ian, Miron. Anyone else we trust to make good recommendations. StructureCentral database and associated services (membership, …) ActivitiesRestaurant identification workflow. Sport fan tracking workflow. DeliverablesRestaurant recommendation for today. Recommendation service. Business plan? PerformanceFood quality. Monitor Miron’s contributions.

13. 13 Building a Virtual Organization l Determine policy u Negotiation, trust management l Determine membership and roles u Terms of engagement u Virtualization & integration of providers l Create VO-wide services u Global behaviors l Manage work u Collaborative problem solving, workflow management l Manage the VO u Monitor performance, report metrics

14. 14 Defining Community: Membership and Laws l Identify VO participants and roles u And map participants to attributes and roles l Specify and control actions of members u Empower members  delegation u Enforce restrictions  federate policy A 12 B 12 A B 1 10 1 1 16

15. 15 Security Services Objectives l It’s all about “policy” u Define the VO’s operating rules u Security services facilitate the enforcement l Policy facilitates “business objectives” u Related to goals/purpose of the VO l Security policy often delicate balance u Legislation may mandate minimum security u More security  Higher costs u Less security  Higher exposure to loss u Risk versus rewards

16. 16 Policy Challenges in VOs l Restrict VO operations based on characteristics of requestor u VO dynamics create challenges l Intra-VO u VO-specific roles u Mechanisms to specify/enforce policy at VO level l Inter-VO u Entities/roles in one VO not necessarily defined in another VO

17. 17 Core Security Mechanisms l Attribute Assertions u C asserts that S has attribute A with value V l Authentication and digital signature u Allows signer to assert attributes l Delegation u C asserts that S can perform O on behalf of C. l Attribute mapping u {A1, A2… An}vo1  {A’1, A’2… A’m}vo2 l Policy u Entity with attributes A asserted by C may perform operation O on resource R

18. 18 Trust in VOs l Do I “believe” an attribute assertion u Used to evaluate cost vs. benefit of performing an operation u E.g., perform untrusted operation with extra auditing l Look at attributes of assertion signer l Rooting trust u Externally recognized source, e.g., CA u Dynamically via VO structure  delegation u Dynamically via alternative sources, e.g., reputation

19. 19 Building Blocks l Attribute Authority (ATA): u Issue signed attribute assertions (including identity, delegation, & mapping) l Authorization Authority (AZA) u Makes decisions based on assertions & policy ATA User A is an admin User B is a member User B can use service X

20. 20 VO Policy at a Service GT4 authorization and delegation services provide first implementations Resource ATA WS Resource AZA VO ATA WS-Subject ATA: Attribute Authority AZA: Authorization Authority

21. 21 Establishing VO-Wide Policy Resource ATA WS Resource AZA VO ATA VO AZA Subject ATA Subject AZA WS-Subject ATA: Attribute Authority AZA: Authorization Authority GT4 authorization and delegation services provide first implementations

22. 22 Attribute Mapping Service X VO ATA VO AZA Mapping ATA VO 2 Service VO User A Delegation Assertion User B can use Service X VO-1 Attr  VO-2 Attr VO User B Resource Admin Attribute VO Member Attribute VO Member Attribute VO 1

23. 23 Protected Health Information Problem l What do we want? u Use clinical data for research u Share clinical data, make research data available u Reuse same infrastructure u Image exchange between health providers l Patient authorizes use of data – consent process u Intact unmodified DICOM workflow for diagnostics u De-identified DICOM workflow for research (Modality profiles) u Group authorization problem: Patient data–to- user (Physician/Researcher) relationship not manageable!

24. 24

25. 25 Patient Authorized Grid Image Workflow MEDICUS (Erberich et al.)

26. 26 HIPAA Compliant Research Access MEDICUS (Erberich et al.)

27. 27 PHI safe workflow with PCA

28. 28 VO as a Service (VOaaS) l Virtual organizations integrate participants and resource providers u Participants are selected or self assemble u Select “best of breed” providers for VO services l Much of this process can be automated u Provisioning of enabling services, at least Function Resource

29. 29 Community Services Provider Content Services Capacity 1 3 4 5 VOs Assemble Services l Integrate services from various sources u Virtualize external services as VO services l Deploy new services for the VO Capacity Provider 2

30. 30 Content Services Capacity Experimental apparatus Servers, storage, networks Metadata catalog Data archive Simulation server Certificate authority Domain-dependentDomain-independent Simulation code Expt design Tele-op monitor Simulation code Expt output Electronic notebook Portal server VOs Assemble Services

31. 31 Providing VO Services l Integrate existing services l Delegate and deploy capabilities/services u Provision service to deliver defined capability u Configure execution environment u Host higher-level functions u GRAM, Workspace Service, EC2, … l Coordinate and compose u Build new functions from individual services

32. 32 Virtualization and VOs: Its Turtles all the Way Down GRAM Managed Service Mgmt Interface WS-Agreement Cluster Capability provisioned for VO Mgmt Interface WS-Agreement Mgr Glidein Embedded VO management Mgmt Interface WS-Agreement DAGMan Application Application Interface Workflow Provisioning, management and monitoring at all levels

33. 33 Appln Service Create Index service Store Repository Service Advertize Discover Invoke; get results Introduce Container Transfer GAR Deploy gRAVI: Ravi Madduri et al., Argonne/U.Chicago & OSU Service Authoring and Deployment l grid Remote Application Virtualization Infrastructure l Builds on Introduce u Define service u Create skeleton u Discover types u Add operations u Configure security l Wrap arbitrary executables Ravi Madduri

34. 34 Registries (E.g., caBIG)

35. 35 Workspace Service (For When You Want a Virtual Machine) Client request VM EPR inspect and manage deploy & suspend use existing VM image Create VM image VM Factory VM Repository VM Manager create new VM image Resource VM start program Kate Keahey Tim Freeman

36. 36 Composing Services

37. 37 Pull “missing” files to a storage system List of required Files GridFTP Local Replica Catalog Replica Location Index Data Replication Service Reliable File Transfer Service Local Replica Catalog GridFTP Service Composition: Data Replication Service “Design and Implementation of a Data Replication Service Based on the Lightweight Data Replicator System,” Chervenak et al., 2005 Replica Location Index Data Movement Data Location Data Replication Ann Chervenak

38. 38 Decomposition Enables Separation of Concerns & Roles User Service Provider “Provide access to data D at S1, S2, S3 with performance P” Resource Provider “Provide storage with performance P1, network with P2, …” D S1 S2 S3 D S1 S2 S3 Replica catalog, User-level multicast, … D S1 S2 S3

39. 39 Policy, Revisited l Traditionally policy is enforced at end points, integrated with application u E.g., PDP call-out in GT container l We can also apply policy at the VO level u Define interactions between services at the organizational level u Factor policy out of service implementations

40. 40 Policy-Driven Service Oriented Architecture l Need stand-alone policy engine to coordinate at VO level l Connection between application policy and infrastructure policy (dynamic provisioning) l Policy extension points designed into services allow u Coordination at VO level u Dynamic policy enforcement across services and service oriented infrastructure Web Services 2.0: Policy-driven Service Oriented Architectures Thomas B Winans and John Seely Brown

41. 41 A Traditional View of the “Grid Problem” Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations Too limited a view

42. 42 We Need an End-to-End Perspective l A organization has an identity and a purpose, which it seeks to fulfill within its environment l The organization’s purpose influences its participants, structure, activities, and deliverables, whether products or services l The organization’s performance can be evaluated with respect to various metrics Then focus on clear identification of roles, separation of concerns, isolation of policy