Sponsored by the National Science Foundation GENI Software Marshall Brinn, GPO Architect January 7, 2013

Sponsored by the National Science Foundation2CC-NIE Workshop January 7, 2013 Outline GENI Principles GENI Software Categories GENI Software Details GENI Software Requirements

Sponsored by the National Science Foundation3CC-NIE Workshop January 7, 2013 GENI Principles The GENI Architecture Team has recently worked to define and publish a set of concise statements that define the GENI project and architecture efforts: –Differentiators: What makes GENI different from other cloud infrastructures or distributed test-beds? –Principles: What are GENI’s essential motivating values and goals? –Requirements: What are the top level system requirements that drive the architecture and implementation? While still a work-in-progress, we hope they convey a good sense of the “what” and “why” of GENI.

Sponsored by the National Science Foundation4CC-NIE Workshop January 7, 2013 GENI Differentiators GENI provides open access to resources to the American academic and research community GENI provides custom, segregated and programmable computation, network and storage topologies GENI provides low-level metrics on hardware substrate to facilitate repeatable experimentation in virtual environments GENI provides access to uncommon or expensive resources to researchers GENI provides resources with broad geographic diversity, spanning the United States and providing access to international federation resources GENI provides the ability for users to 'opt-in' their internet traffic to experimental services or networks

Sponsored by the National Science Foundation5CC-NIE Workshop January 7, 2013 GENI Principles GENI is dedicated to supporting science and network experimentation and researchers GENI is a federation of autonomous test-beds and resources GENI establishes a common trust fabric to allow disparate resources to interoperate reliably GENI establishes and enforces policies that provide assurances to resource owners that their resources will not be misused. GENI federation members agree to abide by these policies in exchange for these assurances. GENI supports interoperability among disparate resources and control frameworks

Sponsored by the National Science Foundation6CC-NIE Workshop January 7, 2013 GENI Requirements GENI will provide custom, segregated and programmable computation, network and storage topologies GENI will provide common authentication and authorization services to support federated aggregates in validating experimenter resource requests GENI will provide support for protecting federated aggregates from misuse by, at least, forensics and slice shutdown services

Sponsored by the National Science Foundation7CC-NIE Workshop January 7, 2013 GENI Software Context Review Experimenter: A researcher seeking to perform network experiments on customized data plane. Federation: A collection of people and institutions who agree to share resources and abide by common procedures in order to share resources in a reliable, mutually beneficial manner. Resources: Physical resources (compute, network, storage) made available to the federation by means of a participating aggregate. Aggregates: Software entities that represent federated resources in transactions with experimenter tools. Tools: Software capabilities that interact with federation resources on behalf of experimenters Clearinghouse: Set of services establishing federation- level authentication, authorization and accountability of experimenter use of federation resources. Operations Center: Processes and tools monitoring activity on GENI resources for adherence to policies. Grey boxes are real-world entities, represented in software by Purple boxes.

Sponsored by the National Science Foundation8CC-NIE Workshop January 7, 2013 GENI Software Suite Aggregate Managers: Allows the owner of a set of resources to share these resources with the GENI federation by means of the GENI Aggregate Manager (AM) API Experimenter Tools: Allows an experimenter to express and implement their needs for resources and topologies and experiment configurations Clearinghouse: Establish federation-level trust, identity, policy GMOC: Support forensics and high-level oversight, monitoring and management of GENI operations

Sponsored by the National Science Foundation9CC-NIE Workshop January 7, 2013 GENI Software Suite: Aggregate Manager Control Plane: Creates custom Data Plane topologies –Slicing Services: HyperVisors (OpenStack, KVM, Xen) –Programmability Services: OpenFlow –Stitching Services: Intra-Aggregate and Inter-Aggregate services for stitching cross-aggregate topologies Management Plane: Monitoring Aggregate behavior, taking protective action if necessary –GMOC Monitoring/Reporting/Control Interface Think of the Aggregate Manager as providing Control Plane and Management Plane operations on customized Data Planes

Sponsored by the National Science Foundation10CC-NIE Workshop January 7, 2013 GENI Software Suite: Aggregate Manager [2] ProtoGENI / InstaGENI: Developed and maintained by University of UTAH, partnered with HP and Princeton, derived from Emulab capability ORCA / ExoGENI: Developed and maintained at RENCI in North Carolina FOAM/FlowVisor : Maintained by Open Network Labs, presents OpenFlow “flow space” as an virtual resource Any service that presents resources in accordance with the GENI AM API is an Aggregate Manager. There are several implementations that are deployed and interoperate within the GENI federation

Sponsored by the National Science Foundation11CC-NIE Workshop January 7, 2013 GENI Software Suite: Experimenter Tools Resource Management Tools: Allow experimenters to express and build custom topologies –GENI Portal: Web-based access to Clearinghouse services and Aggregate resources Emphasis on making “Simple things simple, Difficult things possible” –Omni: Command-line interface to Aggregate resources –FLACK: Graphical interface to building and viewing custom topologies

Sponsored by the National Science Foundation12CC-NIE Workshop January 7, 2013 GENI Software Suite: Experimenter Tools [2] Experiment Management Tools: Support configuring and running experiments on the GENI-provided data plane, and reviewing/analyzing results –Orchestration: OMF, GUSH –Instrumentation/Monitoring: GEMINI and GIMI projects esp. GEMINI Portal, LabWIKI –Archiving/Analysis: iRODS, UNIS

Sponsored by the National Science Foundation13CC-NIE Workshop January 7, 2013 GENI Software Suite: Clearinghouse Series of federation-level services to establish broad common trusted sense of identity and policy –Introduces “Project” level of management of activity on slices/slivers Establishes privileges of experimenters based on their roles on “projects” Establishes accountability (“one neck to wring”) for all activity on a project to that project’s PI –Establishes federation-level certificates and trust roots to enable all tools and aggregates to interoperate reliably –Establishes a common directory of federation-level services for other services to discover one another

Sponsored by the National Science Foundation14CC-NIE Workshop January 7, 2013 GENI Federation Software Architecture Schematic GMOC Experimenter Tool Aggregate Identity Provider Identity Provider Service Authority Service Authority AuthZ Service AuthZ Service Slice Authority Logging Service Logging Service GENI Clearinghouse Project Authority Project Authority Credential Store Member Authority Member Authority

Sponsored by the National Science Foundation15CC-NIE Workshop January 7, 2013 GENI Software Suite: GMOC Forensics: Detailed logging of operations and metrics on resources for real-time monitoring and post-analysis of experiments, failures, misbehavior –What operations were taken by whom when? –What level of network or compute activity was taking place on which resources? –What slivers belong to which slices, projects, PI’s? Management: Ability to determine a misbehaving experiment (intentionally or not) and shut it down on all participating aggregates without impacting other co- located experiments The GENI Meta-Operations Center (GMOC) provides top-level oversight and management services to protect resources against misuse (intentional or not)

Sponsored by the National Science Foundation16CC-NIE Workshop January 7, 2013 Aggregate Manager: Managing Campus Boundaries for Experiments OUTSIDE CAMPUS FW INSIDE CAMPUS FW Aggregate Manager RESOURCE GENI and Campus Resources Control/Management Plane (IP) AM API Requests/Responses GENI CH Credentialing GMOC Control Messages GMOC Monitoring Data Plane (L2) Trans-Aggregate Experiment Traffic Shibboleth AuthN InCommon AuthN PKI-based Credentials VLAN-based segregation Signed, Authenticated Requests Slice/Sliver Expiration FOAM ‘FlowSpace’ Authorization ABAC-based AuthZ (Future) Policy and trust inputs allow the campus to control which requests flow over the control plane, including which resources are connected to the data plane.

Sponsored by the National Science Foundation17CC-NIE Workshop January 7, 2013 GENI Aggregate Authentication GENI participating campuses should be members of the OCI-sponsored InCommon Federation, which provides trusted and validated user credentials –Organizations should provide “Research and Scholarship” InCommon category IDP’s and+Scholarship+Categoryhttps://spaces.internet2.edu/display/InCCollaborate/Research+ and+Scholarship+Category Provides information such as Affiliation, , Name [First, Last], EPPN GPO provides a default IDP for campuses that do not yet provide such an IDP GENI Authentication is based on: InCommon Identity Provider (IDP) of users signing into GENI tools Shibboleth provides single sign-on sessions based on this identity

Sponsored by the National Science Foundation18CC-NIE Workshop January 7, 2013 GENI Policy Management GENI Aggregates use policy to control its responses to critical questions such as: –Which experimenters do I trust? –With which other aggregates am I willing to collaborate? –How many resources should I allocate to which experimenters or experiments? Currently, the GENI Clearinghouse presents a bundle of ‘trusted roots’ that federated aggregates accept and thereby trust any credential signed by someone trusted by GENI. In the future, GENI expects to use the far more expressive ABAC language to capture and police policy statements Note that the expressing and policing of policy statements can and is done in software. But the establishment of these policies and trust are human and inter- organizational (out-of-band) actions.

Sponsored by the National Science Foundation19CC-NIE Workshop January 7, 2013 Deployment Requirements: Software Hardware Configuration –Encoding hardware configuration details such as switch configurations, compute node MAC’s and switch ports, dedicated VLAN’s, capacities/constraints of H/W, QoS budgets Integration with Campus Infrastructure –Integration with site health/reporting tools (is rack up?) –Rack power-down/reboot integration with site management tools GENI Federation –Installation of GENI trust roots –Creation/distribution of GENI-signed credentials This list is not complete and may vary by the type of rack, but provides a sense of the kinds of requirements to deploy a GENI rack

Sponsored by the National Science Foundation20CC-NIE Workshop January 7, 2013 Software Plans and Milestones Software SuiteCurrent StatePlans/ Milestones Rack Aggregate AM API V3 ratified- Rack AM software in test and will be installed on racks as deployed - Aggregates working to be compliant with latest features by Spring ’03 Clearinghouse / Portal - Current racks implement CH Authentication, Authorization and Audit functions in distributed fashion - Alpha version of CH and Portal available to select experimenters and developers - Integration of CH with Rack Aggregates as they deploy, expected Beta available Summer ‘13 - Additional regular Portal releases (every ~4 months) to capture new AM, tool or CH capabilities Experimenter Tools - Several tools (omni, Portal) currently available for topology management - Tools for experiment I&M available as Alpha on certain rack platforms - Stitching tools expected by Summer ‘13 - Tools for experiment I&M expected by Fall ‘13 on all rack platforms GMOC- Reporting interface under Alpha testing by rack developers, GPO - Integration of report interface with CH and AM’s expected late Spring ’03 - Additional reporting, forensics, control tools and capabilities rolled out in Summer-Fall ‘03

Sponsored by the National Science Foundation21CC-NIE Workshop January 7, 2013 Summary The GENI Federation is a collaborative effort among people: experimenters, resource owners and network managers The GENI project provides a broad range of software tools that represent the interests of these people to allow them to share resources in a trusted, efficient manner