Internet2 Technology Update Rick Summerhill Chief Technology Officer, Internet2 Internet2 Fall Member Meeting 9 October 2007 San Diego, CA

Introduction This session will provide an introduction and summary of many of the technology investigations and developments underway in the Internet2 community Technology group on the Internet2 staff Rick Summerhill, CTO Eric Boyd, Deputy Technology Officer, concentrating on Network Architecture and Performance Ken Klingenstein, Senior Director, concentrating on Middleware and Security Matt Zekauskas, Senior Researcher, concentrating on Network Research The session is not meant to include an exhaustive list of everything the community is examining, but rather describe the flavor of new technologies under investigation.

Collaborations Almost all technology development is done through a variety of collaborations Collaborations with members, including campuses, regional networks, and corporate members Almost all of the development in middleware, for example, is done through these types of collaborations. Collaborations with the international community or other national networks like ESnet Much of the work on network performance or architecture includes these types of collaborations. Collaborations with researchers in academia and corporate members For example, network research All of these collaborations are essential to technology development at Internet2

Agenda Some Examples Performance and Architecture Network Research Security and Middleware

Examples Here are a few examples to illustrate how new technologies are undertaken It is crucial that our community push the boundaries on new developments and investigate new ideas. In these first few examples, consider IPv6 Hybrid Networking and the Dynamic Circuit Network

IPv6 IPv6 has long been an area of emphasis for our community IPv6 will likely become very important in the near future given recent ARIN discussions and announcements about the exhaustion of the IPv4 address space The IPv6 initiative is essentially member driven in our community There is an IPv6 working group that meets regularly at the Joint Techs meetings and there are hands-on workshops to support deployment Many of our connectors and members have deployed IPv6 from a network centric point of view.

IPv6 Deployment Although many connectors have deployed IPv6, it is difficult to gauge the deployment deep into the campus IPv6 deployment as a network protocol is fundamentally not difficult Getting campuses and connectors to support IPv6 on crucial applications, however, is often problematic For example, mail servers, web servers, authentication servers - supporting (and porting, in some cases) critical applications to IPv6 lags We encourage you to participate in the IPv6 working group to help set strategic direction for Internet2 in the future

Hybrid Networking There has been tremendous interest from all communities associated with Internet2 to examine services that utilize lower layers of the protocol stack along with IP at layer 3 This has become known as “hybrid networking” It is motivated by applications from the research and education community that require greater capabilities High bandwidth flows (for example, flows that come close to saturating links in the shared IP backbone) Flows with special requirements related to quality of service, for example jitter requirements On the Internet2 network, this takes the following form of an IP network together with the Dynamic Services Network:

Nodes

The Dynamic Circuit Network A Network using protocols different from the normal IP protocols A similar model as an IP network, but with different basic elements - dedicated circuits rather than shared data flows Create Circuits (data paths) in seconds for periods of hours to days between hosts Hosts might be individual hosts or routers on the IP network Tremendous international collaboration on this project - GÉANT2, ESnet and Internet2 Innovative work involving exchange of topology, path computation and scheduling and signaling using web services Demonstration of how this works in the first plenary session

Technology Update: Architecture and Performance Eric Boyd

CI Components Network Performance Infrastructure / Tools Middleware Control Plane …. Bulk Transport 2-Way Interactive Video Real-Time Communications Applications Applications call on Network Cyberinfrastructure …. Phoebus Network Cyberinfrastructure Measurement Nodes Control Plane Nodes

Internet2 DCN and HOPI 10 Gigabit Ethernet 1 Gigabit Ethernet or SONET/SDH OC192 SONET/SDH I2 DCS: Ciena CoreDirector 10 Gigabit Ethernet 1 Gigabit Ethernet I2 HOPI: Force10 E Gigabit Ethernet

Internet2 DCN “Circuits” Physical Connection: 1 or 10 Gigabit Ethernet OC192 SONET Circuit Service: Point to Point Ethernet (VLAN) Framed SONET Circuit Point to Point SONET Circuit (future) Bandwidth provisioning in 100 Mbps increments How do Clients Request? Client must specify [VLAN ID|ANY ID|Untagged], SRC Address, DST Address, Bandwidth Request mechanism options are Web Service API, Web Page, phone call, What is the definition of a Client? Anyone who connects to an ethernet or SONET port on an Ciena Core Director; could be RON, other wide area networks, domain specific applications

Internet2 DCN Circuit IntraDomain Source Address Destination Address Bandwidth VLAN TAG (None | Any | Number) User Identification (certificate) Schedule Client A Client B Circuit Request api Ethernet Mapped SONET or SONET Circuits Dynamically Provisioned Dedicated Resource Path (“Circuit”) Internet2 DCN Service Internet2 IDC api can run on the client, or in a separate machine, or from a web browser XML USER API Actual Network Path To IDC

Internet2 DCN Circuit InterDomain No difference from a client (user) perspective for InterDomain vs IntraDomain RON Dynamic Infrastructure Ethernet VLAN RON Dynamic Infrastructure Ethernet VLAN Internet2 DCS Ethernet Mapped SONET 1. Client Service Request 2. Resource Scheduling 5. Service Instantiation (as a result of Signaling) A. Abstracted topology exchange A A USER API XML

Internet2 DCN Current Status DCN Infrastructure Deployed DCN Control Plane deployed and under test available for use for early adopters General DCN availability planned for January 2008 Instructions for those interested in using Internet2 DCN or in deploying their own dynamic network will be made available soon c

Phoebus Current Status Developed at University of Delaware (Martin Swany) Transport Middleware Configuration per route/host/user UDT for inter-depot communication Transparent operation (library, iptables) Simple file transfer tool (scp) Transparently use Phoebus/Dynamic Circuits Leverage Control Plane Allocate dynamic circuits across Oscars (DCN, others) Authentication and Authorization (currently primitive) Future: Utilize Measurement Infrastructure Help find best routes, provide information about paths and achievable bandwidth

Internet2 Active Measurement Tools OWAMP (Latency) v3.0c (RFC 4645 version) available now Regular tests between all routers, and on-demand BWCTL (Throughput) v2.0 version under development Regular tests between all routers and on-demand NDT (User Diagnostic) v3.4.1 available now Latest version added better logging and error handling NPToolkit (Active Measurement Tool Package) v1.7 available now Knoppix Live-CD bootable system

Internet2 Passive Measurement Tools Circuit Status Service (E2EMON) v1.0 Internet2 implementation of European tool Circuit Status service, Link Status service, Topology service Netflow Anonymized, available to researchers

Internet2 Measurement Framework Why do we need an end-to-end measurement framework? Most organizations can do monitoring and diagnostics of their own network Networking is becoming an increasingly cross-domain effort Monitoring and diagnostics must also become a cross-domain effort What is perfSONAR? A set of protocols and schemas for implementing a service-oriented architecture for sharing and controlling network performance tools A community of users and developers (Internet2, ESnet, GEANT2, and RNP) A set of software (the sample implementation)

Internet2 perfSONAR Current status perfSONAR UI v0.9 available Java release v2.1 available perfSONAR-PS Perl versions of perfSONAR services written by Internet2, ESnet, FNAL, SLAC, and UDel Now Available: Micro-releases of Circuit Status Service, Link Status Service, Lookup Service, Topology Service, SNMP MA Under Development: Micro-releases of perfSONOBUOY, and PingER perfSONAR-PS bundle release planned for early ‘08

Technology Update: Network Research Matt Zekauskas

Research Support in Internet2 Research on the network Learning from measurements Ability to test new theories, protocols and components Research using the network All kinds, not just “network research” Much tends to be “big science”, but it also spans a wide range including new methods of interaction and learning

Philosophy Internet2 does not do network research per se, but seeks to facilitate and support research projects led by faculty at member institutions Make accessible network resources readily available to this community Participate in research collaborations and provide support for proposals Integrate research findings into the evolution of Internet2 network initiatives and services

Making Resources Available Primarily through Internet2 Observatory Two pieces Measurements of Internet2 Network made available Measurements for operations Measurements specifically for research Opportunity to collocate equipment where it makes sense to do so

Existing Measurement Capabilities One way latency, jitter, loss IPv4 and IPv6 (“owamp”) Regular TCP throughput tests – ~1 Gbps IPv4 and IPv6; On-demand available (“bwctl”) ~10GE now also possible (Myricom and Dell 1950, must ask) SNMP Octets, packets, errors; collected 1/min Flow data Addresses anonymized by 0-ing the low order 11 bits Routing updates Both IGP and BGP - Measurement device participates in both Router configuration Visible Backbone – Collect 1/hr from all routers Dynamic updates Syslog; also alarm generation (~nagios); polling via router proxy

Dataset Use Major consumption Flows Most popular (but also one that must be asked for) Routes Configuration Nick Feamster (while at MIT) Dave Maltz (while at CMU) Papers in SIGCOMM, INFOCOM Hard to track folks that just pull data off of web sites

Current Collocation VINI, a Planetlab followon Will provide some sort of private network Congruence with routed network useful 100x100: programmable network processors Again, want private interconnect More details in Research talk Phoebus Break TCP sessions to allow hosts that are not tuned or on flawed networks to effectively use wide-area network May also take advantage of circuits or non-TCP

Current Research Collaborations Ultralight (NSF) Research support for upcoming LHC Physics data flows Project led by Caltech 100x100 (NSF) Focused on understanding the technical & economic requirements for providing 100-Mbps connectivity to 100 million U.S. homes Project led by CMU, Stanford and Rice Hybrid Multi-layer Network (DoE) Look at interoperability issues with new dynamic circuit networks. Data plane interoperability, control plane interoperability… Project led by U New Mexico, USC ISI; includes ESnet and UltraScienceNet

Other, More Ad-Hoc, Collaborations Buffer sizing project (Stanford): Reduce buffers available to router interfaces (software controlled) Take an anonymized but correlated packet trace Look for throughput and latency anomalies Rapid raw SNMP to test link capacity measurement programs Occasionally run programs on behalf of researchers on backbone machines

Small Grant Participation Network Measurement for International Connections I’m PI, but work is done in close collaboration with Matt Mathis (who also has a small grant) and the International Research Network Connection PIs. Research current state and propose solutions Suggest common measurements Identify areas for improvement Work to establish a program-wide measurement group

Futures Work with Research Advisory Council to determine futures Restart some focus on outreach and dialog that was begun under a different small grant on the use of Internet2 facilities for research Provide the best possible data from our network, and facilitate other opportunities that come our way Come see the Network Research update late this afternoon for more details on current activity

Technology Update: Security and Middleware Ken Klingenstein

Security REN-ISAC - CSI2 Real time security exchanges Google analytics Disaster Recovery FWNA and eduRoam

Middleware Developments SAML and Shibboleth InCommon and international federations Collaboration management platforms NSF-Mellon Scientific and Scholarly Workflow

SAML and Shibboleth Shibboleth 1.3 widely deployed as federating software; openSAML widely used as a library Shibboleth 2.0 completes Shib/SAML integration; now in beta Missing pieces (e.g. personal attribute release) becoming evident and being addressed Google, MS, others now provide some financial support; service companies now available

InCommon Growing steadily now; 65 members and 1.3M user base Major applications include outsourced services, content providers, wiki and collaboration tools NIH and federal follies elsewhere Apple, Google and Microsoft in contract review InCommon Bronze and Silver now under discussion

Prague Meeting on Inter-federation International R&E federations (5 continents) plus Liberty Alliance and a few others Prague, September 3 Lots of topics: Attribute mapping, Privacy Policies, Dispute resolution, Financial considerations, Technical direction setting UK drafting an analysis of International Peering needs, opportunities, etc.

Peering Parameters Parameters: LOA Attribute mapping Legal structures Liability Adjudication Metadata VO Support Economics Privacy

Collaboration Management Platforms Management of collaboration a real impediment to collaboration, particularly with the growing variety of tools Goal is to develop a “platform” for handling the identity management aspects of many different collaboration tools Platform includes a framework and model, specific running code that implements the model, and applications that take advantage of the model This space presents possibilities of improving the overall unified UI as well as UI for specific applications and components.

COManage Leverages federated identity and the attribute ecosystem heavily Shib-enabled; uses Grouper to manage groups, Signet to manage privileges, Eddy for diagnostics Built completely on open protocols, using open source components Open and proprietary applications can be plumbed to work with it Sympa, wikis, audioconferencing, sharepoint, calendaring are comanageable, to varying degrees, now Web-based file shares, rich wikis next…

Comanage dimensions of growth In the applications that can be driven by it Collaboration and domain science prime areas Largely a function of the application’s respect for middleware In the areas being managed - diagnostics In the identities being managed In the coupling of autonomous and diverse instances

Upcoming Talks Middleware: The Big Picture Gets Bigger Happening now, look at slides online Network Research Update Tuesday, 4:30, Grand Hall Performance Update Wednesday, 10:30 AM, Golden West Dynamic Circuit Network Update Thursday, 8:45 AM, California Room General Session: Cyberinfrastructure: The Way Forward Thursday, 10:15 AM, Grand Hall