National Science Foundation International Research Network Connections Program Kickoff Julio Ibarra, PI Heidi Alvarez, Co-PI Chip Cox, Co-PI John Silvester, Co-PI March 11, 2005

Outline l The WHREN-LILA project l US - Latin America Connectivity l Links Interconnecting Latin America (LILA) year 1 l The AtlanticWave Project l Monitoring and Measurement Activities l WHREN-LILA Year 1 Milestones l E-Science and Engineering Collaborations 2

The WHREN-LILA Project l Proposal submitted by Florida International University (FIU) and the Corporation for Education Network Initiatives in California (CENIC) - Award# l Links Interconnecting Latin America (LILA) aims to Improve connectivity in the Americas through the establishment of new inter- regional links l Western-Hemisphere Research and Education Networks (WHREN) is a coordinating body of organizations from across North and South America that aims to leverage the network resources of participating members to foster collaborative research and advance education throughout the Western Hemisphere 3

Project Goals l Improve network connectivity between North and South America through the deployment, operation and evolution of LILA links l Evolve the LILA links to their fullest capacities as resources and economies permit l Foster collaborative research and advance education throughout the Western Hemisphere and other world regions l Support the evolving needs of US science and engineering researchers l Foster new inter-regional and inter-disciplinary communities of researchers and learners 4

WHREN - Coordination in the Western Hemisphere l WHREN will establish a consortium of participating western-hemisphere organizations that will collectively oversee the assignment of lightpaths across administrative domains l A Governance Committee (GC) will collectively oversee the assignment and management of lightpaths and provide coordination between member organizations l A Research Advisory Committee will be formed to advise the GC on program and network needs for the broad research and education community l An Engineering Committee (EC) will be comprised of engineering managers from the various networks participating in WHREN 5

LILA Project Coordination l Participating Organizations l Project Steering Committee formed, with one member from each participating organization, for project implementation and operational decisions l Engineering Committee, comprised of network engineers from each participation organization, to make network engineering and operational recommendations to the Steering Committee FIU (Awardee)  FIU (Awardee)  CENIC (Awardee)  ANSP (Sao Paulo)  CLARA (Latin America)  CUDI (Mexico)  RNP (Brazil)  REUNA (Chile) 6

US - Latin America Connectivity before IRNC l Argentina, Brazil (national and the State of Sao Paulo), Chile, Panama and Venezuela connections through Miami l Mexico connections through San Diego and El Paso l Peerings with Internet2 and other US R&E networks through AMPATH, CalREN and UTEP l International and FedNet peerings at STARTAP/Starlight from Miami provided by AMPATH 7

Regional Development l Cooperation of Latin American research networks (CLARA) Alliance of the Internet Society funded program, providing 10 Million Euros for interconnecting R&D communities of Latin America and Europe l Creates a regional backbone in Latin America l Direct connectivity to Europe from Sao Paulo, Brazil l Intraregional connectivity between connected countries in Latin America l 3 DS3s from AMPATH to support CLARA initiative Argentina (RETINA) Brazil (RNP) Chile (REUNA) Costa Rica (CRNET) Panama (REDCYT) Paraguay (ARANDU) Peru (RAAP) Uruguay (RAU) Venezuela (REACCIUN) Ecuador (CEDIA) El Salvador (RAICES) Guatemala (RAGIE) Mexico (CUDI) Nicaragua (RENIE) (NRENs in formation indicated in RED) 8

Expected CLARA network topology 155 Mbps backbone ring 622 Mbps connection to Europe local traffic remains within the region 10 to 45 Mbps spur links 4Mbps satellite link to Cuba Network to be operated by CLARA (through CUDI and RNP) Network Characteristics: 9

RedCLARA Routed Network 10

Current US - Latin America Topology l RedCLARA network starts operating in August 2004 l Brazil/RNP and Chile/REUNA today transit CLARA, then GEANT to reach US R&E networks l NSF CHEPREO project and collaboration with Sao Paulo R&E community establishes STM-4 link between US and Brazil l ITN services and transit to FedNets through Abilene l Argentina, Panama and Venezuela maintaining direct connections to US through AMPATH l Mexico has direct connection to US through UTEP 11

Links Interconnecting Latin America (LILA) Year 1 l Increases Miami - Sao Paulo link from 622Mbps to 1.2Gbps  Q  Evolving to 2.5Gbps l Establishes a dark fiber segment between San Diego and Tijuana for a 1Gbps link  May 2005 l Enables interregional peerings through east and west coasts 12

LILA Miami - Sao Paulo link design AMPATH Router (Cisco GSR 12012) AMPATH 1/10 GigE Switch (Foundry FastIron400) LILA-Brasil Optical Mux (Cisco ONS 15454) ANSP router POP RedCLARA NAP of the Americas Miami COTIA Sao Paulo POP RNP USP 2 x VC4-4C (2 * 622Mbps = 1.2Gbps) Abilene LILA-Miami Optical Mux (Cisco ONS 15454) AtlanticWave OC192 Ethernet switch GigE l Provides dedicated Gig-E interface to ANSP l Provides shared Gig-E interface to CLARA and RNP l Support planned for lightpath provisioning and deterministic transport services through AtlanticWave l Peering with Internet2’s Abilene and other R&E networks through AMPATH 13

LILA San Diego - Tijuana Link l Provides dedicated Gig-E interface to CLARA l Provides dedicated Gig-E interface to CUDI l Growth across border possible through purchase of additional Gig-Es up to maximum of 6 l Connections are to CENIC’s CalREN/HPR routed network l Peering through CalREN to Internet2 and other R&E networks 14

LILA San Diego - Tijuana link design CalREN/HPR CLARA CUDI CENIC at UCSD/SDSC BESTEL in Tijuana GX 3180 University Telnor in Tijuana Cisco 7206 VXR switch CLARA router switch CUDI router switch CalREN/HPR CUDI Provided Equipment/fiber CLARA Provided WHREN-LILA Provided Logical Diagram Physical Diagram 15

US - Latin America Year 1 Topology l LILA links reestablish direct connectivity to South America from east and west coasts l Reduces delay reaching sites in Chile and Brazil from the US and Asia-Pacific l Introduces an infrastructure to develop a distributed international exchange and peering l Leverages network resources to provide route diversity and high-availability production services 16

AtlanticWave l AtlanticWave is an International Peering Fabric  US, Canada, Europe, South America  Distributed IP peering points:  NYC, WDC, ATL, MIA, SPB l SURA, FIU-AMPATH-CHEPREO, the IEEAF, MAX, SoX/SLR, MANLAN, and in partnership with the Academic Network of Sao Paulo (ANSP) are combining efforts to establish AtlanticWave l Described as an integral component of the WHREN-LILA proposal to extend LILA on the Atlantic side to MANLAN in NYC l Complements the PacificWave distributed peering facility on the west coast 17

AtlanticWave Topology l A-Wave provides multi- layer/multi-protocol services between participating networks  Layer 3 peering services over ethernet  GLIF “light path” services  Others TBD l A-Wave will to provide a Layer 3 distributed exchange capability  Ethernet based  Best effort packet exchange  Linear topology – unprotected (NLR based)  1 GE, 10GE LAN, 10GE WAN client access  Jumbo frame support 18

AtlanticWave Design MIA Miami, FL NYCWDCATL Atlanta, GAWashington, DCNew York City, NY To South America To Europe & Canada, Initial East Coast US A-Wave Backbone Future backbone Extensions within NA & EU Ethernet services Sonet services Prepared by Jerry Sobieski 19

Sonet switch Sonet switch NLR Node Ethernet switch Ethernet switch Generic A-Wave Node Architecture (using separate switching fabrics) A-Wave Backbone OC192c Regional sonet handoff: OC192c or OC48c Payload agnostic Ethernet WAN PHY Regional ethernet handoff: 10Gbs LAN or 1Gbs Prepared by Jerry Sobieski 20

Deployment Plans & Timeline l Phase 1: Deploy backbone OC192cSept 05  Between MIA-ATL, ATL-WDC, WDC-NYC  10Gbs WAN PHY ethernet over NLR wave initially.  Migration of existing exchange switches/networks  Regional backhaul  Reconfiguration of existing exchange services and networks l Phase 2: Sonet switch deploymentDec 05  Map IP/Ethernet Peering Fabric across “appropriate” sized VCG (GFP- F & VCAT)  Engineer and deploy GLIF Common Services in conjunction with other GLIF domains l Phase 3: Deploy dynamic light path servicesMar 06 l Phase 4: ExpansionAug 06 ->  Integrate links between A-Wave, P-Wave, Northern Tier, etc 21

Monitoring and Measurement Activities l NSF REU Distributed Network Monitoring using NetFlow and MonALISA project l An integrated networking monitoring system using Cisco Netflow, MonALISA and NLANR PMA/AMP tools l PingER and NLANR AMP active measurement resources in service for links to Latin America 22

Distributed Network monitoring using Netflow and MonALISA l Development project that integrates NetFlow monitoring data with the MonALISA framework l Collaboration between FIU, Caltech and CERN l Builds a lightweight, scalable and customizable utility that enables viewing flows based on source-destination pairs l Allows for a historical trend analysis of network behavior as well as a close-to- realtime look at flow data l Builds a framework for an Integrated Passive Flow Measurement Toolkit (next slide) l Support from NSF REU Award SCI l REU Report available at 23

Integrated Passive Flow Measurement Toolkit l A real-world deployable network monitoring toolkit that improves the functionality of MonALISA by integrating NetFlow and NLANR PMA l Cisco University Research Program (URP) Award to Caltech and FIU l Leverages NSF REU project that integrates NetFlow into MonALISA l Integrates MonALISA, Cisco NetFlow and NLANR PMA tools to facilitate efficient network planning and operation for the at large networking community l Being deployed to monitor traffic flows for CHEPREO between the US and Brazil l Toolkit will be available by Sept

Measurement and Monitoring Plan l Active measurement  SLAC PingER (deployed and available, a Telnet server and a PingER collector for delay/loss information from/to AMPATH) l Passive measurement  NLANR PMA (now taking a 90-second trace every hour, tunable upon request)  NetFlow (available via the MonALISA service module and SNMP/MRTG graphs) l Implementation and Deployment  Service discovery and service integration based on MonALISA  Modular design to allow flexible and robust deployment and extensibility 25

Year 1 Milestones l Implement LILA links l Implement interregional peering through CalREN and AMPATH l Establish Coordination and Control mechanisms for service management l Deploy AtlanticWave OC192c backbone l Deploy Next-Gen SONET switches May 05 June 05 Sept. 05 Dec

E-Science and Engineering Collaborations 27

CMS Experiment Global LHC Data Grid Hierarchy Online System CERN Computer Center Argentina USA Mexico Brazil UNAM GBytes/s >10 Gb/s Gb/s Gb/s Tier 0 Tier 1 Tier 3 Tier 4 Tier 2 Physics caches PCs UBA USPCaltech UF ~10s of Petabytes/yr by ~1000 Petabytes in < 10 yrs? FIU 28

Grand Challenge Research:CHEPREO l An interregional grid-enabled Center for High- Energy Physics Research and Educational Outreach (CHEPREO) l Fosters an integrated program of research, network infrastructure development, and education and outreach  Collaboration with FIU, Caltech, University of Florida, Florida State University, the State University of Rio de Janeiro, University of Sao Paulo  Augments bandwidth capacity to Brazil l Joint funding by U.S. NSF (MPS ) and State of Sao Paulo Research Foundation (FAPESP) 29

 Integrated hybrid experimental network, leveraging Transatlantic R&D network partnerships; packet-switched + dynamic optical paths  10 GbE across US and the Atlantic: NLR, LHCNet, NetherLight, UKLight, etc.; Extensions to Korea, Brazil, Taiwan  End-to-end monitoring; Realtime tracking and optimization; Dynamic bandwidth provisioning  Agent-based services spanning all layers of the system, from the optical cross-connects to the applications. UltraLight Collaboration:  Caltech, UF, UMich, SLAC,FNAL, CERN, FIU, NLR, CENIC, UCAID, Translight, UKLight, Netherlight, UvA, UCLondon, KEK, Taiwan, KNU (Korea), UERJ (Rio), USP (Sao Paolo)  Cisco 30

Grid3  Open Science Grid l Build on Grid3 experience  Persistent, production-quality Grid, national + international scope l Ensure U.S. leading role in international science  Grid infrastructure for large-scale collaborative scientific research l Create large computing infrastructure  Combine resources at DOE labs and universities to effectively become a single national computing infrastructure for science l Provide opportunities for educators and students  Participate in building and exploiting this grid infrastructure  Develop and train scientific and technical workforce  Transform the integration of education and research at all levels 31

Source: Cindy Zheng 32

Grand Challenge Science Instruments Gemini-South Optical Observatory NRAO telescopes La Serrena, Chile Atacama Large Millimeter Array (ALMA) Atacama plains Atacama plains in Chile 33

Traditional VLBI The Very-Long Baseline Interferometry (VLBI) Technique (with traditional data recording) The Global VLBI Array (up to ~20 stations can be used simultaneously) 34

Pan-American Advanced Studies Institute NSF sponsored program to offer a series of lectures at the advanced graduate and postgraduate level involving domain researchers, students and practitioners. Award# , OISE Americas Program Aims to disseminate advanced scientific and engineering knowledge, stimulate collaborative learning and cooperation among the research communities of the Americas CIARA, along with collaborators from the U.S., Argentina and Brazil, is organizing a PASI to offer a series of lectures on the role of Grid Computing and Advanced Networking for High-Energy Physics and Astronomy Our PASI is planned for May 15-20, 2005 in Mendoza, Argentina Approximately 40 students from the Americas will learn of the major experiments, Grid and advanced networking technologies and how the growing interdependence between the science and the technologies are forming global collaborations Mendoza Argentina 35

Thank You! l WHREN-LILA, AMPATH infrastructure, CHEPREO, science application support, education, outreach and community building efforts are made possible by funding and support from:  National Science Foundation (NSF) awards STI , MPS , OISE and SCI  Florida International University  Latin American Research and Education community  The many national and international collaborators who support our efforts 36