About the EnLightened Computing Project Prepared for Grid Computing Class at UNCW and UNCC Carla S. Hunt Senior Solutions Architect MCNC April 12, 2007

Your Presenter Education: B.S. Computer Science, University of Wilmington M.S. Information Sciences, North Carolina Central University Industry Experience IBM, Network and Systems Engineer Robert Taylor Associates, Chief Internet Services Engineer Sentinel Services, Information Technology Consultant MCNC, Senior Solutions Architect

MCNC & NCREN Who we are: MCNC (Microelectronics Center of NC) We own and operate NCREN (NC Research and Education Network) One of the most advanced statewide research and education networks in the country Provide Internet, video and related services to University of North Carolina’s 16 campus system, Duke, Wake Forest, etc. Non-profit organization, about 50 employees History: Commissioned by the General Assembly in 1980 to further economic development in the state of North Carolina What we do: Provide non-commodity Internet and related services to more than half a million users daily! Network has an optical ring based architecture with speeds up to 2.5 Gbps

EnLIGHTened Computing Project International recognition and collaboration NSF seed funded 10 Gbps national research testbed Grid Middleware developed to co-schedule and co-allocate network and compute resources Major Accomplishments: First ever demonstration of trans-pacific, inter-domain co- scheduling of network and compute resources across two national grid computing research testbeds First ever demonstration of External Network to Network Interface (E-NNI) capability to establish on-demand end-to-end light paths via Calient optical switches First ever high definition video broadcast over the Internet of classroom instruction—LSU high performance computing class

EnLightened and G-Lambda GLIF Demonstration Who: EnLIGHTened (US) and G-Lambda (Japan) Where: 11 th floor of AIST building in Tokyo, Japan When: September 11 th -13 th, 2006 What: Researchers from US and Japan demonstrated trans-pacific, inter-domain advance reservation of coordinated network and compute resources across two national research testbeds during GLIF’s 6 th Annual Global Lambdagrid Workshop

Organization of Research Teams Testbed Network and Compute Resources Application Middleware Performance and monitoring

EnLIGHTened Testbed

Testbed Components Compute Resources: Clusters Storage Client machines/displays Network Resources: Core Optical Switches Other Network Devices

US Domain: Compute Resources X1U RA1 BT2 CH1 VC1 LA1 BT1

Caltech—LA1 4 nodes (LA1-LA4) 2 nodes---4x2.2GHz Sun Opteron 2 nodes---4x2.4GHz Tyan S2891 Opteron RHELv4, 64 bit Torque and Maui Globus Toolkit 10 Gig Nic cards Shared resource sometimes

CCT at LSU: BT1 Pelican.cct.lsu.edu 14 nodes 8 x 1.9 GHz IBM POWER5 processors 16 GB RAM per node AIX 5.2 w/ Loadleveler Shared resource

CCT at LSU: BT2 Santaka.cct.lsu.edu SGI shared memory 32 – 1.5 GHz Intel Itanium2 processors 128 GB RAM per system SUSE Enterprise 9/ PBSPro 10 Gig Nic card (I think) Shared resource

MCNC: RA1 Kite1.enlightenedcomputing.org Approximately 40 nodes available 2x2.8GHz Intel Xeon 4 GB RAM RHELv4 Torque/Maui Globus Toolkit v Shared resource

VCL at NCSU: VC1 vclb5-1.hpc.ncsu.edu IBM Blade Center RHELv4 Torque/Maui Globus Toolkit v Dedicated Resource

Coordinating Compute Resources Policy for shared & borrowed resources User accounts Security Installation and configuration of Operating systems Cluster management software (scheduler, resource mgr) Grid middleware (globus toolkit and grid security) IP address management Other Application requirements (software libraries or pkgs) Performance tuning (tcp parameters for network card)

KDDI NRM CRM Cluster CRM Cluster CRM Cluster NTT NRM CRM Cluster CRM Cluster EL NRM CRM Cluster CRM Cluster CRM Cluster JAPAN CRM US Application (MPI) Application (Visualization) Request Network bandwidth and Computers Reservation From xx:xx to yy:yy Request Network bandwidth and Computers Reservation From xx:xx to yy:yy

Japan Application GL Grid Resource Scheduler EL→GL wrapper KDDI NRM CRM Cluster CRM Cluster CRM Cluster NTT NRM CRM Cluster CRM Cluster EL NRM CRM Cluster CRM Cluster CRM Cluster US Application EL Grid Resource Coordinator/ App. Launcher EL→GL GNS-WSI wrapper GNS-WSI JAPAN US GL: G-lambda EL: Enlightened GL request EL request G-lambda/Enlightened middleware coordination diagram EL→GL CRM wrapper HARC Acceptor

Resource map X2S Japan South Japan North US X2N X1N X1U X1S KMF FUK KHN AKB RA1 BT2 KAN TKB OSA CH1VC1 4G 5G 2G NR3 (UO1) (UO2) (UO4) (UO3) (UR1) (UR2) (UR3) X1 X2 LA1 BT1 BT3 0.11a a a a.2

Performance and Monitoring Tools Mona Lisa Monitor connections between Calient Optical Switches Iperf Test throughput end to end Smartbits Test throughput on pipes only

Demonstration Display

Conclusions Recently forged new partnership with Phosphorus, EU researchers--- ”Collaboration Across Three Continents” Questions about the project or these slides? Remember any researcher is welcome to test his/her application on the EnLIGHTened Testbed!!