Sponsored by the National Science Foundation GENI and Cloud Computing Niky RIga GENI Project Office

Sponsored by the National Science Foundation2IC2E‘14 – 10 March 2014www.geni.net Cloud, HPC, and Distributed Computing Why Cloud / Distributed Computing –Performance –Locality –Dynamic resource allocation –Reliability Slide by Paul Ruth Image by:

Sponsored by the National Science Foundation GENI Ricks and SDN SDN and Clouds GENI and other Cloud Providers International Collaborations GENI and Cloud Research

Sponsored by the National Science Foundation4IC2E‘14 – 10 March 2014www.geni.net Cloud, HPC, and Distributed Computing Combine multiple resources –Compute, Network, Storage Need to distribute tasks –PBS/Torque, MPI, HTCondor, Hadoop, Slirm –Tightly or loosely coupled Cloud (GENI) –Allocate compute, network, and storage –GENI can allocate WAN resources Slide by Paul Ruth

Sponsored by the National Science Foundation5IC2E‘14 – 10 March 2014www.geni.net Cloud Providers Observatory Wind tunnel Workflow Slide by Paul Ruth Cloud, HPC, and Distributed Computing

Sponsored by the National Science Foundation6IC2E‘14 – 10 March 2014www.geni.net GENI Racks as Clouds A GENI Rack –Raw PCs –100s of VMs –Separates control and data planes –Flexibility on host OSes –Hosts isolated experiments Racks can act as cloud providers, or to deploy clouds

Sponsored by the National Science Foundation7IC2E‘14 – 10 March 2014www.geni.net GENI Racks InstaGENIExoGENI * Nodes per rack510workers + 1head Cores per rack60120 Network interfaces4x 1Gbit2x 10Gbit with SR-IOV Storage1 TB local150GB+500 GB local + 6 TB SAN SwitchesHP ProCurve 5406 (Vlan-based OpenFlow) IBM G8264R (Port- based OpenFlow) * Listed are the specs for ExoGENI IBM-based racks. InstaGENI: Less powerful more places ExoGENI: More powerful fewer places Each rack has two raw PCs

Sponsored by the National Science Foundation8IC2E‘14 – 10 March 2014www.geni.net High Throughput Computational Genomics MotifNetwork –Jeffery L. Tilson (RENCI) –Identification of functional domains and the identification of conserved functional relationships across large numbers of genomes. –Workflow ensemble (Pegasus/HTCondor) –Task data sets on the order of GB. –Implicit iteration –Scales to 1000s of tasks Slide by Paul Ruth

Sponsored by the National Science Foundation9IC2E‘14 – 10 March 2014www.geni.net Urgent Computing ADCIRC –Brian Blanton (RENCI) –Storm surge and tide model –Finite element model –MPI tightly coupled –Approved by FEMA for computing storm surge flood hazard simulations –Used for Digital Flood Insurance Rate Maps (DFIRMs) –Scales to MPI processes Slide by Paul Ruth

Sponsored by the National Science Foundation GENI and Cloud Research SDN and Clouds GENI and other Cloud Providers International Collaborations SDN and Clouds

Sponsored by the National Science Foundation11IC2E‘14 – 10 March 2014www.geni.net Cloud Research and Networking Cloud deployments rise new network challenges –Within one Datacenter –In distributed Datacenters Google’s SDN WAN SDN enables easy, customizable network innovation

Sponsored by the National Science Foundation12IC2E‘14 – 10 March 2014www.geni.net GENI OpenFlow Deployment OpenFlow-enabled hardware switch at: –Each GENI Rack –Backbone and regional networks (between racks)

Sponsored by the National Science Foundation13IC2E‘14 – 10 March 2014www.geni.net GENI Experiment: Virtual Desktop Clouds

Sponsored by the National Science Foundation14IC2E‘14 – 10 March 2014www.geni.net Prasad Calyam,U. of Missouri Program realtime load- balancing functionality deep into the network to improve QoE GENI Experiment: Virtual Desktop Clouds

Sponsored by the National Science Foundation GENI and Cloud Research SDN and Clouds GENI and other Testbeds International Collaborations GENI and other Cloud Providers

Sponsored by the National Science Foundation16IC2E‘14 – 10 March 2014www.geni.net Interconnecting grid internet

Sponsored by the National Science Foundation17IC2E‘14 – 10 March 2014www.geni.net Interoperate internet e.g. LabWiki GENI AM API GENI Tools

Sponsored by the National Science Foundation18IC2E‘14 – 10 March 2014www.geni.net NowCast System Slide by Mike Zink, UMass Amherst Short-term weather prediction (1-15 mins) Forecasts as we know them: Data from many sensors: Radar, satellite, balloons, Usually for large regions Takes super computers to calculate

Sponsored by the National Science Foundation19IC2E‘14 – 10 March 2014www.geni.net Multi-radar NetCDF Data Nowcast Processing 1.Spin up system in Amazon commercial EC2 and S3 services on demand “raw” live data Generate “raw” live data ViSE/CASA radar nodes Generate “raw” live data ViSE/CASA radar nodes ViSE views steerable radars as shared, virtualized resources ViSE views steerable radars as shared, virtualized resources Nowcast images for display Nowcast images for display Weather NowCasting University of Massachusetts David Irwin et al Create and run realtime “weather service on demand” as storms turn life-threatening

Sponsored by the National Science Foundation GENI and Cloud Research SDN and Clouds GENI and other Cloud Providers International Collaborations International Collaborations

Sponsored by the National Science Foundation21IC2E‘14 – 10 March 2014www.geni.net International Collaboration Modified slide from: deration.pdf Efforts to provide L2, highspeed, SDN network between International testbeds

Sponsored by the National Science Foundation22IC2E‘14 – 10 March 2014www.geni.net GENI Experiments: TransGEO The Intercloud will be about the seamless movement of computation Design and development of protocols, security procedures, architectures, economic models that permit computation to move to data, rather than the other way around

Sponsored by the National Science Foundation23IC2E‘14 – 10 March 2014www.geni.net GENI Experiments: TransGEO Compute how “Green” a City using satellite images Transcontinental Federation of Cloud Systems with Private 10Gb/s transcontinental network linking sites Perform distributed query on TransCloud

Sponsored by the National Science Foundation24IC2E‘14 – 10 March 2014www.geni.net QUESTIONS?