FutureGrid Cyberinfrastructure for Computational Research

FutureGrid Goals  An environment that can be used for middleware developers to develop and test middleware  An environment that can instantiate a wide variety of computing environments (ranging from current TeraGrid software, VMs, cloud, grid systems other than TG) for development and testing of applications on experimental basis in a test bed, in replicable ways  Education related to all of the above  US Industry research related to the above

FutureGrid Components Compute – flexible deployment of images, VM's, clouds Storage – local storage sufficient to facilitate projects, Lustre for larger projects Network – Dedicated networks between FutureGrid sites, Impairment Device

Compute Hardware System type# CPUs# CoresTFLOPSTotal RAM (GB) Secondary Storage (TB) Site Status Dynamically configurable systems IBM iDataPlex *IU New System Dell PowerEdge TACC New System IBM iDataPlex UC New System IBM iDataPlex SDSC Existing System Subtotal Systems not dynamically configurable Cray XT5m *IU New System Shared memory system TBD *IU New System 4Q2010 IBM iDataPlex UF New System High Throughput Cluster PU Existing System Subtotal Total

Storage Hardware System TypeCapacity (TB)File SystemSiteStatus DDN 9550 (Data Capacitor) 339LustreIUExisting System DDN GPFSUCNew System SunFire x417096ZFSSDSCNew System Dell MD300030NFSTACCNew System FutureGrid has a dedicated network (except to TACC) and a network fault and delay generator Experiments can be isolated by request Additional partner machines may run FutureGrid software and be supported (but allocated in specialized ways)

Logical Network Diagram

Network Impairments Device Spirent XGEM Network Impairments Simulator for jitter, errors, delay, etc Full Bidirectional 10G w/64 byte packets up to 15 seconds introduced delay (in 16ns increments) 0-100% introduced packet loss in.0001% increments Packet manipulation in first 2000 bytes up to 16k frame size TCL for scripting, HTML for manual configuration

08/25/10http://futuregrid.org

FutureGrid Architecture Open Architecture allows to configure resources based on images Managed images allows to create similar experiment environments Experiment management allows reproducible activities Through our modular design we allow different clouds and images to be “rained” upon hardware. Will support deployment of preconfigured middleware including TeraGrid stack, Condor, BOINC, gLite, Unicore, Genesis II

Software Goals Open-source, integrated suite of software to – instantiate and execute grid and cloud experiments. – perform an experiment – collect the results – tools for instantiating a test environment TORQUE, Moab, xCAT, bcfg, and Pegasus, Inca, ViNE, a number of other tools from our partners and the open source community Portal for interacting with tools – Benchmarking 08/25/10http://futuregrid.org10

Draft GUI for FutureGrid Dynamic Provisioning Draft GUI for FutureGrid Dynamic Provisioning

Command line fg-deploy-image – host name – image name – start time – end time – label name fg-add – label name – framework hadoop – version 1.0 Deploys an image on a host Adds a feature to a deployed image 08/25/10http://futuregrid.org12

FG Stratosphere Objective – Higher than a particular cloud – Provides all mechanisms to provision a cloud on a given FG hardware – Allows the management of reproducible experiments – Allows monitoring of the environment and the results Risks – Lots of software – Possible multiple path to do the same thing Good news – We worked in a team, know about different solutions and have identified a very good plan – We can componentize Stratosphere 08/25/10http://futuregrid.org13

Change underlying system to support current user demands Linux, Windows, Xen/KVM, Nimbus, Eucalyptus Stateless images Shorter boot times Easier to maintain Stateful installs Windows Use Moab to trigger changes and xCAT to manage installs 08/25/1014http://futuregrid.org Dynamic Provisioning

Experiment Manager Objective – Manage the provisioning for reproducible experiments – Coordinate workflow of experiments – Share workflow and experiment images – Minimize space through reuse 08/25/10http://futuregrid.org15

Getting started with FutureGrid Account/Project request – Nimbus/Eucalyptus account for those systems, if needed Getting started information at Mail for any

Acknowledgements FutureGrid - NSF Award OCI NSF Solicitation – ViNe - Nimbus - Eucalyptus - VAMPIR - Pegasus -