1. Center for High Performance Software
Why Performance Models Matter for Grid Computing
Ken Kennedy
Center for High Performance Software
Rice University

2. Vision: Global Distributed Problem Solving
Where We Want To Be
Transparent Grid computing
Submit job
Find & schedule resources
Execute efficiently
Where We Are
Some simple (but useful) tools
Programmer must manage:
Heterogeneous resources
Scheduling of computation and data movement
Fault tolerance and performance adaptation
What Do We Propose as A Solution?
Separate application development from resource management
Through an abstraction called the Virtual Grid
Provide tools to bridge the gap between conventional and Grid computation
Scheduling, resource management, distributed launch, simple programming models, fault tolerance, grid economies
Database
Supercomputer
Database
Supercomputer
Intent of this slide is to define the problem
Key points:
Want to enable researcher sitting in Houston to use significant resources at other sites (e.g. use supercomputers to correlate huge genome databases)
This is too hard today - requires detailed systems programming, rare expertise, not feasible for the discipline scientist (unless she has a collaboration with the CS department)
VGrADS is raising this level of abstraction

3. The VGrADS Team
VGrADS is an NSF-funded Information Technology Research project
Keith Cooper
Ken Kennedy
Charles Koelbel
Richard Tapia
Linda Torczon
Dan Reed
Jack Dongarra
Lennart Johnsson
Can also use this slide to thank our program managers (Frederica Darema, Kamal Abdali, Mike Foster)
Students involved in demos at SC:
Wahid Chrabakh (GridSAT, UCSB)
Anirban Mandal (EMAN, Rice)
Bo Lu (EMAN, UH)
Carl Kesselman
Rich Wolski
Fran Berman
Plus many graduate students, postdocs, and technical staff!

4. VGrADS Project Vision Virtual Grid Abstraction
Separation of Concerns
Resource location, reservation, and management
Application development and resource requirement specification
Permits true scalability and control of resources
Tools for Application Development
Easy application scheduling, launch, and management
Off-line scheduling of workflow graphs
Automatic construction of performance models
Abstract programming interfaces
Support for Fault Tolerance and Reliability
Collaboration between application and virtual grid execution system
Research Driven by Real Application Needs
EMAN, LEAD, GridSAT, Montage

5. VGrADS Application Collaborations
EMAN Electron Micrograph Analysis
GridSAT Boolean Satisfiability
BPEL Workflow
Engine
LDM Service
GridFTP
Service
WRF Service
vgES
Information
Rice
Scheduler
Ensemble
Broker
Visualization
Data arrives
Resource
Data Mining
Start
End
Static
Workflow
Dynamic
LEAD Atmospheric Science
Montage Astronomy

6. VGrADS Big Ideas Virtualization of Resources
Application specifies required resources in Virtual Grid Definition Language (vgDL)
Give me a loose bag of 1000 processors, with 1 GB memory per processor, with the fastest possible processors
Give me a tight bag of as many Opterons as possible
Virtual Grid Execution System (vgES) produces specific virtual grid matching specification
Avoids need for scheduling against the entire space of global resources
Generic In-Advance Scheduling of Application Workflows
Application includes performance models for all workflow nodes
Performance models automatically constructed
Software schedules applications onto virtual Grid, minimizing total makespan
Including both computation and data movement times

7. Workflow Scheduling Results
Dramatic makespan reduction of offline scheduling over online scheduling — Application: Montage
Value of performance models and heuristics for offline scheduling — Application: EMAN
”Scheduling Strategies for Mapping Application Workflows onto the Grid”
HPDC’05
”Resource Allocation Strategies for Workflows in Grids”
CCGrid’05

8. Virtual Grid Results
Virtual Grid prescreening of resources produces schedules dramatically faster without significant loss of schedule quality
Huang, Casanova and Chien. Using Virtual Grids to Simplify Application Scheduling. IPDPS 2006.
Zhang, Mandal, Casanova, Chien, Kee, Kennedy and Koelbel. Scalable Grid Application Scheduling via Decoupled Resource Selection and Scheduling. CCGrid06.
Current VGrADS heuristics are quadratic in number of distinct resource classes
Two-phase approach brings this much closer to linear time

9. Performance Model Construction
Problem: Performance models are difficult to construct
By-hand and models take a great deal of time
Accuracy is often poor
Solution (Mellor-Crummey and Marin): Construct performance models automatically
From binary for a single resource and execution profiles
Generate a distinct model for each target resource
Current results
Uniprocessor modeling
Can be extended to parallel MPI steps
Memory hierarchy behavior
Models for instruction mix
Application-specific models
Scheduling using delays provided as machine specification

10. Performance Prediction Overview
Object Code
Dynamic Analysis
Binary Instrumenter
Instrumented Code
Execute
Binary Analyzer
Communication Volume & Frequency
Memory Reuse Distance
BB Counts
Control flow graph
Loop nesting structure
BB instruction mix
Post Processing Tool
Our approach aims at building parameterized models of black-box applications in a semi-automatic way, in terms of architecture-neutral Application Signatures. We build models using information from both static and dynamic analysis of an application’s binary. We use static analysis to construct the control flow graph of each routine in an application and to look at the instruction mix inside the most frequently executed loops.
We use dynamic analysis to collect data about the execution frequency of basic blocks, information about synchronization among processes and reuse distance of memory accesses.
By looking at application binaries instead of source code, we are able to build language-independent tools and we can naturally analyze applications that have modules written in different languages or are linked with third party libraries. By analyzing binaries, the tool can also be useful to both application writers and to compiler developers by enabling them to validate the effectiveness of their optimizations.
Performance Prediction for Target Architecture
Static Analysis
Architecture neutral model
Scheduler
Architecture Description
Post Processing

11. Modeling Memory Reuse Distance
- it is a histogram for a single program reference
- explain what you mean by normalized frequency
- explain that you are showing measured data then the model.
- it shows surprisingly complex behavior
- at least three qualitatively different types of behavior
- constant
- slowly growing (likely linear)
- likely quadratic behavior

12. Execution Behavior: NAS LU 2D 3.0

13. Value of Performance Models
True Grid Economy
All resources have a “rate of exchange”
Example: 1 CPU unit Opteron = 2 units Itanium (at same Ghz)
Rate of exchange determined by averaging over all applications
Weighted by application global resource usage percentage
Improved Global Resource Utilization
A particular application may be able to take advantage of performance models to reduce costs
Example: For EMAN, 1 Opteron unit = 3 Itanium units
Thus, EMAN should always favor Opterons when available
If all applications do this, the total system resources will be used far more efficiently

14. Performance Models and Batch Scheduling
Currently VGrADS supports scheduling using estimated batch queue waiting times
Batch queue estimates are factored into communication time
E.g., the delay in moving from one resource to another is data movement time + estimated batch queue waiting time
Unfortunately, estimates can have large standard deviations
Next phase: limiting variability through two strategies:
Resource reservations: partially supported on the TeraGrid and other schedulers
In-advance queue insertion: submit jobs before data arrives based on estimates
Can be used to simulate advance reservations
Exploiting this requires a preliminary schedule indicating when the resources are needed
Problem: how to build an accurate schedule when exact resource types are unknown

15. Solution to Preliminary Scheduling Problem
Use performance models to specify alternative resources
For step B, I need the equivalent of 200 Opterons, where 1 Opteron = 3 Itanium = 1.3 Power 5
Equivalence from performance model
This permits an accurate preliminary schedule because the performance model standardizes the time for each step
Scheduling can then proceed with accurate estimates of when each resource collection will be needed
Makes advance reservations more accurate
Data will arrive neither egregiously early nor egregiously late
It may provide a mixture to meet the computational requirements, if the specification permits
Give me a loose bag of tight bags containing the equivalent of 200 Opterons, minimize the number of tight bags and the overall cost
Solution might be 150 Opterons in one cluster and 150 Itaniums in another

16. The LEAD Vision: Adaptive Cyberinfrastructure
DYNAMIC OBSERVATIONS
Analysis/Assimilation
Quality Control
Retrieval of Unobserved
Quantities
Creation of Gridded Fields
Prediction/Detection
PCs to Teraflop Systems
Product Generation,
Display,
Dissemination
Models and Algorithms Driving Sensors
The CS challenge: Build cyberinfrastructure services that provide adaptability, scalability, availability, usability, and real-time response.
End Users
NWS
Private Companies
Students

17. Scheduling to a Deadline
LEAD Project has a feedback loop
After each preliminary workflow, results are used to adjust Doppler radar configurations to get more accurate information in the next phase
This must be done on a tight time constraint
Performance models make this scheduling possible
What happens if the first schedule misses the deadline?
The time must be reduced, either by doing less computation or by using more resources
But how many more resources?
Suppose we can differentiate the model to compute for each step, the sensitivity of running time to more resources
Automatic differentiation technologies exist, but other strategies may also make this possible
Derivatives can be used to predict resources needed to meet the deadline along the critical path

18. LEAD Workflow
Terrain data files
NAM, RUC, GFS data 3 7 1
3D Model Data
Interpolator
(Initial Boundary Conditions)
3D Model Data
Interpolator
(lateral Boundary Conditions)
Terrain
Preprocessor 11
IDV
Bundle 4
Radar data
(level II)
Surface data,
upper air mesonet data,
wind profiler
88D Radar
Remapper 9 10
Radar data
(level III) 5
ARPS to WRF Data
Interpolator
WRF
NIDS Radar
Remapper
ADAS 12 6
Satellite data
WRF to ARPS Data
Interpolator
Satellite Data
Remapper 8
Surface, Terrestrial
data files 13
ARPS Plotting
Program 2
Triggered if a storm is detected
Run Once per
forecast Region
WRF Static
Preprocessor
Repeated for periodically
for new data
Visualization on users request

19. Maximizing Robustness
Two sources of robustness problems
Running time estimates are really probabilistic, with a distribution of values
Critical path might vary if some tasks take longer than expected
Resources to which steps are assigned might fail
Solution Strategies
Schedule the workflow to minimize the probability of exceeding the deadline
Allocate the slack judiciously
Use sensitivity-based strategy described earlier
Replication driven by reliability history can be used to overcome this problem to within a specified tolerance
More of a less reliable resource might be better
Note of caution: Tolerance cannot be too small if costs are to be managed

20. Summary VGrADS Project Uses Two Mechanisms for Scheduling
Abstract resource specification to produce preliminary collection of resources
More precise scheduling on abstract collection using performance models
Combined strategy produces excellent schedules in practice
Performance models can be constructed automatically
New Challenges Require Sophisticated Methods
Minimizing cost of application execution
Scheduling with batch queues and advanced reservations
Scheduling to deadlines
Scheduling for robustness
Current Driving Application
LEAD: Linked Environments for Atmospheric Discovery
Requires deadlines and efficient resource utilization

21. Relationship to Future Architectures
Future supercomputers will have heterogeneous components
Cray, Cell, Intel, CPU + coprocessor (GPU), …
Scheduling subcomputations onto units while managing data movement costs will be the key to effective usage of such systems
Adapt VGrADS strategy
Difference: must consider alternative compilations of the same computation, then model performance of the result
Could be used to tailor systems or chips to particular users’ application mixes