1. Gary M. Zoppetti Gagan Agrawal
Compiler and Runtime Support for Adaptive Irregular Applications on a Multithreaded Architecture
Gary M. Zoppetti
Gagan Agrawal
In this talk I will describe my thesis work…

2. Multithreaded Architectures (MTA’s)
MTA characteristics
multiple threads of execution in hardware
each processor maintains several loci of control
fast thread switching
efficient communication & synchronization mechanisms
special hardware and/or software runtime system (RTS) support
MTA capabilities
masking communication and synchronization latencies
dynamic load balancing
Well-suited for irregular applications
MTAs overcome conventional arch limitations.
maintain several contexts
First 2 achieved thru RTS support
Unstructured communication typically results in high latency so the ability to mask latency is important
Dynamic control flow typically results in load imbalances so the ability of the architecture to adapt to a changing workload is vital

3. Unstructured Mesh Processing
for (tstep = 0; tstep < num_steps; tstep++) {
for (i = 0; i < num_edges; i++) {
node1 = nodeptr1[i];
node2 = nodeptr2[i];
force = h(node1, node2);
reduc1[node1] += force;
reduc1[node2] += -force; }
Unstructured mesh is used to model an irregular geometry (airplane wing, particle interactions which are inherently sparse)
Time-step loop iterates until convergence
Point out indirection arrays
reduction arrays
Associative, commutative operator

4. Irregular Reductions Irregular Reduction Loops
Elements of LHS arrays may be incremented in multiple iterations, but only using commutative & associative operators
No loop-carried dependences other than those on elements of the reduction arrays
One or more arrays are accessed using indirection arrays
Codes from many scientific & engineering disciplines contain them (simulations involving irreg. meshes, molecular dynamics, sparse codes)
Irregular reductions well-studied for DM, DSM, and cache optimization
Compute-intensive

5. Execution Strategy Overview
Partition edges (interactions) among processors
Challenge: updating reduction arrays
Divide reduction arrays into NUM_PROCS portions – revolving ownership
Execute NUM_PROCS phases on each processor
--each processor eventually will own every reduction array portion

6. Execution Strategy
To exploit multithreading, use (k*NUM_PROCS) phases and reduction portions P0 P1 P2 P3
Reduction Portion # 1
Phase 0 2
4 processors, k = 2, so 8 phases
Ownership of reduction portions is offset by a factor of k
K provides opportunity for overlap of computation with communication by way of intervening phases
Phase 2 3 4 5 6 7

7. Execution Strategy (Example)
for (phase = 0; phase < k * NUM_PROCS; phase++) {
Receive (reduc1_array_portion)
from processor PROC_ID + 1;
// main calculation loop
for (i = loop1_pt[phase]; i < loop1_pt[phase + 1]; i++) {
node1 = nodeptr1[i];
node2 = nodeptr2[i];
force = h(node1, node2);
reduc1[node1] += force;
reduc1[node2] += -force; }
: : :
Send (reduc1_array_portion)
to processor PROC_ID - 1;
--send & receive is asynchronous
--usually 2 indirection arrays that represent an edge or interaction
--iterate over the edges local to the current phase

8. Execution Strategy
Make communication independent of data distribution and values of indirection arrays
Exploit MTA’s ability to overlap communication & computation
Challenge: partition iterations into phases (each iteration updates 2 or more reduction array elements)
--2 goals of
Mention inspector/executor approach
--total communication volume = NUM_PROCS * REDUCTION_ARRAY_SIZE

9. Execution Strategy (Updating Reduction Arrays)
Edge (4, 0)  Phases 2, 0  Phase 0
buffer node 4’s value during Phase 0 and
update from buffer during Phase 2
// main calculation loop
for (i = loop1_pt[phase]; i < loop1_pt[phase + 1]; i++) {
node1 = nodeptr1[i];
node2 = nodeptr2[i];
force = h(node1, node2);
reduc1[node1] += force;
reduc1[node2] += -force; }
// update from buffer loop
for (i = loop2_pt[phase]; i < loop2_pt[phase + 1]; i++ {
local_node = lbuffer_out[i];
buffered_node = rbuffer_out[i];
reduc1[local_node] += reduc1[buffered_node];
Suppose we assign to lesser of two phases
Compiler creates a second loop

10. Runtime Processing Responsibilities
Divide iterations on each processor into phases
Manage buffer space for reduction arrays
Set up second loop
// runtime preprocessing on each processor
LightInspector (. . .);
for (phase = 0; phase < k * NUM_PROCS; phase++) {
Receive (. . .);
// main calculation loop
// second loop to update from buffer
Send (. . .); }
To make the execution strategy possible, the runtime processing is responsible for…
Call it LightInspector b/c it is significantly lighter weight than traditional inspector – no inter-processor communication is required

11. Runtime Processing (Example)
4 buffers 2 4 6 8
reduc1
remote area
Input: 7 4 1
nodeptr1
nodeptr2
Output:
Phase # 1 2 3
nodeptr1_out
nodeptr1_out 9
nodeptr2_out 1
K=2, 2 processors, thus 4 phases
Number of nodes (vertices) = 8
Phase # 1 2 3
copy1_out 4
copy2_out 9

12. Compiler Analysis Identify reduction array sections
updated through an associative, commutative operator
Identify indirection array (IA) sections
Form reference groups of reduction array sections accessed through same IA sections
Each reference group can use same LightInspector
EARTH-C compiler infrastructure
Now I’ll present the compiler analysis that utilizes the execution strategy and runtime processing previously described

13. Adaptive Codes Could just re-run inspector
for (tstep = 0; tstep < num_steps; tstep++) {
if (tstep % update_freq == 0)
update (nodeptr1, nodeptr2); // update IAs
for (i = 0; i < num_edges; i++) {
node1 = nodeptr1[i];
node2 = nodeptr2[i];
force = h(node1, node2);
reduc1[node1] += force;
reduc1[node2] += -force; }
Could just re-run inspector
Developed incremental inspector and a pre-incremental inspector

14. Runtime Processing (Adaptive)2 4 6 8
reduc1
remote area
Input: 7
Extra space
for phase
insertions
nodeptr1 4
nodeptr2 2
Output: 9 9 2
Phase # 1 2 3
nodeptr1_out
nodeptr2_out
Simple example to convey idea
Phase # 1 2 3
copy1_out 4
copy2_out 9

15. Runtime Processing (Adaptive)
Incremental inspector
One iteration over edges, comparing new and old indirection array values
Edges (iterations in 1st loop) move to new phases if necessary (28 cases)
Update edges (iterations in 2nd loop) are modified if necessary
Reuse buffer locations when possible
Pre-incremental inspector
Similar to non-adaptive inspector
Extra space allocated for edge movement (mappings maintained for efficiency)
Saves values for subsequent runs of the incremental inspector

16. Experimental Results Euler 10k p = 0.02 Euler 10k p = 0.10
We’ve introduced two parameters: ‘p’ is extent of adaptivity, probability an edge changes
Iters is rate of adaptivity, # of iterations before the IA’s are modified
Key kernel – whole benchmark would allow better amortization of overhead
Iters = 5 is a little unrealistic: at minimum around 10
Iters = 5: Abs 0.92 Relative (32): 11.97
Iters = 20: Abs 1.14 Relative (32): 10.35
Iters = 5: Abs 0.49 Relative (32):15.03
Iters = 20: Abs 0.92 Relative (32): 9.63

17. Experimental Results Moldyn 2k p = 0.02 Moldyn 2k p = 0.10
Iters = 5: Abs 1.10 Relative 10.61
Iters = 20: Abs 1.22 Relative 10.01
Iters = 5: Abs 0.80 Relative 11.84
Iters = 20: Abs 1.07 Relative 9.84

18. Summary and Conclusions
Class II
frequency and volume of communication independent of contents of indirection arrays
no mesh partitioning or communication optimizations required
initially incur overheads (locality), but high relative speedups
near linear scaling of inspector times wrt number of processors and extent of adaptivity