1. CS 351/ IT 351 Modeling and Simulation Technologies HPC Architectures Dr. Jim Holten

2. CS 351/ IT 351 Overview Parallel Program Concerns Algorithm “Architectures” Data “Architectures” Hardware Architectures

3. CS 351/ IT 351 Parallel Program Concerns What are parallel processes? Why go parallel? What goes “parallel” in a program?

4. CS 351/ IT 351 Parallel Processes? The processor – size, speed, capability The communications – topology, types, shared links File I/O – access links topology, file system sizes, shared accesses Overall architecture models?

5. CS 351/ IT 351 Why parallel? Too large for one processor’s memory! Takes too long to execute! Amdahl’s law? T = S + P  T = S + P/m (  T = S + max(P i )) Lim(T) = S, as m  ∞

6. CS 351/ IT 351 What goes “parallel”? Data parallel Partition the data. Same algorithm, different data Task parallel Partition the code. Different algorithms, different data Task Parallel Pipeline Partition the code Different algorithms, same data (possibly augmented)

7. CS 351/ IT 351 Data Parallel Partition the primary data set. Project the partitioning to secondary data sets Identify data sharing –“Sharing” communications maps (where?) –Data set members’ needing shared (who?) –Actual attributes to be shared (what?)

8. CS 351/ IT 351 Data Parallel: Applications Mesh Models Top cells – primary data set Other cells – secondary (dependant) data sets Network Models Nodes – primary data set Links, agent models – secondary data sets Others?

9. CS 351/ IT 351 Task Parallel Code and data interdependencies Relatively independent code blocks Identify each code block’s needs Initial data into process (from another) Run time data sharing Result data from process (to another)

10. CS 351/ IT 351 Task Parallel: Applications Independent operations System management (Comm, I/O, load monitoring) User interface control – different user categories Many pattern finders, one data set Data mining Computer vision One stage in pipeline

11. CS 351/ IT 351 Task Parallel Pipeline Successive stages of separate algorithms Data stream input to first stage Each stage’s output is next stage’s input Data stream output from last stage

12. CS 351/ IT 351 Task Parallel Pipeline: Applications Computer vision Surveillance Robotics Real-time sensor monitoring Large data set scanning Data mining Security scans

13. CS 351/ IT 351 Combining Parallel Approaches Keep up with stream data rates Pipeline stages must execute within sample acquisition windows (frame rate) Video, radar scan, sensor suite acquisition frame rate Other sensor data blocks – audio blocks for frequency calculations Number of stages gives fixed delay for results, but at frame rate.

14. CS 351/ IT 351 Combining Parallel Approaches Pipeline stage to slow? Due to too much data? – make it data parallel Due to too many patterns to check? – make it task parallel Due to too long an algorithm execution? – pipeline that step (increasing overall delay!)