1. - 1 - Workshop on Pattern Analysis Data Flow Pattern Analysis of Scientific Applications Michael Frumkin Parallel Systems & Applications Intel Corporation May 6, 2005

2. - 2 - Workshop on Pattern Analysis Outline  Why Data Flow Pattern Analysis?  CFD Applications  The NAS Parallel Benchmarks  The NAS Grid Benchmarks  Trace File Analysis  Conclusions

3. - 3 - Workshop on Pattern Analysis Why Data Flow Pattern Analysis?  Scientific applications –model few natural processes –new effects are added infrequently –influence on the existing data flows are insignificant  Knowledge of data flow in program helps with –program understanding –program optimization, parallelization, multithreading –building application performance model

4. - 4 - Workshop on Pattern Analysis Design of Scientific Applications  Time represented as an outer loop –Iterations over time step  Space is represented by structured/unstructured grids –Important for understanding data locality –Data access patterns –Spatial parallelism  Physics is represented by an operator at each grid point –Data flow –Operator level of parallelism/dependence

5. - 5 - Workshop on Pattern Analysis CFD Data Flow Patterns  Solve the Navier-Stokes equation K(u i+1 )=Lu i K(u i+1 )=Lu i –u is five-dimensional vector –K is non-linear operator  Solver  RHS computation

6. - 6 - Workshop on Pattern Analysis ADI Pattern x-solve y-solve z-solve Multipartition x-solvey-solvez-solve ADI method  ADI method K~Kx*Ky*Kz  Multilevel parallelism

7. - 7 - Workshop on Pattern Analysis BT Communication

8. - 8 - Workshop on Pattern Analysis Explicit Operators  Stencil operators (explicit methods)  At each point of a 3-dimensional mesh apply: seven-point27-point

9. - 9 - Workshop on Pattern Analysis )  Two-dimensional pipeline  Hyperplane algorithm -1 0 0 1 0 0 0 -1 0 0 1 0 0 0 -1 0 0 1 Dependence Matrices (() Lower-Upper Triangular

10. - 10 - Workshop on Pattern Analysis LU Communication

11. - 11 - Workshop on Pattern Analysis Multigrid V-Cycle Interpolation & Smoothing Smoothing Projection Projection Projection Projection

12. - 12 - Workshop on Pattern Analysis MG Communication

13. - 13 - Workshop on Pattern Analysis BT x_solve (serial) Call Graph Data Flow Analysis do k=1,ksize do i=1,isize do j=1,jsize

14. - 14 - Workshop on Pattern Analysis Nest Data Flow Graph Nest Data Flow Graph do_45do_134do_330 Each arc represents Affinity Relation

15. - 15 - Workshop on Pattern Analysis NAS Parallel Benchmarks  Application Benchmarks –CFD –BT, SP, LU – Data Intensive –DC, DT, BTIO – Computational Chemistry –UA  Kernel Benchmarks –FT, CG, MG, IS  Verification  Performance Model  FORTRAN, C, HPF, Java*  Serial, MPI, OpenMP, Java* Threads www.nas.nasa.gov/Software/NPB * Other names and brands may be claimed as the property of others.

16. - 16 - Workshop on Pattern Analysis NPB Performance on Altix* ** * Other names and brands may be claimed as the property of others. ** Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing.

17. - 17 - Workshop on Pattern Analysis Basic Data Flow Patterns  Shuffles –Sorting –FFT –Routing  Gather/Scatter –Conjugate Gradient –MD and FE codes –Sparse matrices  Transpose –FFT –Sorting  Tree –Parallel prefix, Reduction –Sorting

18. - 18 - Workshop on Pattern Analysis HPC Challenge Benchmarks  HPL*  DGEMM*  STREAM*  PTRANS*  FFTE*  RandomAccess*  Effective Bandwidth b_eff* icl.cs.utk.edu/hpcc * Other names and brands may be claimed as the property of others.

19. - 19 - Workshop on Pattern Analysis Programming With Directed Graphs  Arc –Arc* newArc(Node *tail, Node *head) –AttachArc(DGraph *dg) –deleArc(Arc *ar)  Node –newNode(char *name) –Node* AttachNode(DGraph *dg) –deleteNode(Node *nd)  DGraph –DGraph* newDGraph(char *name) –writeGraph(DGraph *dg, char* fname) –DGraph * readGraph(char* fname) Implemented in DT of NPB and in NGB do_134

20. - 20 - Workshop on Pattern Analysis Directed Graphs Around  Parse trees  File Systems  Application task graphs  Device Schematics  VCG tool  Edge tool  Tom Sawyer Software  Commercial tools Visualization and layout Tools

21. - 21 - Workshop on Pattern Analysis Cart3D*  Performs CFD analysis on complex geometries  Uses six executables –Intersect* – intersects geometry –Cubes* – produces Cartesian meshes –Reorder* – reorders meshes –Mgprep* – coarsens mesh –flowCart* – convergence acceleration –Clic* – analyzes the flow  Executables communicate via files  Returns relevant forces –Lift, Drag, Side Force Task Graphs are rapidly growing * Other names and brands may be claimed as the property of others.

22. - 22 - Workshop on Pattern Analysis The NAS Grid Benchmarks  Reflect task level programming paradigm  Contain four patterns –Embarrassingly Distributed (ED) –Helical Chain (HC) –Visualization Pipeline (VP) –Mixed Bag (MB) Launch Report SP Embarrassingly Distributed (ED) Launch Report BT MG FT BT MG FT BT MG FT Visualization Pipeline (VP) Report Launch FT 8 FT 2 LU 2 LU 4 LU 8 MG 4 MG 8 MG 2 Mixed Bag (MB) #steps Report Launch BTSPLU BTSPLU BTSPLU Helical Chain (HC)

23. - 23 - Workshop on Pattern Analysis Data Dependent Patterns  Intermittent patterns –Useful for application performance tuning  Visualization is important –Allows to employ human eye ability to detect patterns  Automatic Pattern Mining –OLAP approach  MPI communication patterns Automatic Trace Analysis Using OLAP

24. - 24 - Workshop on Pattern Analysis Conclusions Data Flow in Applications Data Flow in Applications  Application Parallelization  Application Understanding  Application Mapping  Application Performance