Rice01, slide 1 Characterizing NAS Benchmark Performance on Shared Heterogeneous Networks Jaspal Subhlok Shreenivasa Venkataramaiah Amitoj Singh University of Houston Heterogeneous Computing Workshop, April 15, 2002

Rice01, slide 2 Mapping/Adapting Distributed Applications on Networks Data Sim 1 Vis Sim 2 Stream Model Pre ? Application Network

Rice01, slide 3 Automatic node selection m-6 m-5 m-4 m-7 m-1 m-2 m-3 Congested route Compute nodes Routers m-8 Busy nodes selected nodes Select 4 nodes for execution : Choice is easy

Rice01, slide 4 Automatic node selection m-6 m-5 m-4 m-7 m-1 m-2 m-3 Congested route Compute nodes Routers m-8 Busy nodes selected nodes Select 5 nodes: choice depends on application

Rice01, slide 5 Mapping/Adapting Distributed Applications on Networks Data Sim 1 Vis Sim 2 Stream Model Pre ? ApplicationNetwork 1)Discover application characteristics and model performance in a shared heterogeneous environment 2)Discover network structure and available resources (e.g., NWS, REMOS) 3)Algorithms to map/remap applications to networks

Rice01, slide 6 Methodology for Building Application Performance Signature Performance signature = model to predict application execution time under given network conditions 1.Execute the application on a controlled testbed 2.Measure system level activity during execution –such as CPU, communication and memory usage 3.Analyze and discover program level activity (message sizes, sequences, synchronization waits) 4.Develop a performance signature No access to source code/libraries assumed

Rice01, slide 7 Discovering application characteristics 500MHz Pentium Duos ethernet switch (crossbar) 100 Mbps links Executable Application Code Benchmarking on a controlled testbed and analysis Model as a Performance Signature capture patterns of CPU loads and traffic during execution

Rice01, slide 8 Results in this paper Executable Application Code Benchmarking on a controlled testbed Measure performance with resource sharing Demonstrate that measured resource usage on a testbed is a good predictor of performance on a shared network for NAS benchmarks 500MHz Pentium Duos ethernet switch (crossbar) 100 Mbps links capture patterns of CPU loads and traffic during execution

Rice01, slide 9 Experiment Procedure Resource utilization of NAS benchmarks measured on a dedicated testbed –CPU probes based on “top” and “vmstat” utility –Bandwidth using “iptraf”, “tcpdump”, SNMP queries Performance of NAS benchmark measured with competing loads and limited bandwidth –Employ dummynet and NISTnet to limit bandwidth All measurements presented are on 500MHz Pentium Duos, 100 Mbps network, TCP/IP, FreeBSD All results on Class A, MPI, NAS Benchmarks

Rice01, slide 10 Discovered Communication Structure of NAS Benchmarks BT CG IS EP LU MG SP 2

Rice01, slide 11 Performance with competing computation loads Increase beyond 50% due to lack of coordinated (gang) scheduling and synchronization Correlation between low CPU utilization and smaller increase in execution time (e.g. MG shows only ~60% CPU utilization) Execution time is lower if least busy node has a competing load (20% difference in the busyness level for CG)

Rice01, slide 12 Performance with Limited Bandwidth (reduced from 100 to 10Mbps) on one link Close correlation between link utilization and performance with a shared or slow link

Rice01, slide 13 Performance with Limited Bandwidth (reduced from 100 to 10 Mbps) on all links Close correlation between total network traffic and performance with all shared or slow links

Rice01, slide 14 Results and Conclusions (not the last slide) Computation and communication patterns can be captured by passive, near non-intrusive, monitoring Benchmarked resource usage pattern is a strong indicator of performance with sharing –strong correlation between application traffic and performance with low bandwidth links –CPU utilization during normal execution a good indicator of performance with node sharing Synchronization and timing effects were not dominant for NAS Benchnmarks

Rice01, slide 15 Discussion and Ongoing Work (the last slide) Capture application level data exchange pattern from network probes (e.g. MPI message sequence, sizes) –slowdown different for different message sizes Infer the main synchronization/waiting patterns –Impact of unbalanced execution and lack of gang scheduling Capture impact of CPU scheduling policy for accurate prediction with sharing –Policies try to compensate for waits Goal is to build a quantitative “performance signature” to estimate execution time under any given network conditions, and use it in a resource management prototype system