Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier

ICL Benchmarking Activities Linpack benchmark ParkBench LLCbench The Performance Database Server TOP500 Performance Analysis Tool for NAS

Linpack Benchmark 3 main ‘flavors –100x100 –1000x1000 –NxN R max from NxN used in the TOP500

ParkBench Collection of –LowLevel benchmarks sequential MPI PVM –Kernels Linear Algebra NAS PB Kernels –Applications NAS PB (simulated applications) PSTSWM

LLCbench Low-Level Tests for various hardware aspects: MPBench –Core MPI contructs BLASBench –BLAS routines CacheBench –In and out of cache data movement

Performance Database Server Searchable Interface to DB of benchmark results such as: –Linpack –SPEC –(Dhrystone etc)

Basis for analysing the HCP market - Quantification of observations - Detection of trends market, architecture, technology

Listing of the 500 most powerful Computers in the World Yardstick: R max /LINPACK Updating halfyearly TOP500 Procedure

TOP500 list - Data shown ManufacturerManufacturer or vendor Computer Type indicated by manufacturer or vendor Inst. SiteCustomer LocationLocation and country YearYear of installation/last major update Field of Appl.Academic,Research,Industry,Vendor,Class. # Proc.Number of processors R max Maxmimal LINPACK performance achieved R peak Theoretical peak performance N max Problemsize for achieving R max N 1/2 Problemsize for achieving half of R max N world Position within the TOP500 ranking

TOP10

Performance Development

Manufacturer

Architectures

Performance Development

NAS Performance Analysis Tool To provide a tool to NAS users which allows –Access to performance data of NAS reference codes –Performing a variety of predefined performance tests on target systems in an easy and comparable ways –Analyzing user performance measurements using advanced statistical methods Analyze scalability of his/her application Compare its efficiency to other codes Get a first prediction of attainable performance ranges on other NAS systems

Necessary Functionality Broad variety of predefined performance tests Simple interface do define custom performance tests Reference measurement of NAS applications Generation and maintenance of performance database

Necessary Functionality Black Box performance analysis for individual codes Cross-code and cross-platform performance analysis. Tool for performance prediction for NAS applications Detection and analysis of system and code performance signatures

System Architecture Test suite setup file GUI: Benchmark Skeleton codes Test benchmarks Performance DB Connected by same keys Create script Test definition database - All predefined performance test with performance models Test results Execute script - Result commit to include in DB - Test suite setup - Test definition - Result display - Result analysis using various methods - User data entry

Functions available for non-linar regression Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis - best model Data and regression display GUI - Analysis Component