1. Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier

2. ICL Benchmarking Activities Linpack benchmark ParkBench LLCbench The Performance Database Server TOP500 Performance Analysis Tool for NAS

3. Linpack Benchmark 3 main ‘flavors –100x100 –1000x1000 –NxN R max from NxN used in the TOP500

4. ParkBench Collection of –LowLevel benchmarks sequential MPI PVM –Kernels Linear Algebra NAS PB Kernels –Applications NAS PB (simulated applications) PSTSWM

5. LLCbench Low-Level Tests for various hardware aspects: MPBench –Core MPI contructs BLASBench –BLAS routines CacheBench –In and out of cache data movement

6. Performance Database Server Searchable Interface to DB of benchmark results such as: –Linpack –SPEC –(Dhrystone etc)

7. Basis for analysing the HCP market - Quantification of observations - Detection of trends market, architecture, technology

8. Listing of the 500 most powerful Computers in the World Yardstick: R max /LINPACK Updating halfyearly TOP500 Procedure

10. TOP500 list - Data shown ManufacturerManufacturer or vendor Computer Type indicated by manufacturer or vendor Inst. SiteCustomer LocationLocation and country YearYear of installation/last major update Field of Appl.Academic,Research,Industry,Vendor,Class. # Proc.Number of processors R max Maxmimal LINPACK performance achieved R peak Theoretical peak performance N max Problemsize for achieving R max N 1/2 Problemsize for achieving half of R max N world Position within the TOP500 ranking

11. TOP10

12. Performance Development

13. Manufacturer

14. Architectures

15. Performance Development

18. www.top500.org.

19. NAS Performance Analysis Tool To provide a tool to NAS users which allows –Access to performance data of NAS reference codes –Performing a variety of predefined performance tests on target systems in an easy and comparable ways –Analyzing user performance measurements using advanced statistical methods Analyze scalability of his/her application Compare its efficiency to other codes Get a first prediction of attainable performance ranges on other NAS systems

20. Necessary Functionality Broad variety of predefined performance tests Simple interface do define custom performance tests Reference measurement of NAS applications Generation and maintenance of performance database

21. Necessary Functionality Black Box performance analysis for individual codes Cross-code and cross-platform performance analysis. Tool for performance prediction for NAS applications Detection and analysis of system and code performance signatures

22. System Architecture Test suite setup file GUI: Benchmark Skeleton codes Test benchmarks Performance DB Connected by same keys Create script Test definition database - All predefined performance test with performance models Test results Execute script - Result commit to include in DB - Test suite setup - Test definition - Result display - Result analysis using various methods - User data entry

23. Functions available for non-linar regression Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis - best model Data and regression display GUI - Analysis Component