1. © Copyright 2008, SoftWell Performance AB 1 Performance Testing Distributed Systems Concepts and Terminology v0.6.1

2. © Copyright 2008, SoftWell Performance AB 2 Performance Testing - Concepts and Terminology The document structure The document describes performance testing in two parts: General performance concepts Performance testing concepts

3. © Copyright 2008, SoftWell Performance AB 3 Performance Testing - Concepts and Terminology The document structure MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology General performance concepts 7. Abstract Performance metrics 6. Measurement data objectives and attributes 5. Measured objects 4. Performance characteristics 8. Performance data processing 9. General performance test concepts 10. Performance test environment 11. Performance test specification concepts 12. Workload concepts Performance testing concepts

4. © Copyright 2008, SoftWell Performance AB 4 Part 1 General performance concepts

5. © Copyright 2008, SoftWell Performance AB 5 Performance Testing - Concepts and Terminology Performance characteristics MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 4. Performance characteristics Categorizing performance characteristics Powerfulness characteristics Reliability characteristics Efficiency characteristics

6. © Copyright 2008, SoftWell Performance AB 6 Performance Testing - Concepts and Terminology Measured objects MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 5. Measured objects Measured services Measured components Service concepts Service characteristics Service interfaces

7. © Copyright 2008, SoftWell Performance AB 7 Performance Testing - Concepts and Terminology Measurement data objectives and attributes MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 6. Measurement data objectives and attributes Performance metric objectives Measurement data attribute sets Identification attributes Unit attributes Conditional attributes Processing attributes Metric types

8. © Copyright 2008, SoftWell Performance AB 8 Performance Testing - Concepts and Terminology Abstract performance metrics MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 7. Abstract Performance metrics Abstract powerfulness metrics Abstract reliability metrics Abstract efficiency metrics

9. © Copyright 2008, SoftWell Performance AB 9 Performance Testing - Concepts and Terminology Performance data processing MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 8. Performance data processing Steps in performance data processing Collection and storage of raw performance data Condensation and normalization of raw performance data Performance data computations Evaluation of performance data Presentation of performance data

10. © Copyright 2008, SoftWell Performance AB 10 Part 2 Performance testing concepts

11. © Copyright 2008, SoftWell Performance AB 11 Performance Testing - Concepts and Terminology General performance test concepts MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 9. General performance test concepts Performance test concepts Performance test phases Performance test objectives Performance objectives and performance requirements Performance measurement conditions Performance targets Performance measurement standards Some performance measurement characteristics

12. © Copyright 2008, SoftWell Performance AB 12 Performance Testing - Concepts and Terminology Performance test environment MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 10. Performance test environment Test environment concepts System Under Test concepts Test System concepts

13. © Copyright 2008, SoftWell Performance AB 13 Performance Testing - Concepts and Terminology Performance test specifications MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 11. Performance test specifications Elements of performance test specifications Test objectives Test conditions Test configurations Test data specifications Test evaluation specifications

14. © Copyright 2008, SoftWell Performance AB 14 Performance Testing - Concepts and Terminology Workload concepts MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 12. Workload concepts Workload set or Traffic set Workload content Workload volume Load concepts Workload time distribution

15. © Copyright 2008, SoftWell Performance AB 15 What is next?

16. © Copyright 2008, SoftWell Performance AB 16 Performance Testing – Next step Proposal The next document should describe Methods in all phases of performance tests How methods are applied The purpose is to Describe methodology of performance testing in a generalized manner Describe how methods are applied Start the groundwork on a framework for performance testing based on a common view of abstract performance metrics and characteristics MTS – Performance Testing Distributed Systems 2 – Methodology and Realization

17. © Copyright 2008, SoftWell Performance AB 17 MTS objectives for Performance Testing

18. © Copyright 2008, SoftWell Performance AB 18 MTS objectives for Performance Testing The MTS objectives should be to: Develop a formalized view of performance Develop a formalized view of performance test methods Develop a formalized view of performance tests That can be used as: A framework for applied performance tests and test suites A guideline for design of performance tests tools A guideline for performance test standards

19. © Copyright 2008, SoftWell Performance AB 19 Some long term goals on Performance and Performance Testing

20. © Copyright 2008, SoftWell Performance AB 20 Long term goals for Performance Testing Performance is an increasingly important aspect of a design or a system. Performance will be an increasingly important aspect of a standards. A validated function works under most circumstances in production, validated performance is not a bulletproof guarantee for production. A formalized view of performance and performance testing enables: A complete specification of a system Modeling and design of a system’s performance Simple monitoring of performance in design and production Far better ways for predicting performance in production Intelligent performance test tools that can be built into the system A continuously improved understanding of performance A safer investment in software development Some examples

21. © Copyright 2008, SoftWell Performance AB 21 “To measure is to know” - Lord Kelvin