System Testing 12/09

Hierarchy of Testing Testing Program Testing Top Down Bottom Up Integration TestingUnit Testing System Testing Big Bang Sandwich Black Box White Box Function Performance Reliability Availability Acceptance Testing Properties Security Equivalence Boundary Decision Table State Transition Use Case Domain Analysis Control FlowData Flow Usability Documentation Portability Capacity Ad hoc Benchmark Pilot Alpha Beta

Describe the following types of System Testing Function Testing Performance Testing Acceptance Testing Installation Testing Groups

Types of System Testing Function Testing –Integrated system performs specified function Performance Testing –Integrated system meets non-functional requirement Acceptance Testing –Customers test system (beta tests) Installation Testing –If acceptance tests are in different environment

Function Testing Thread: Set of actions associated with a function Test one thread at a time Most of the black box techniques discussed apply here

Performance Testing Stress tests Volume tests Configuration tests Compatibility tests Security tests Timing tests Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests –Maximum throughput on all subsystems simultaneously Volume tests Configuration tests Compatibility tests Security tests Timing tests Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests Volume tests –Can system handle maximum amount of data Configuration tests Compatibility tests Security tests Timing tests Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests Volume tests Configuration tests –Can system handle specified configurations Compatibility tests Security tests Timing tests Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests Volume tests Configuration tests Compatibility tests –Does software work with other software Security tests Timing tests Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests Volume tests Configuration tests Compatibility tests Security tests –Does software pass basic security screen –(entire class here …) Timing tests Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests Volume tests Configuration tests Compatibility tests Security tests Timing tests –Response and throughput times Recovery tests Documentation tests Human factors tests

Performance Testing Stress tests Volume tests Configuration tests Compatibility tests Security tests Timing tests Recovery tests –Does system recover from known faults Documentation tests Human factors tests

Other things overload testing: exceed specifications robustness testing: test things not in specifications regression testing: test it again (build test bench) testing real time systems: a research problem –be aware the random number generators are not

Acceptance Testing Benchmark tests –Typical operation tests prepared by customer Pilot tests –Install and let users use system Alpha tests –In-house pilot Beta tests –On-site pilot