1 Software Testing and Quality Assurance Lecture 31 – Testing Systems

2 Lecture Objectives Types of Testing Measuring Test Coverage Software Maintenance Activities

3 Types of testing Stress testing Life-cycle testing Performance testing

4 Types of testing: stress testing Stress testing is operating a system under conditions that come close to exhausting the resources needed by the system: Filling RAM with objects. Filling hard disk with records. Move the mouse back and forth. Natural growth of information that accumulates as the system is operated. For OO systems: Create large number of instances of classes

5 Types of testing: steps in stress testing Identify the variable resources that increase the amount of work the system has to do If there are relationships among these resources, develop a matrix that lists combinations of resource levels to use Create test cases that use each combination Execute and evaluate the results

6 Types of testing: life-cycle testing Two types of life cycles that do make sense to test: Domain life cycles: these correspond to the key processes in the domain (for accounting system run a series of test to cover a full fiscal year). Computer application life cycles: it begins with the installation of the application and ends with its removal (test the installer and uninstaller programs).

7 Types of testing: performance testing Perception: object-oriented systems are inherently slow. Testing for performance is like measuring the reliability of a piece of software. Establishing the context within which the performance will be measured: Context: description of the environment in which the measurement will be made.

8 Types of testing: performance testing technique Define the context in which performance measure applies: Describe the state of the application to be tested. Describe the execution environment in terms of the platform being used. Describe the other applications running at the time of tests

9 Types of testing: performance testing technique Identify the extremes within the context. Define, as expected result, what will constitute acceptable performance. Execute the tests and evaluate the results.

10 Types of testing: problems with real data in testing Problems with real data in testing: Extensive past data available. Analysis of the data requires time.

11 Types of testing: problems with real data in testing It might be quicker to manufacture data that has specific properties than to use real data: Analyze the existing data to identify patterns. Construct test data that follow these patterns but for which the expected results are more easily determined Design test cases that use the test data in the context of the of a complete life cycle. Execute and evaluate the results.

12 Testing different types of systems Reactive systems Embedded systems Multitiered systems Distributed systems

13 Measuring test coverage Coverage: is the metrics that indicates level of confidence we have in our testing. What is to be covered? To measure coverage two categories can be used: Inputs: how many of the possible inputs have been used in test cases. Outputs: how many of the possible outputs that a system can produce have been produced during testing.

14 Measuring test coverage As systems have become more integrated into Life-critical and mission- critical systems, Expectations have increased to include “does the system do any thing it is not suppose to do?”

15 Measuring test coverage (cont...) When is coverage measured? Coverage data is collected continuously during testing When is coverage used? “at release time”, coverage is part of the decision criteria for release.

16 Software maintenance activities Corrective maintenance are activities of handling problems resulting from faults. Adaptive Maintenance involves adjusting the application to changes in the environment, or changes from other parts of the system.

17 Software maintenance activities Perfective maintenance involves making changes to improve some aspect of the system, even when the changes are not suggested by faults. Preventive maintenance involves changing some aspect of the system to prevent failures.

18 Software maintenance activities (cont...)

19 Key points Types of testing: stress testing, life- cycle testing, and performance testing Coverage: is the metrics that indicates level of confidence we have in our testing.

20 Announcement SWE Revised Program Discussion Monday December 29, 2008 at 12:10-1:00 in 24/141