1. Software Testing and Quality Assurance
Lecture 29 - Introduction to Software Testing

2. Lecture Outline Introduction to testing types. Testing activities.
Introduction to test cases, test oracles and their execution.

3. Testing Policy
Only exhaustive testing can show a program is free from defects. However, exhaustive testing is impossible;
Testing policies define the approach to be used in selecting system tests; For example,
All functions accessed through menus should be tested;
Combinations of functions accessed through the same menu should be tested;
Where user input is required, all functions must be tested with correct and incorrect input.

4. Software Testing & Development Life Cycle
Requirements
System Testing
Integration
Testing
Architectural
Design
Unit Testing
Detail Design
Implementation

5. Unit Testing
module
to be
tested
results
software
engineer
test cases

6. Unit Testing module to be tested interface local data structures
boundary conditions
independent paths
error handling paths
test cases

7. Unit Testing test cases RESULTS driver Module stub stub interface
local data structures
Module
boundary conditions
independent paths
error handling paths
stub
stub
test cases
RESULTS

8. Integration Testing
Involves building a system from its components and testing it for problems that arise from component interactions.
Top-down integration
Develop the skeleton of the system and populate it with components.
Bottom-up integration
Integrate infrastructure components then add functional components.
To simplify error localisation, systems
should be incrementally integrated.

9. Integration Testing
The Big-bang Approach.
Incremental Approach.

10. Top-Down Integration A top module is tested with stubs B F G
stubs are replaced one at
a time, "depth first" C
as new modules are integrated,
some subset of tests is re-run D E

11. Bottom-Up IntegrationF G
drivers are replaced one at a
time, "depth first" C
worker modules are grouped into
builds and integrated D E
cluster

12. Thread-Based Integration
Top modules are
tested with stubs B F G C
Worker modules are grouped into
builds and integrated D E
cluster

13. System Testing System Functional Test System Performance Test
Test entire system against the functional requirements.
System Performance Test
Test the non-functional requirements of the system. For example,
Response times, load testing etc.
System Acceptance Test
Set of tests that the software must pass before it is accepted by the client.

14. Scenario-based Testing
Trivial Example
You have been asked to write a term paper on ‘Integration Testing in Component Based System’. To do this, you need to find references from a range of library databases.
You logs on to the KFUPM library system and uses the search facility to find relevant papers from IEEE, ACM and Elsevier databases.
One paper of special interest requires authentication and you have to fill an online form to receive the paper.
If you are allowed, the paper will be downloaded and ready for collection.
An will be send to you once the paper is ready.
Scenario-based Testing

15. Student Activity
Identify the possible interactions for the system testing of library system.

16. Trivial Example - System Testing
Test the login mechanism using correct and incorrect login.
Test the search facility using queries against known source to check that the search mechanism is actually documents.
Test system presentation facility to check that information about documents is displayed properly.
Test the mechanism to request permission for downloading.
Test the response indicating that the download document is available.

17. Need for an automatic test suite execution.
Regression Testing
Change do not always effect the entire program.
Change in one part of system can effect other part.
After each change
Entire test suite of a system must be run again.
Need for an automatic test suite execution.

18. Test Activities
Boils down to selecting and executing test cases. Test case consists of……
Set of test inputs, of if the program is non-terminating, a sequence of test inputs.
Expected results when the inputs are executed; and
Execution conditions or execution environment in which the inputs are to be executed.
These steps generally remain same from
unit testing to system testing.

19. Test Case Selection Coverage criterion;
Equivalence Partitioning
Boundary-Value Analysis
Coverage-Based Testing
Control-flow
Data-flow
Expected behavior of every test input to be generated. (Test Oracles)
Testing environment.

20. Difficult to automate or to assess their quality
Test Oracles
Determines whether or not the program has passed or failed the test case.
A test oracle is
A program
A process
A body of data
In many cases - directly form the requirements.
For example, a test case assessing performance - performance threshold.
Difficult to automate or to assess their quality

21. Generally can be automated to an extend !!!!
Test Execution
Test inputs on the ‘program-under-test’
Record the actual behavior.
Generally can be automated to an extend !!!!

22. Generally can be automated to an extend !!!!
Test Evaluation
Compare the actual behavior with the expected behavior.
Generally can be automated to an extend !!!!

23. Generally can be automated to an extend !!!!
Test Reporting
Report the outcome of the testing.
Developers
Project Mangers etc.
Generally can be automated to an extend !!!!

24. Key Points A system typically undergoes a range of testing types.
Each type of testing is aimed at detecting different kinds of failures.
Testing boils down to the selection and execution of test cases.