Lecture 8: Testing, Verification and Validation Dr Valentina Plekhanova University of Sunderland, UK

Software Testing Fundamentals: Some Testing Terminology Faults - a mistake in the code that causes the software to not behave as expected (causes). Failures - the act of a product not behaving as expected - the manifestation of a fault (symptoms). Lecture 8 Valentina Plekhanova

Some Testing Terminology Validation - establishing the fitness of a software product for its use - "are we building the right product?" Verification - establishing the correspondence between the software and its specification - "are we building the product right?" Lecture 8 Valentina Plekhanova

Some Testing Terminology Test case - the collection of inputs, predicted results and execution conditions for a single test. Pass/fail criteria - decision rules used to determine whether a product passes or fails a given test. Test suite - a collection of test cases necessary to "adequately" test a product. Lecture 8 Valentina Plekhanova

Some Testing Terminology Test plan - a document describing the scope, approach, resources and schedule of intended testing activity - identifies features to be tested, the testing tasks, who will do each task, and any risks requiring contingency planning. Lecture 8 Valentina Plekhanova

Some Testing Terminology Bug report - a method of transmitting the occurrence of a discrepancy between actual and expected output to someone who cares for "follow-up" - also known as discrepancy report, defect report, problem report, etc. Lecture 8 Valentina Plekhanova

What is Software Testing? Process of executing a computer program and comparing the actual behaviour with the expected behaviour. The comparison is intended to detect deviations (if any) between the actual behaviour and the expected behaviour. Testing serves as a barrier to allowing low quality products from reaching the customer. Lecture 8 Valentina Plekhanova

Testing Objectives Testing is a process of executing a program with the intent of finding an error. A benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications. Lecture 8 Valentina Plekhanova

Testing Objectives The data collected through testing can also provide an indication of the software's reliability and quality. But, testing cannot show the absence of defect -- it can only show that software defects are present. Functional Requirements and Non-functional Requirements must be tested. Lecture 8 Valentina Plekhanova

Software Testing is Not Debugging Testing establishes the existence of defects Debugging is concerned with locating and correcting these defects. Lecture 8 Valentina Plekhanova

software testing discovers unexpected behaviour; defect classification identifies the situation as a software error, a design error, a specification error, a testing error, etc. - if available, a "work-around" is identified; if a software error, then debugging is performed which tracks down the cause of the error, and attempts to fix it. Lecture 8 Valentina Plekhanova

The Testing Process Unit Testing Sub-System Testing System Testing Acceptance Testing or Alpha Testing Beta Testing Lecture 8 Valentina Plekhanova

The Testing Process: Major Stages Unit Testing: Individual components are tested to ensure that they operate correctly.  Module Testing: A module is a collection of dependent components such as an objects, procedures, and functions. A module encapsulates related components so can be tested without other system modules. Lecture 8 Valentina Plekhanova

The Testing Process Sub-System Testing: This phase involves testing collections of modules, which have been integrated into sub-system. The sub-system test process should concentrate on the detection of interface errors by rigorously exercising these interfaces. Lecture 8 Valentina Plekhanova

The Testing Process: System Testing System Testing: The testing process is concerned with finding errors, which normally result from unanticipated interactions between sub-systems and components. It is also concerned with validating that the system meets its functional and non-functional requirements. Lecture 8 Valentina Plekhanova

The Testing Process: Acceptance Testing Acceptance Testing: Acceptance testing often reveals errors and omissions in the system requirements definition [Sommerville]. Lecture 8 Valentina Plekhanova

The Testing Process Acceptance testing is sometimes called alpha testing. When a system is to be marked as a software product, a testing process called beta testing is often used. Lecture 8 Valentina Plekhanova

The Testing Process: Beta Testing Beta testing involves delivering a system to a number of potential customers who agree to use that system and to report problems to the system developers. Lecture 8 Valentina Plekhanova

The Testing Process: Regression Testing Errors in program components may be identified at the later stage of the testing process. Repairing program defects may introduce new defects so testing should be repeated after the system is modified. This is sometimes called regression testing [Sommerville]. Lecture 8 Valentina Plekhanova

White Box Testing [Sommerville] White box testing is an approach to testing where the tests are derived from knowledge of the software structure. Test cases can be derived that guarantee that all independent paths within a module have been exercised at least once; exercise all logical decisions on their true and false sides; execute all loops at their boundaries and within their operational bounds; exercise internal data structures to ensure their validity. Lecture 8 Valentina Plekhanova

Black Box Testing Black box testing attempts to derive sets of inputs that will fully exercise all the functional requirements of a system. It is not an alternative to white box testing. Lecture 8 Valentina Plekhanova

Black Box Testing incorrect or missing functions; interface errors; errors in data structures or external database access; performance errors; initialisation and termination errors. Lecture 8 Valentina Plekhanova

White Box & Black Box Testing White box testing should be performed early in the testing process, while black box testing tends to be applied during later stages. Lecture 8 Valentina Plekhanova

Tests are designed to answer the following questions: How is the function's validity tested? What classes of input will make good test cases? Is the system particularly sensitive to certain input values? How are the boundaries of a data class isolated? What data rates and data volume can the system tolerate? What effect will specific combinations of data have on system operation? Lecture 8 Valentina Plekhanova