1. ISQB Software Testing
Section Meeting
10 Dec 2012

2. Why testing is necessary
Increasing complexity of software
Cost of software failures
"The internal [...] software exception was caused during execution of a data
conversion from 64-bit floating point to 16-bit signed integer value. The floating point number which was converted had a value greater than what could be represented by a 16-bit signed integer. [...].“
ARIANE 5, Flight 501 Failure, Report by the Inquiry Board

3. Test and Quality
Quality: the degree to which a component, system or process meets specified requirements and/or user/customer needs and expectations.
Software quality: the totality of functionality and features of a software product that bear on its ability to satisfy stated or implied needs.

4. ISO 9126 Software Quality Software Quality Characteristics:
Functionality:
Specified or expected functions (behavior): suitability, accuracy, interoperability, compliance, security
Reliability:
Maintenance of a certain level of performance: maturity, fault tolerance, recoverability
Usability:
Effort needed for use, user-friendliness: understandability, learnability, operability
Efficiency:
Ratio of level of performance and used resources: time behaviour, resource behaviour
Maintainability:
Effort to perform defined changes: analysability, changeability, stability, testability
Portability:
Ability to transfer to a different environment (SW, HW, etc.): adaptability, installability, conformance, replaceability

5. Testing principles
Testing can show the presence of defects but not their absence
Exhaustive testing is impossible
Early testing
Defect clustering
Pesticide paradox
Testing is context dependent
Absence-of-errors fallacy

6. Fundamental Test Process
Test Level Acceptance Testing
Test Level System Testing
Test Level Integration Testing
Test Level Component Testing
Planning
Analysis &
Design
Implementation & Execution
Exiting criteria
Testing Closure
Cont ro l
Test cycle 1...n

7. Test Cases Specification

8. Logical and Concrete Test Cases
Example: Logical test case
Example: Concrete test case

9. Test Oracle
After each executed test case it has to be decided whether there is a failure or not:
the observed result/outcome is compared to the expected result/outcome
therefore the expected result/outcome has to be determined in advance for each test case
this expected outcome has to be gathered from an appropriate source: the test oracle

10. Testing in Software Development Models

11. Overview of Test Types

12. Static Testing Review Static Analysis Inspection, walkthrough ...
Control flow analysis
Data flow analysis
Metrics
Coding standards

13. Control Flow Analysis Example

14. CERN recommended static analysis tools

15. Test design techniques

16. Equivalent Class Partitioning
Set of Input values
Equivalent class
Combinations of equivalence classes for building test cases:
All possible combinations of all valid equivalence classes build valid test cases. Even with a few parameters this can lead to a high number of test cases. For this reason, often only frequent combinations are considered
An invalid input equivalence class should only be combined with valid input equivalence classes

17. Boundary Value Analysis: Example
A function calculates z, the sum of two integer numbers x and y
For the input the following ranges are defined:
x: [ ]
y: [ ]
Parameter
Valid classes
Invalid class x
0,50
-1,51 y
20,100
19,101

18. Structures of Decision Tables

19. Statement Testing Statement coverage Decision coverage
Every instruction (statement) is executed at least once
Decision coverage
Every branch is executed at least once

20. Path Coverage
Path coverage
Each path is executed at least once

21. Error guessing Testing
Examples of potential errors out practice
"pathological" situations:
Empty input fields, blanks in strings
Repeated access (many times) to the same data
No more space available, limits
Negative numbers, etc.
Situations that are explicitly forbidden
Situations that "never arise"
Special characters (a, s, €, etc.)
Abuse and misunderstandings by the user
Change from and to daylight saving time, leap year

22. Exploratory Testing Continuing learning process
Become acquainted with the product
Learn how errors in software arise and which effect those errors have
Discover the weaknesses of the software
Learn how to test the software
Test the software
Report the problems
Create new test cases on the basis of what you just have learned

23. Testing Organization Forms

24. Balance Between Test Cost and Failure Cost

25. Early Failure Detection as a Goal

26. Testing Management Tasks
Test Management
Test Organization
Test Planning and Estimation
Risk Analysis
Test Progress Monitoring
Failure based metrics
number of failures found, per release, per class, per failure status,etc.
Test case based metrics
number of blocked tests, number of successful tests, etc.
Test object based metrics
code-coverage, dialogue-coverage, etc.
Test process based metrics
test effort, on schedule milestones, etc.
Test Control
Corrective measures have to be taken:
Requesting additional resources
Updating of the test schedule
Re-assigning priorities
Initiation of additional test cycles
Configuration Management
Incident Management
Norms and Standards

27. Thanks for attending!