1. Prepared by Soha Makady
Software Testing
Prepared by
Soha Makady
(c) Copyright Soha Makady 2016

2. Software Quality Landscape
Change and Configuration Management
Change and Configuration Management
Document Control
Software Quality Landscape
Reviews
Collaborative Development / Reviews
Requirements
V & V
Guidelines, Standards and Templates
Design Review
Bug Tracking
Surveys
Contract
Review
QA Planning and Budgeting
Testing
Testing
Metrics
Secure Coding
Secure Coding
Debugging
Refactoring
Planning
Deployment &
Maintenance
Design
Requirements
Testing
Development
M a n a g e m e n t

3. Assessment 40% Coursework 10% Midterm (Mid Nov.) = 12% marks 50% Final
A1= 5, A2 = 7, A3 ~= 8
A4 = 5 , (A5, A6) -> will come soon
10% Midterm (Mid Nov.) = 12% marks
50% Final

4. Assessment Midterm 25 Nov. Open Book – 2 hrs
Assignments 5 & 6 will be posted soon
Tutorials made specially for this course are
given with a non disclosure agreement.

5. Readings so far … Code Complete, Chapter 20
Software QA from Theory to Practice, Chapters 1, 2, 5, 6, 7, 18
Laurie Williams, Reviews and Pair Programming, 2004
CMP by USDA
Article on how much of the world runs on bad SW.

6. Readings
Chapter 1 from: Ron Patton Software Testing (2nd edition).
Chapter 18 from: Ron Patton Software Testing (2nd edition).
Chapter 18: Reporting What You Find
Chapters 1 & 2 from The Art of Software Testing, Second Edition, Glenford J. Myers

7. Readings for this lecture
Chapter 4 from: Ilene Burnstein Practical Software Testing (2nd edition).
Chapter 4: Strategies and Methods for Test Case Design I

8. (c) Copyright Soha Makady 2016
Outline
Levels of Testing
Black-box vs. White-box testing
Black-box Techniques
Equivalence partitioning
Boundary value analysis
Decision tables
Cause-effect graphs
Software bug: A formal definition (so..we ran our tests. What can we consider a bug?)  ch1. Ron Patton  start with the caculator, and explain the different behaviours, and whether they’d be considered bugs or not
(c) Copyright Soha Makady 2016

9. Levels of Testing
Figure from chapter 6. Practical Software testing.

10. © Copyright Joint MSc in Software Engineering – FCI-CU
Testing Levels – Unit testing
During unit testing Individual components are tested independently to ensure their quality.
Unit tests are carried out by the developer of the given program unit.
Techniques used in unit testing are white-box and black-box
The focus is to uncover errors in design and implementation, including
- data structure in a component
- program logic/structure in a component
- component interface
- functions and operations of a component
© Copyright Joint MSc in Software Engineering – FCI-CU

11. © Copyright Joint MSc in Software Engineering – FCI-CU
Testing Levels – Unit testing
The focus is to uncover errors in design and implementation, including
- data structure in a component
- program logic/structure in a component
- component interface
- functions and operations of a component
The focus is to uncover errors in design and implementation, including
- data structure in a component
- program logic/structure in a component
- component interface
- functions and operations of a component
© Copyright Joint MSc in Software Engineering – FCI-CU

12. © Copyright Joint MSc in Software Engineering – FCI-CU
Testing Levels – Integration testing
A group of components are composed and tested together to ensure the quality of their integration.
Component #1
Operations and
Functions with I/O
input
interface
operation
Component #2
output
The objective is to detect errors in:
Design and construction of software architecture
Integrated functions/operations at sub-system level
Interfaces and interactions between them
Resource integration and/or environment integration
© Copyright Joint MSc in Software Engineering – FCI-CU

13. Testing Levels – Integration testing
The objective is to detect errors in:
Design and construction of software architecture
Integrated functions/operations at sub-system level
Interfaces and interactions between them
Resource integration and/or environment integration
The objective is to detect errors in:
Design and construction of software architecture
Integrated functions/operations at sub-system level
Interfaces and interactions between them
Resource integration and/or environment integration

14. © Copyright Joint MSc in Software Engineering – FCI-CU
Testing Levels – System testing
The system software is tested as a whole.
Verifies that all system functions and performance are achieved in the target environment.
The focus areas are:
- System functions and performance
- System reliability and recoverability (recovery test)
- System installation (installation test)
- System behavior in the special conditions (stress and load test)
- Hardware and software integration and collaboration
- Integration of external software and the system
© Copyright Joint MSc in Software Engineering – FCI-CU

15. © Copyright Joint MSc in Software Engineering – FCI-CU
Testing Levels – System testing
The focus areas are:
System functions and performance
System reliability and recoverability (recovery test)
System installation (installation test)
System behavior in the special conditions (stress and load test)
Hardware and software integration and collaboration
Integration of external software and the system
The focus areas are:
- System functions and performance
- System reliability and recoverability (recovery test)
- System installation (installation test)
- System behavior in the special conditions (stress and load test)
- Hardware and software integration and collaboration
- Integration of external software and the system
© Copyright Joint MSc in Software Engineering – FCI-CU

16. © Copyright Joint MSc in Software Engineering – FCI-CU
Testing Levels – Acceptance testing
Acceptance test is a customer’s way to verify that what was wanted is what is built
It is performed to test that the overall system is functioning as required to decide whether to accept the system based on the test result.
System
(Operations &
Functions
& Behavior)
User
interface
User
External interfaces
Section 6.1  Practical software testing
© Copyright Joint MSc in Software Engineering – FCI-CU

17. (c) Copyright Soha Makady 2016
The Waterfall Model
At the end of each step, the team holds a review to determine if they can move to the next step.
Advantages: Projects specification is highly studied, so with a disciplined development team , there would not be many ambiguities in the specs.
Disadvantages:
Disadvantages: Testing is done once at the end. IF an error creeps in early, it would be very expensive to fix, or would escape to the product release.
There is a large emphasis on the project specification
Advantages?
Disadvantages?
(c) Copyright Soha Makady 2016

18. (c) Copyright Soha Makady 2016
The V-Model
The V-model is a model that illustrate how the testing activities can be integrated into each phase of the SDLC.
Using the V-model, we will look at test-related activities in each lifecycle phase, and the deliverables that result.
(c) Copyright Soha Makady 2016

19. (c) Copyright Soha Makady 2016
The V-Model
Figure 8.5. Ilene’s book.
(c) Copyright Soha Makady 2016

20. (c) Copyright Soha Makady 2016
The V-Model
Requirements phase:
Testers use the requirements to create system tests, acceptance tests, based on functional requirements, quality requirements, and system’s specifications.
Black-box methods are used in test design.
Deliverables include:
An initial master test plan
An initial version of system/acceptance tests
Figure 8.5. Ilene’s book.
(c) Copyright Soha Makady 2016

21. (c) Copyright Soha Makady 2016
The V-Model
Design phase:
When design is complete, the structure of the system components is available.
Testers create integration tests, based on the design document.
The test plan can be reviewed to include new info from the added tests
Deliverables include:
An reviewed master test plan
An initial version of integration tests.
Figure 8.5. Ilene’s book.
(c) Copyright Soha Makady 2016

22. (c) Copyright Soha Makady 2016
The V-Model
Coding phase:
Testers create unit tests, based on the design document.
The master test plan, and other test plans can be reviewed to include new info from the added tests
Deliverables include:
An reviewed master test plan
Unit test plan
Code for the unit tests.
Figure 8.5. Ilene’s book.
(c) Copyright Soha Makady 2016

23. (c) Copyright Soha Makady 2016
The V-Model
Tests execution
According to the V-model, many tests have already been developed by the time the coding is complete.
The test team can now execute the planned tests at unit, integration, and system levels.
Deliverables include:
The tested code
Test logs
Test measurement data
Test summary report.
(c) Copyright Soha Makady 2016

24. (c) Copyright Soha Makady 2016

25. (c) Copyright Soha Makady 2016
Smart Tester
Software components have defects, no matter how well our defect prevention activities are implemented.
Developers cannot prevent/eliminate all defects during development.
Therefore, software must be tested before it is delivered to users.
It is the responsibility of the testers to design tests that (i) reveal defects, and (ii) can be used to evaluate software performance, usabilty, and reliability.
(c) Copyright Soha Makady 2016

26. (c) Copyright Soha Makady 2016
Smart Tester
To achieve these goals, testers must select a finite number of test cases, often from a very large execution domain.
Unfortunately, testing is usually performed under budget and time constraints.
The smart tester must plan for testing, select the test cases, and monitor the process to insure that the resources and time allocated for the job are utilized effectively.
Testers need proper education and training.
(c) Copyright Soha Makady 2016

27. Black- vs. White (glass)-Box Testing
System
Specification
Implementation
Missing functionality: Cannot be revealed by white-box techniques
Unexpected functionality: Cannot be revealed by black-box techniques
Copyright Vahid Garousi 2015
(c) Copyright Soha Makady 2016

28. Black- vs. White (glass)-Box Testing
Black-box testing
The tester knows what the software is supposed to do.
The tester does NOT know how the software operates
Also called functional testing or behavioral testing  You test how it behaves
White-box testing
The tester has access to the program’s code.
Based on the code, the tester may find that some numbers are more/less likely to fail.
Also called structural testing.
“Implementation-based”
(c) Copyright Soha Makady 2016

29. (c) Copyright Soha Makady 2016

30. (c) Copyright Soha Makady 2016
Random Testing
Select random values from the input domain
Examples ?
If input domain is between 1 and 100
Assume we picked 23, 65 and 111?
Are these the best for revealing bugs on …..
How about –ve, float, char, 0, 100?
May save time and effort in selecting test cases but is less efficient in revealing bugs.
(c) Copyright Soha Makady 2016

31. (c) Copyright Soha Makady 2016
Black-Box Testing
Equivalence class partitioning
Pair-wise and n-way testing
Boundary value analysis
Category partition
Decision tables
Cause effect graphs
Error guessing
You check the tires, check the paint, look under it  Static
You start it up, listen to the engine, drive down the road  Dynamic.
(c) Copyright Soha Makady 2016

32. Equivalence Partitioning
Partition the input domain of the software under-test.
The finite number of partitions or equivalence classes.
Tester selects a given member of an equivalence class as a representative of that class.
It is assumed that all members of an equivalence class are processed in an equivalent way by the target software.
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

33. Equivalence Partitioning
If one test case in a particular equivalence class reveals a defect, all the other test cases based on that class would be expected to reveal the same defect.
If a test case in a given equivalence class did not detect a particular type of defect, then no other test case based on that class would detect the defect.
Input domain (as in specs) is partitioned.
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

34. Equivalence Partitioning
Reduces the huge (infinite) set of possible test cases, into a much smaller, still equally effective, set.
Recall our Calculator program.
What problem did we face when trying to test the addition functionality.
Let’s say we tested 1 + 1, 1+ 2, 1+3, 1+4.
Do we need to test 1+6?
How about 1+15?
How about (the maximum number that you can type in)?
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

35. Equivalence Partitioning
Reduces the huge (infinite) set of possible test cases, into a much smaller, still equally effective, set.
Recall our Calculator program.
What problem did we face when trying to test the addition functionality.
Let’s say we tested 1 + 1, 1+ 2, 1+3, 1+4.
Do we need to test 1+6?
How about 1+15?
How about (the maximum number that you can type in)?
Belongs to a different equivalence partition
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

36. Equivalence Partitioning
Equivalence partitions group inputs that receive the same treatment from the system.
Back to our Calculator program.
1+ 4  represents a normal addition operation involving two small numbers.
 the system would need to handle an overflow condition.
Hence, those two inputs belong to different equivalence partitions.
Let’s see other examples.
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

37. Calculator Equivalence Partitioning
The Edit menu shows Copy and Paste commands.
There are 5 different ways to perform the copy command.
Click Copy menu item
Type C
Type c
Press Ctrl + c
Press Ctrl + Shift + c
We have 3 equivalence classes  Clicking menu items, typing a single character, typing a combination of character.
You want to check that the functionality is NOT block. You do have workarounds.
What equivalence partitions do we have?
(c) Copyright Soha Makady 2016

38. File-Save As Equivalence Partitioning
A Windows filename can contain any character except \ / : * ? “ < > and |
A windows filename can have from 1 to 255 characters
(c) Copyright Soha Makady 2016

39. File-Save As Equivalence Partitioning
What equivalence partitions do we have?
How many inputs do I have?  One: file name
How many attributes do I have? Characters, length.
For each attribute, identify its equivalence classes
Valid characters
Invalid characters
Valid length names
Names that are too short
Names that are too long
A Windows filename can contain any character except \ / : * ? “ < > and |
A windows filename can have from 1 to 255 characters
(c) Copyright Soha Makady 2016

40. Equivalence Partitioning
1. It eliminates the need for exhaustive testing, which is not feasible. 2. It guides a tester in selecting a subset of test inputs with a high probability of detecting a defect. 3. It allows a tester to cover a larger domain of inputs/outputs with a smaller subset selected from an equivalence class.
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

41. Equivalence Partitioning
The tester and the analyst interact during the analysis phase to develop
(i) a set of testable requirements, and
(ii) a correct and complete input/output
specification.
From these the tester develops,
(i) a high-level test plan, and
(ii) a preliminary set of black box test cases for
the system.
If I tested 1+ 1, 1+2, 1+3, 1+4
What is so special about  It demands a DIFFERENT treatment.
(c) Copyright Soha Makady 2016

42. (c) Copyright Soha Makady 2016
References
Ron Patton Software Testing (2nd Edition). Sams, Indianapolis, IN, USA.
Glenford J. Myers, Corey Sandler, and Tom Badgett The Art of Software Testing (3rd ed.). Wiley Publishing.
Ilene Burnstein Practical Software Testing: A Process-Oriented Approach (1st ed.). Springer Publishing Company, Incorporated.
(c) Copyright Soha Makady 2016