1. COSC 4506/ITEC 3506 Software Engineering
Classic Testing (3)

2. Outline Testing in the large Module testing Integration testing
System testing
Acceptance testing
Analysis

3. Testing in the large Module (Unit) testing testing a single module
Integration testing
integration of modules and subsystems
System testing
testing the entire system
Acceptance testing
performed by the customer

4. Review: V Graph Requirements System testing Analysis High-Level
Design
Integration
testing
Low-Level
Design
Unit
testing
Here shows the development activities and the associated testing activities for checking the correctness.
Coding
Unit testing
Delivery
Acceptance testing
Maintenance
Regression testing

5. Module testing
Verify: given module has been implemented correctly (it expected external behavior)
Example
Mi USES Mh, Mi USES Mk, Mh and Mk not available
how to test Mi in isolation? Provide temporary context that simulates the real context to be provided by Mh and Mk

6. Testing a functional module

7. Module testing
stubs: a piece of code (dummy unit) that stimulates the activity of the missing component, but a highly simplified code
it answers the calling sequence and passes back output data that lets the testing process continue
It might produce its expected results by reading them from a file, returning some random values etc

8. Module testing
driver
module (containing main method) activating the module under test
a piece of code, simulates use of module being tested by others, and passes test cases to another piece of code

9. Example
Develop a “calculator” that converts between different units (feet to inches)
“stubbing out the UnitConverter class
All non-void methods return an appropriate default so the class can be compiled
public class UnitConverter {
public UnitConverter () {}
public double convert (double value, String from, String to)
{ return 0.0;}
public double getMultiplier (String from, String to)
{ return 0.0} // return the multiplier needed for a conversion }

10. Example public class Driver { public static void main (String [] args)
We need to write a preliminary driver to test the UnitConverter class
// all the methods calls are all hard coded
public class Driver {
public static void main (String [] args)
double converted, original =10.0;
String from, to;
UnitConverter calculator = new UnitConverter ();
from = “inches”; to = “feet”;
converted = calculator.convert (original, from, to);
System.out.println (original+” ” +from+ “ =“ +” “ + to);
} } // this will compile and run

11. Example Next version has a complete implementation of convert method
public class UnitConverter {
public UnitConverter () {}
public double convert (double value, String from, String to)
double result;
result = value * getMultiplier (from, to);
return result; }
public double getMultiplier (String from, String to)
return 1.0/12.0;
} // stub

12. Next version has a complete implementation of getMultiplier method
public class UnitConverter
{ ……
public double getMultiplier (String from, String to) {
double multiplier =1.0;
if (from.equals(“inches”) {
if (to.equals (“feet”)) multiplier =1.0/12.0;
else if (to.equals(yards”) multiplier =1.0/12.0/3.0; }
else if (from.equals (“feet”)) {
if (to.equals (“inches”)) multiplier =12.0;
else if (to.equals(yards”) multiplier =1.0/3.0;
else if (from.equals (“yards”)) {
if (to.equals (“inches”)) multiplier =3*12.0;
else if (to.equals(“feet”) multiplier =3.0;
return multiplier;

13. Example { public class Driver public static void main (String [] args)
We need to revise the driver to have more test cases
public class Driver {
public static void main (String [] args)
double converted, original =10.0;
String from, to;
UnitConverter calculator = new UnitConverter ();
from = “inches”; to = “feet”;
converted = calculator.convert (original, from, to);
System.out.println (original+” ” +from+ “ =“ +” “ + to);
……………………
} }

14. Example This is a complete version of UnitConverter
public class Driver {
public static void main (String [] args) {
double converted, original;
int i, j;
String from, to;
String [] units ={“inches”, “feet”, “yard”);
UnitConverter calculator = new UnitConverter ();
original =10.0;
for (i=0; i<units.length; i++)
for (j=0; j<units.length; j++) {
from = units[i];
to = units[j];
converted = calculator.convert (original, from, to);
System.out.println (original+” ” +from+ “ =“ +” “ + to); }
} }

15. Integration build strategies
All at once
Big Bang: All coding precedes all integration
Growing Strategies (Incremental Approach)
Bottom Up: Start at low-level utility modules
Top Down: Start at high-level control modules
Sandwich: Integrate control modules top down and utility modules bottom up

16. Integration testing Big-bang approach
code & test each module separately
link all 13 modules together, test entire product

17. Integration testing problems lack of fault isolation
product as a whole fails for a test case – where?
could be in any of 13 interfaces or 13 modules

18. Integration testing Problems of Big-Bang
Critical modules receive no extra testing
Integration has to wait for coding to finish
Major design flaws are discovered very late
No flexibility in scheduling

19. Integration testing Incremental approach
modules are progressively integrated and tested
Advantages
Easy to discover errors at early stages
easy to localize errors incrementally
integration of n modules, then n+1
reduce the need for stubs or drivers
can proceed both top-down and bottom-up according to the USES relation
Wrt  w.r.t.

20. Integration testing How about the stubs and drivers? Top-down ?
if module mAbove calls module mBelow, mAbove implemented & integrated before mBelow
one ordering
a, b, c, d, e, f, g, h, i, j, k, l, & m
alternative in parallel
[a] b, e, h
[a] c, d, f, i
[a d] g, j, k, l, m
How about the stubs and drivers?

21. Top Down Integration Implement highest level component
Create stubs for called component
Test the component with the stubs
Implement and test stubs one by one
use stubs for any called components
Repeat until all stubs are implemented

22. Top Down Integration Disadvantages
important low-level utilities are not tested as well (reusable)
simple high-level drivers may not need the extra testing
Advantages
no need to write drivers
high-level drivers are well tested
unimportant low-level utilities are not as well tested
Decision modules test first, and more time

23. Integration testing How about the stubs and drivers? Bottom-up ?
if module m1 calls m2, then m2 is implemented & integrated 1st
one ordering
l, m, h, i, j, k, e, f, g, b, c, d, a
alternative in parallel
h, e, b
i, f, c, d
l, m, j, k, g [d]
a [b, c, d]
How about the stubs and drivers?

24. Bottom Up Integration Implement a module Implement a test driver
often a quick prototype
Execute tests
Replace test driver with its implementation
Implement a test driver for new module
Repeat until all modules are integrated

25. Bottom Up Integration Advantages no need to write stubs
low level utilities are well tested (reusable)
high-level (often simple) drivers are not as well tested
Disadvantages
not all low level utilities are important
emphasis on low level functionality
major design flaws are discovered late

26. Integration testing Sandwich ?
logic modules implemented & integrated top-down
operational modules implemented & integrated bottom-up
finally, interfaces between the two groups tested

27. Integration testing Sandwich approach Advantages
major design flaws are discovered earlier
low level utilities are well tested (reusable)
Fault isolation at all times

28. Testing in the large: System Testing
A series of tests to verify that all system elements have been properly integrated.
Purpose of integration testing: to ensure that the code implemented the design properly
Purpose of System Testing: to ensure that the system does what the customer wants it to be

29. Testing in the large: System Testing
Function Testing
example
Performance Testing:
Some typical performance testings
Recovery Testing:
addresses response to 1) presence of faults 2) loss of data, devices, service
loss system resources and see if it recovers properly

30. Testing in the large: System Testing
Performance Testing:
Overload testing
evaluate when goes beyond limit
quality testing
evaluate system’s reliability, maint. Availability
robustness testing
test under unexpected cond. Like erroneous user commands, power failure
Regression testing
When a change is made to a large application, we need to validate the change did not disturb the existing functionality

31. Testing in the large: System Testing
Installation testing
to allow users to exercise system functions and document additional problems that result from being the actual site
Acceptance testing
tested by the customer according to their understanding of requirements
sometimes alpha test by users within org. before beta test by customer (OS)

32. Alpha & Beta test software developer site customer site software
customer tests
Alpha test
Beta test
software
developer
reviews
customer
tests
developer site
customer site

33. Definitions Verification determining if phase completed correctly
“are we building the product right?”
“Testing” (Execution-based testing)
running test cases against executing code
Analysis (Synthesis, Nonexecution-based testing)
reviewing code or documents carefully
why should we also do this for code?

34. Synthesis: Non-Execution-based Testing
Underlying principles
cannot review own work – why?
team of reviewers – why?
Two types
informal synthesis
walkthroughs
inspections
key difference?
walkthroughs have fewer steps & are less formal
inspections record detailed data & use it in later phases & projects
Formal synthesis
proof

35. Walkthroughs Team of 4-6 members, chaired by SQA (why?)
spec writer & manager, client, designers, SQA
more experienced & senior better – why?
Distribute info in advance, each reviewer creates
list of confusing items
list of items in error
Process
detect suspected faults, don’t correct (why?)
document-driven & interactive, not participant-driven
verbalization leads to fault finding spontaneously by presenter

36. Software inspections
Involve people examining the source representation with the aim of discovering anomalies and defects
Do not require execution of a system so may be used before implementation
May be applied to any representation of the system (requirements, design, test data, etc.)
Very effective technique for discovering errors

37. Inspection success
Many different defects may be discovered in a single inspection. In testing, one defect ,may mask another so several executions are required
The reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly arise

38. Inspections and testing
Inspections and testing are complementary and not opposing verification techniques
Inspections can check conformance with a specification but not conformance with the customer’s real requirements
Inspections cannot check non-functional characteristics such as performance, usability, etc.