1. Higher-Level Testing and Integration Testing

2. Context
Higher-level testing begins with the integration of (already unit-tested) modules to form higher-level program entities (e.g., components).
The primary objective of integration testing is to discover interface and design errors associated with interactions between elements being integrated.
Once the elements have been integrated (i.e., once they are able to function together), the higher-level element can be tested thoroughly (via component, product, or system testing).

3. Unit Testing (Reminder)
Unit testing is a method by which individual units of source code are tested to determine if they are fit for use.
A unit is the smallest testable part of an application. In procedural programming a unit may be an individual function or procedure. In object-oriented programming a unit is a method or class. Unit tests are created by programmers or occasionally by white box testers.

4. Integration Testing
Integration testing is carried out when we combine “lower-level” system elements (individual units or groups of them).
The strategy employed can significantly affect the time and effort required to yield a working, higher-level element.

5. Why do we do Integration Testing?
Unit tests only test the unit in isolation
Many failures result from faults in the interaction of units or subsystems
Often many off-the-shelf components are used that cannot be unit tested
Failures not discovered until system testing are more expensive to fix.
Failures that are not discovered in integration testing may not be discovered until after the system is deployed and can be very expensive.

6. Integration Testing (cont’d)
Note that ‘‘integration testing’’ is sometimes defined as the level of testing between unit and system.
We use a more general model of the testing process, in which integration testing is used when we combine:
Units or modules to form a component
Components to form a sub-system
Sub-systems to form a system

7. Levels of Testing A More General View A Popular View System Test
Integration Test
System Test
Subsystem Test
Integration Test
Integration Test
Unit Test
Component Test
Integration Test
Unit Test

8. Arch. Diagram Example: Compiler
tokens
AST
Lexical
Analyzer
source
code
Parser
High Level
Optimizer
AST
optimized
FRONT END
optimized
object
code
Code
Generator
Low Level
Optimizer
object
code
BACK END

9. Integration Testing Strategies
The first issue to address is the choice between instantaneous and incremental integration testing.
The former is sometimes referred to as the big bang approach.

10. Disadvantages of Big Bang
Defects present at the interfaces of components are identified at a very late stage.
It is very difficult to isolate the defects found, as it is very difficult to tell whether a defect is in component or interface.
There is a high probability of missing some critical defects which might surfaced in production.
It is very difficult to make sure that all the cases for integration testing are covered.

11. Incremental Integration Testing Strategies
Incremental integration testing results in some additional overhead, but can significantly reduce error localization and correction time.
The optimum incremental approach is inherently dependent on the individual project and the pros and cons of the various alternatives.

12. Modeling the Integration Process Graphically
driver A
A common system model
used when considering
integration testing is a
call graph. B C C D D E F G G H H I I
In a call graph, nodes
represent system
modules, and an edge
from N1 to N2 indicates
that N1 calls N2 J K L

13. Modeling the Integration Process Graphically
driver A
More generally, though,
the relationship we’re
interested in depicting
by edges is one of
“dependence” of one
module on another.
For example, M1 might
not call M2, but it might
require its services in order
to proceed – for example,
if M2 makes some data available
that M1 requires, via say a database
or other data transmission method
This is more like the “Uses” relationship. B C C D D E F G G H H I I J K L

14. Incremental Strategies
Suppose we are integrating a group of modules to form a component, the control structure of which forms the following uses graph. An incremental integration strategy imposes a specific, well-thought-out order to the way in which nodes in the graph (modules) are considered. A B C D E F G H I J K L

15. Incremental Strategies (cont’d)
In what order should the modules be integrated?
From the top (“root”) module toward the bottom?
From bottom (“leaf”) modules toward the top?
By function?
Critical or high-risk modules first?
By availability?

16. Incremental Strategies (cont’d)
How many should be combined at a time?
What scaffolding (i.e., drivers and stubs to exercise the modules and oracles to interpret or inspect test results) will be required?
Is scaffolding ever required outside the context of integration testing?

17. Stubs and Drivers Driver: A component, that calls the Tested Unit
Controls the test cases
Stub:
A component the Tested Unit depends on
Partial implementation
Returns fake values.
Driver
Tested
Unit
Stub

18. Stubs int main (int argc, char **argv) { if (argc<4) usage();
double x = f(argv[1],argv[2]); … }
void usage() {
return;
double f(char *arg1, char *arg2) {
return 3.0

19. Drive execution of sqrt function with inputs from 1 to 10
Drivers
int main( int argc, char **argv) {
int i;
for (i=0; i<10;i++) {
printf(“SQRT of %i is %i”, i, sqrt(i)); }
Drive execution of sqrt function with inputs from 1 to 10

20. Top-Down Strategy
Start with the ‘‘root’’ and one or more used modules.
Test this group using stubs to take the place of missing used modules, and one driver (if necessary) to call the root module.
Add one or more other called modules, replacing and providing new stubs as necessary.
Continue the process until all elements have been integrated and tested.

21. Top-Down Strategy (cont’d)B C C D D E F G G H H I I J K L

22. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B
stub
stub
stub
stub

23. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C
stub
stub
stub
stub
stub

24. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D
stub
stub
stub
stub
stub
stub

25. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E
stub
stub
stub
stub
stub

26. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F
stub
stub
stub
stub
stub

27. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F G G
stub
stub
stub
stub

28. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F G G H H
stub
stub
stub
stub

29. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F G G H H I I
stub
stub
stub

30. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F G G H H I I J
stub
stub

31. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F G G H H I I J K
stub

32. Top-Down Strategy (cont’d)
driver A B C D E F G H I J K L A B C C D D E F G G H H I I J K L

33. Top-Down Strategy (cont’d)
Advantages:
Allows early verification of high-level behavior.
One driver (at most) is required.
Modules can be added one at a time with each step if desired.
Supports both ‘‘breadth first’’ and ‘‘depth first’’ approaches.

34. Top-Down Strategy (cont’d)
Disadvantages:
Delays verification of low-level behavior.
Stubs are required for missing elements.
When stubs replace lower-level modules it is more difficult to formulate tests in which significant data flows “upward”

35. Bottom-Up Strategy
Start at the bottom of the hierarchy with two or more sibling leaf modules, or an only-child leaf module with its parent.
Test this group using a driver. (No stubs are required.)
Add one or more additional siblings, replacing drivers with modules only when all modules they use have been integrated.
(cont’d)

36. Bottom-Up Strategy (cont’d)
Continue the process until all elements of the subtree have been integrated and tested.
Repeat the steps above for the remaining subtrees in the hierarchy (or handle in parallel).
Incrementally combine the sub-trees and then add the root module.

37. Bottom-Up Strategy (cont’d)F G G H H I I J K L

38. Bottom-Up Strategy (cont’d)F G H I J K L
driver F J

39. Bottom-Up Strategy (cont’d)F G H I J K L
driver E F J

40. Bottom-Up Strategy (cont’d)F G H I J K L
driver B E F J

41. Bottom-Up Strategy (cont’d)F G H I J K L
driver B E F
driver J K L

42. Bottom-Up Strategy (cont’d)F G H I J K L
driver B
driver E F H H J K L

43. Bottom-Up Strategy (cont’d)F G H I J K L
driver B
driver E F H H I I J K L

44. Bottom-Up Strategy (cont’d)F G H I J K L
driver
driver B D D E F H H I I J K L

45. Bottom-Up Strategy (cont’d)F G H I J K L
driver
driver B D D
driver E F G G H H I I J K L

46. Bottom-Up Strategy (cont’d)F G H I J K L
driver
driver
driver B C C D D E F G G H H I I J K L

47. Bottom-Up Strategy (cont’d)F G H I J K L
driver
driver B C C D D E F G G H H I I J K L

48. Bottom-Up Strategy (cont’d)F G H I J K L
driver
driver B C C D D E F G G H H I I J K K L

49. Bottom-Up Strategy (cont’d)F G H I J K L
driver B C C D D E F G G H H I I J K L

50. Bottom-Up Strategy (cont’d)F G H I J K L
driver B C C D D E F G G H H I I J K L

51. Bottom-Up Strategy (cont’d)F G H I J K L
driver A B C C D D E F G G H H I I J K L

52. Bottom-Up Strategy (cont’d)
Advantages:
Allows early verification of low-level behavior.
No stubs are required.
Easier to formulate input data for some subtrees.
Easier to interpret output data for others.

53. Bottom-Up Strategy (cont’d)
Disadvantages:
Delays verification of high-level behavior.
Drivers are required for missing elements.
As subtrees are combined, a large number of elements may be integrated at one time.

54. Sandwich Integration
Combination of bottom-up and top-down integrations
System is viewed as composed of layers
Approach 1:
Top-down approach is used for the top layer
Bottom-up approach is used for the bottom layer
Allows integration to begin early in the testing phase

55. Sandwich Integration Approach 2: Start with a layer in the middle
Use drivers to and stubs to check
Work out from middle
Allows integration to begin early in the testing phase

56. Hybrid Incremental Integration Approaches
Risk Driven
Start by integrating the most critical or complex modules together with modules they call or are called by.
Schedule Driven
To the extent possible, integrate modules as they become available.

57. Hybrid Incremental Integration Approaches (cont’d)
Function or Thread Driven
Integrate the modules associated with a key function (thread); continue the process by selecting another function, etc. A B C D E F G H I J K L