Chapter 13 Integration Testing Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Levels of Testing (in the famous V-Model) Requirements Specification System Testing Preliminary Design Integration Testing Detailed Design Unit Testing Coding Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

The Mars Climate Orbiter Mission mission failed in September 1999 completed successful flight: 416,000,000 miles (665.600.600 km) 41 weeks flight duration lost at beginning of Mars orbit An integration fault: Lockheed Martin used English units for acceleration calculations (pounds), and Jet Propulsion Laboratory used metric units (newtons). NASA announced a US$ 50,000 project to discover how this happened. (We know!) Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Testing Level Assumptions and Objectives Unit assumptions All other units are correct Compiles correctly Integration assumptions Unit testing complete Unit goals Correct unit function Coverage metrics satisfied Integration goals Interfaces correct Correct function across units Fault isolation support System goals Correct system functions Non-functional requirements tested Customer satisfaction. System assumptions Integration testing complete Tests occur at port boundary Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Goals/Purpose of Integration Testing Presumes previously tested units Not system testing Tests functionality "between" unit and system levels Basis for test case identification? Emphasis shifts from “how to test” to “what to test” (Model-Based Testing) Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Approaches to Integration Testing (“source” of test cases) Functional Decomposition (most commonly described in the literature) Top-down Bottom-up Sandwich “Big bang” Call graph Pairwise integration Neighborhood integration Paths MM-Paths Atomic System Functions Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Basis of Integration Testing Strategies Functional Decomposition applies best to procedural code Call Graph applies to both procedural and object- oriented code MM-Paths Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Continuing Example (pseudo-code skeleton) Main NextDate /*NextDate computes the date of the next day after a given date*/ Function monthEnd(month, year) /* returns number of days in a given month */ Function isLeap(year) /* determines if a year is a leap year */ End (Function isLeap) /* calls isLeap */ End (Function monthEnd) Procedure GetDate(aDate) /* gets an input date */ Function ValidDate(aDate) /* calls MonthEnd *? End (Function ValidDate) /* calls ValidDate */ End (Procedure GetDate) /* checks if a date is valid */ Procedure IncrementDate(aDate) /*increments a date */ /* calls ValidDate */ End (IncrementDate) Procedure PrintDate(aDate) /* prints a date */ End (PrintDate) /*Main program begins here */ /* calls GetDate, IncrementDate, and PrintDate */ End Main Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Integration Based on Functional Decomposition Start with functional decomposition (usually derived from code, sometimes designed first) Choose integration order (top-down, bottom up, sandwich) Create stubs or drivers Create test cases Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

NextDate Functional Decomposition Main monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Step 1-Test main program logic with all stubs monthEndStub GetDateStub incrementDateStub PrintDateStub Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Steps 2 - n: Test main program logic, replace one stub at a time monthEnd GetDateStub incrementDateStub PrintDateStub Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

(assures fault isolation to the unit level) Replacing one stub at a time... (assures fault isolation to the unit level) Main monthEnd GetDate incrementDateStub PrintDateStub Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Main monthEnd GetDate incrementDate PrintDateStub Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Last Stub Replacement for Main monthEnd GetDate incrementDate PrintDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Step n + 1 - Move down one level, and repeat the stub introduction Main monthEnd GetDate incrementDate PrintDate isLeapStub Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

But, there is a problem... The Main program does not invoke MonthEnd MonthEnd is in the Functional Decomposition tree because of scoping rules of most compilers. (this is more of a problem in larger systems) Same problem occurs in bottom-up integration Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

(Start with leaf nodes, and test with drivers) Bottom-up Integration (Start with leaf nodes, and test with drivers) Main monthEnd GetDateDriver incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

At Each Step, Replace a Driver With Actual Code Main monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

When All Drivers at a Level Have Been Replaced, Move up one Level and Replace One Unit at a Time MainDriver monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Note the Fault Isolation MainDriver monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Next Step in Integration MainDriver monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

This Level Tests the MainDriver Logic monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Last Step in Bottom-up Integration Main monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

(no stubs, no drivers, but less fault isolation) Sandwich Integration (no stubs, no drivers, but less fault isolation) Main monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

An Impossible Sandwich (Why?) Main monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

(No stubs, no drivers, no fault isolation) Big Bang Integration (No stubs, no drivers, no fault isolation) Main monthEnd GetDate incrementDate PrintDate isLeap ValidDate Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Call Graph Based Integration Derive Call Graph from source code Nodes are units There is an edge from unit A to unit B if A calls B Choose approach (pairwise or neighborhood) Develop test cases Need special tools (or code) to check “before and after” interface interaction Resolves question of impossible interfaces (e.g., monthEnd) Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

NextDate Call Graph Main GetDate incrementDate PrintDate ValidDate monthEnd isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Pairwise Integration - test each edge in Call Graph Main pair 2 incrementDate pair 1 pair 3 GetDate PrintDate pair 4 ValidDate pair 7 pair 5 monthEnd pair 6 isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Redundancy in Pairwise Integration Each node may be involved in several tests (degree of node) (and the non-existent interface problem is fixed) Node Number of pairs Main 3 GetDate 2 incrementDate PrintDate 1 ValidDate MonthEnd IsLeap Total tests 14 Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Neighborhood Integration A neighborhood of a node is the set of all other nodes a given distance (usually 1) away. GetDate Main ValidDate incrementDate PrintDate monthEnd isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Neighborhood of monthEnd GetDate Main ValidDate incrementDate PrintDate monthEnd isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Neighborhood of ValidDate GetDate Main ValidDate incrementDate PrintDate monthEnd isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Redundancy in Neighborhood Integration There is one neighborhood (of radius 1) for each node. 7 nodes yield 7 integration sessions. In general, neighborhood integration reduces the number of test sessions required by pairwise integration. This reduction comes at the expense of increased fault isolation effort. Some special code may still be needed. Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

NextDate Class Descriptions Attributes Methods testIt calendarItem (abstract class) value getValue setValue IncrementValue Date Day Month Year getDate printDate incrementDate Day Month Year endOfMonth isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

UML Collaboration Diagram for NextDate 1: create 2: increment 3: getYear Year 1: create 2: increment 3:setMonth 1:isLeap 1: create 2: increment 3: printDate 4: getMonth testIt Date Month 1: create 2: increment 3: setDay 4: getDay Day Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Integration Based on UML Collaboration Diagram A UML Collaboration Diagram is essentially an undirected graph. Not a problem since the direction of edges in Call Graph integration is not used. Therefore UML Collaboration Diagram based integration testing reduces to Call Graph based testing. The tendency in the object-oriented community is to use pairwise integration. (Neighborhood integration is a better choice!) Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Integration Based on MM-Paths an MM-Path is an alternating sequence of unit execution paths and unit invocations (or messages). In data-driven programs, MM-Paths begin in the Main Program. In Event-driven programs, MM-Paths begin in the event sensing units. MM-Paths “end” when no further messages are sent. (message quiescence) Techniques apply equally well to procedural and object- oriented code. MM-Paths are always feasible paths. Increased effort to identify MM-Paths. Increased identification effort offset by elimination of stub/driver code. AND, offset by superb fault isolation capability. Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Extensions to Definitions A source node in a program is a statement fragment at which program execution begins or resumes. A sink node in a program is a statement fragment at which program execution terminates. A module execution path is a sequence of statements that begins with a source node and ends with a sink node, with no intervening sink nodes. A message is a programming language mechanism by which one unit transfers control to another unit. Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Definitions for Integration Testing An MM-Path is an interleaved sequence of module execution paths and messages. Given a set of units, their MM-Path graph is the directed graph in which nodes are module execution paths and edges correspond to messages and returns from one unit to another. An atomic system function (ASF) is an action that is observable at the system level in terms of port input and output events. Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

An MM-Path A B C 1 1 1 2 2 2 3 3 4 4 3 5 5 4 6 Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Question: What data determines MM-Paths 1 and 2? Two MM-Paths Question: What data determines MM-Paths 1 and 2? Main incrementDate PrintDate GetDate ValidDate MM-Path 1 monthEnd MM-Path 2 isLeap Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Program Graphs of MonthEnd and IsLeap 2 3 isLeap monthEnd 15 4 16 18 20 5 12 7 17 19 21 6 8 9 22 23 10 24 11 13 25 14 26 Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

An MM-Path Test Case as a Series of Assertions Unit Invocation Parameter/value Unit Execution Paths Return Parameter/value main today. Month =11 today.Day = 11 today.Year =2004 <87> IncrementDate aDate. Month=11 aDate.Day = 11 aDate.Year=2004 <69, 70> monthEnd aDate.Month 11 aDate.Year 2004 <2, 15, 18, 19, 25, 26 > monthEnd= 30 < 71, 78, 79> aDate.Month=11 aDate.Day = 12 tomorrow. Month =11 tomorrow.Day = 12 tomorrow.Year =2004 <87 > Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Comparison of Integration Testing Strategies Strategy Basis Ability to test Interfaces Ability to test co-functionality fault isolation resolution Functional acceptable but can limited to pairs of good, to faulty unit Decomposition be deceptive units Call Graph acceptable MM-Path excellent complete excellent, to faulty unit execution path Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

Case Study: NextDate Revisited as a Main Program with Functions Pseudo-code showing messages Program graphs Functional decomposition tree Call graph MM-Paths Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

(Integration Version) NextDate Main Program (Integration Version) 1. Main integrationNextDate Dim today As Date Dim tomorrow As Date 2. GetDate(today) 'msg1 3. PrintDate(today) 'msg2 4. tomorrow = IncrementDate(today) 'msg3 5. PrintDate(tomorrow) 'msg4 6. End Main Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

isLeap Function isLeap(year) If (year divisible by 4) Then 10 11 12 13 14 15 16 17 18 19 20 Boolean If (year is NOT divisible by 100) Then isLeap = True Else If (year is divisible by 400) Then isLeap = True Else isLeap = False EndIf EndIf Else isLeap = False EndIf End (Function isLeap) Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing

lastDayOfMonth 21 Function lastDayOfMonth(month, year) Integer 22 Case month Of 23 Case 1: 1, 3, 5, 7, 8, 10, 12 24 lastDayOfMonth = 31 25 Case 2: 4, 6, 9, 11 26 lastDayOfMonth = 30 27 Case 3: 2 28 If (isLeap(year)) 'msg5 29 Then lastDayOfMonth = 29 30 Else lastDayOfMonth = 28 31 EndIf 32 EndCase 33 End (Function lastDayOfMonth) Software Testing: A Craftsman’s Approach, 3rd Edition Integration Testing