Graph: Data Flow Criteria CS 4501 / 6501 Software Testing

Slides:



Advertisements
Similar presentations
Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.
Advertisements

Introduction to Software Testing Chapter 1 Paul Ammann & Jeff Offutt SUMMARY OF PARTS 1 AND 2 FROM LAST WEEK.
Paul Ammann & Jeff Offutt
Graph Coverage (2).
CITS5501 Software Quality and Testing Logic and Path Coverage From Introduction to Software Testing, 2 nd ed, Ammann & Offutt.
Graph Coverage Criteria Structural Coverage Criteria : Defined on a graph just in terms of nodes and edges Data Flow Coverage Criteria : Requires a graph.
Introduction to Software Testing Chapter 2.1, 2.2 Overview Graph Coverage Criteria Paul Ammann & Jeff Offutt
Lecture 2 White-box Testing and Structural Coverage.
COVERAGE CRITERIA FOR TESTING 1. ill-defined terms in Testing  complete testing  exhaustive testing  full coverage Are poorly defined terms because.
Tutorial 3, Q1 Answers Consider the following simple program (in no particular language) to find the smallest prime factor of a number: smallest(int p)
The Application of Graph Criteria: Source Code  It is usually defined with the control flow graph (CFG)  Node coverage is used to execute every statement.
Introduction to Software Testing Chapter 1 Introduction Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 2.3 Graph Coverage for Source Code Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Overview Graph Coverage Criteria ( Introduction to Software Testing Chapter 2.1, 2.2) Paul Ammann & Jeff Offutt.
Introduction to Software Testing Chapters 1-5 Coverage Summary Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 2.3 Graph Coverage for Source Code Paul Ammann & Jeff Offutt
Software Verification Graph Model. 2 Graph Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Use cases Specs Design Source.
Software Testing and Maintenance Lecture 2.1 Overview Graph Coverage
1 Graph Coverage (3). Reading Assignment P. Ammann and J. Offutt “Introduction to Software Testing” ◦ Section 2.2 ◦ Section
Control Flow Graphs : The if Statement 1 if (x < y) { y = 0; x = x + 1; } else { x = y; } x >= yx < y x = y y = 0 x = x + 1 if (x < y) { y = 0;
1 Test Coverage Coverage can be based on: –source code –object code –model –control flow graph –(extended) finite state machines –data flow graph –requirements.
Paul Ammann & Jeff Offutt
Data Flow Coverage Criteria
Software Testing and Maintenance Lecture 3 Graph Coverage for Source Code Paul Ammann & Jeff Offutt Instructor: Hossein Momeni Mazandaran.
Data Flow Testing. Introduction to Software Testing (Ch 2) © Ammann & Offutt 2 Definition of a Graph A set N of nodes, N is not empty A set N 0 of initial.
Chapter 2 : Graph Coverage (part 2)
Introduction to Software Testing (2nd edition) Chapter 5 Criteria-Based Test Design Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
ISP Coverage Criteria CS 4501 / 6501 Software Testing
Software Testing and Maintenance 1
Paul Ammann & Jeff Offutt
Graph Coverage Criteria CS 4501 / 6501 Software Testing
Paul Ammann & Jeff Offutt
Input Space Partition Testing CS 4501 / 6501 Software Testing
Graph Coverage for Specifications CS 4501 / 6501 Software Testing
Coverage-Based Test Design CS 4501 / 6501 Software Testing
Jeff Offutt SWE 637 Software Testing
Paul Ammann & Jeff Offutt
Logic Coverage CS 4501 / 6501 Software Testing
Graph Coverage for Design Elements CS 4501 / 6501 Software Testing
Syntax-based Testing CS 4501 / 6501 Software Testing
Paul Ammann & Jeff Offutt
“White box” or “glass box” tests
Paul Ammann & Jeff Offutt
Logic Coverage CS 4501 / 6501 Software Testing
ISP Coverage Criteria CS 4501 / 6501 Software Testing
Graph Coverage for Design Elements CS 4501 / 6501 Software Testing
Graph: Data Flow Coverage Criteria CS 4501 / 6501 Software Testing
Sudipto Ghosh CS 406 Fall 99 November 16, 1999
Graph Coverage for Source Code
Paul Ammann & Jeff Offutt
Jeff Offutt SWE 637 Software Testing
Graph Coverage Criteria CS 4501 / 6501 Software Testing
Activity: Prime Path Coverage CS 4501 / 6501 Software Testing
Graph Coverage Criteria
Graph Coverage Criteria CS 4501 / 6501 Software Testing
Paul Ammann & Jeff Offutt
White-Box Testing Techniques II
Graph Coverage Criteria
Paul Ammann & Jeff Offutt
George Mason University
Paul Ammann & Jeff Offutt
Review for Final – Spring 2018
Paul Ammann & Jeff Offutt
Review for Final – Spring 2019
Presentation transcript:

Graph: Data Flow Criteria CS 4501 / 6501 Software Testing

Structures for Criteria-Based Testing Four structures for modeling software Input space Graph Source Design Specs Use cases Applied to Logic Source Specs FSMs DNF Applied to Syntax Source Models Integration Inputs Applied to FSM = Finite State Machine DNF = Disjunctive Normal Form ---R R--R RI-R RIPR

Graph Coverage Criteria Review Satisfaction Given a set TR of test requirements for a criterion C, a set of tests T satisfies C on a graph if and only if for every test requirement in TR, there is a test path in path(T) that meets the test requirement tr Two types Structural coverage criteria Define a graph just in terms of nodes and edges Data flow coverage criteria Requires a graph to be annotated with references to variables

Today’s Objectives Analyze data flow of software artifacts Understand how to integrate data flow into a graph model of the program under test Focusing on the flow of data, understand how to define criteria and design tests All-Defs Coverage (ADC) All-Uses Coverage (AUC) All-DU-Paths Coverage (ADUPC)

Value are carried from defs to uses, refer to as “du-pairs” Data Flow Criteria Goal: Ensure that the values are created and used correctly How: Focus on definitions and uses of values Definition (def): A location where a value for a variable is stored in a memory Use: A location where a variable’s value is accessed Also known definition-use, def-use, du associations Value are carried from defs to uses, refer to as “du-pairs”

Data Flow Criteria Data flow coverage criteria define test requirements TR in terms of the flows of data values in a graph G Steps: Develop a model of the software as a graph Integrate data flow into the graph A test requirement is met by visiting a particular node or edge or by touring a particular path

Def, Use, and DU Pairs def(n) or def(e): The set of variables defined by node n or edge e use(n) or use(e): The set of variables used by node n or edge e DU-pair: A pair of locations (li, lj) such that a variable v is defined at li and used at lj

Example defs: uses: du-pairs: for variable X def(1) = { X } def(5) = { Z } def(6) = { Z } X = 42 Z = X-8 Z = X*2 1 3 2 7 4 6 5 uses: use(5) = { X } use(6) = { X } du-pairs: for variable X (1, 5) (1, 6)

Example (2) All variables involved in a decision are assumed to be used on the associated edges a < b 1 2 3 4 a > b a = b use(1,2)={a,b} def(1)={a,b} use(1,3)={a,b} use(1,4)={a,b}

Def-clear and Reach Def-clear: A path from li to lj is def-clear with respect to variable v if v is not given another value on any of the nodes or edges in the path The values given in defs should reach at least one, some, or all possible uses. However, a def of a variable may or may not reach a particular use Why? No path goes from a location where a variable is defined to a location where the variable is used A variable’s value may be changed by another def before it reaches the use Reach: If there is a def-clear path from li to lj with respect to v, the def of v at li reaches the use at lj

DU-Paths du-path: A simple subpath that is def-clear with respect to a variable v from a def of v to a use of v du(ni, nj, v): the set of du-paths from ni to nj du(ni, v): the set of du-paths that start at ni Keep the path simple to ensure a reasonably small number of paths

Example defs: uses: du-paths: du-pairs: for variable X def(1) = { X } def(5) = { Z } def(6) = { Z } X = 42 Z = X-8 Z = X*2 1 3 2 7 4 6 5 uses: use(5) = { X } use(6) = { X } Def-clear? Simple path? du-paths: du(1, 5, X) = {[1,2,4,5], [1,3,4,5]} du(1, 6, X) = {[1,2,4,6] [1,3,4,6]} du-pairs: for variable X (1, 5) (1, 6)

More Example subject, pattern are forwarded parameters 1 2 3 4 5 6 8 9 7 10 11 NOTFOUND=-1 iSub=0 rtnIndex=NOTFOUND isPat=false subjectLen=subject.length() patternLen=pattern.length() iSub+patternLen–1 < subjectLen && isPat == false subject.charAt(iSub) == pattern.charAt(0) (iSub+patternLen-1 >= subjectLen || isPat != False rtnIndex=iSub isPat=true iPat=1 subject.charAt(iSub) != pattern.charAt(0) iPat < patternLen iPat >= patternLen subject.charAt(iSub+iPat) == pattern.charAt(iPat) break iSub++ != pattern.charAt(iPat) isPat=false; iPat++ return(rtnIndex)

More Example 1 2 3 4 5 6 8 9 7 10 11 def(1)={subject, pattern} use(3,11)=use(3,4)={iSub, patternLen, subjectLen, isPat} def(5)={rtnIndex, isPat, iPat} use(5)={iSub} use(4,10)=use(4,5)={subject, iSub, pattern} use(6,10)=use(6,7)={iPat, patternLen} break use(11)={rtnIndex} def(2)={NOTFOUND, iSub, rtnIndex, isPat, subjectLen, patternLen} use(2)={subject, pattern} use(7,8)=use(7,9)={subject, pattern, iSub, iPat} def(9)={iPat} use(9)={iPat} def(8)={rtnIndex, isPat} use(8)={NOTFOUND} def(10)={iSub} use(10)={iSub}

Categorizing DU-Paths The core of data flow testing – allowing definitions to flow to uses The test criteria for data flow will be defined as sets of du- paths. Thus, we first categorize the du-paths according to: def-path set du(ni, v): All simple paths w.r.t. a given variable v defined in a given node def-pair set du(ni, nj, v): All simple paths w.r.t. a given variable v from a given definition (ni) to a given use (nj)

Example: DU-Paths Consider iSub, categorize def-path sets and def-pair sets def-path sets du(10, iSub) = {[10,3,4], [10,3,4,5], [10,3,4,5,6,7,8], [10,3,4,5,6,7,9], [10,3,4,5,6,10], [10,3,4,5,6,7,8,10], [10,3,4,10], [10,3,11]} def-pair sets du(10, 4, iSub) = {[10,3,4]} du(10, 5, iSub) = {[10,3,4,5]} du(10, 8, iSub) = {[10,3,4,5,6,7,8]} du(10, 9, iSub) = {[10,3,4,5,6,7,9]} du(10, 10, iSub) = {[10,3,4,5,6,10], [10,3,4,5,6,7,8,10], [10,3,4,10]} du(10, 11, iSub) = {[10,3,11]}

Touring DU-Paths A test path p du-tours subpath d with respect to v if p tours d and the subpath taken is def-clear with respect to v Sidetrips can be used, just as with previous touring du-path sets du(1, X) = {[1,2,4,5], [1,3,4,5], [1,2,4,6], [1,3,4,6]} X = 42 Z = X-8 Z = X*2 1 3 2 7 4 6 5 du-pair sets du(1, 5, X) = {[1,2,4,5], [1,3,4,5]} du(1, 6, X) = [1,2,4,6], [1,3,4,6]} Test path [1,2,4,5,7] du-tours du-path [1,2,4,5]

Data Flow Coverage Criteria All-Defs Coverage (ADC) Use every def All-Uses Coverage (AUC) Get to every use All-du-Paths Coverage (ADUPC) Follow all du-paths

All-Defs Coverage (ADC) For each set of du-paths S = du(n,v), TR contains at least one path d in S For each def, at least one use must be reached X = 42 Z = X-8 Z = X*2 TR for X = {[1,2,4,5]} 1 3 2 7 4 6 5 Test paths = {[1,2,4,5,7]}

All-Uses Coverage (AUC) For each set of du-paths S = du(ni, nj, v), TR contains at least one path d in S For each def, all uses must be reached X = 42 Z = X-8 Z = X*2 TR for X = {[1,2,4,5], [1,2,4,6]} 1 3 2 7 4 6 5 Test paths = {[1,2,4,5,7], [1,2,4,6,7]}

All-DU-Paths Coverage (ADUPC) For each set of du-paths S = du(ni, nj, v), TR contains every path d in S For each def-use pair, all paths between defs and uses must be covered TR for X = {[1,2,4,5], [1,3,4,5], [1,2,4,6], [1,3,4,6]} X = 42 Z = X-8 Z = X*2 1 3 2 7 4 6 5 Test paths = {[1,2,4,5,7], [1,3,4,5,7], [1,2,4,6,7], [1,3,4,6,7]}

More Example 1 2 3 4 5 6 8 9 7 10 11 def(1)={subject, pattern} use(3,11)=use(3,4)={iSub, patternLen, subjectLen, isPat} def(5)={rtnIndex, isPat, iPat} use(5)={iSub} use(4,10)=use(4,5)={subject, iSub, pattern} use(6,10)=use(6,7)={iPat, patternLen} break use(11)={rtnIndex} def(2)={NOTFOUND, iSub, rtnIndex, isPat, subjectLen, patternLen} use(2)={subject, pattern} use(7,8)=use(7,9)={subject, pattern, iSub, iPat} def(9)={iPat} use(9)={iPat} def(8)={rtnIndex, isPat} use(8)={NOTFOUND} def(10)={iSub} use(10)={iSub} To enter Data flow graph coverage web app Edges 1 2 2 3 3 4 4 5 4 10 3 11 5 6 6 7 6 10 7 8 7 9 8 10 10 3 9 6 Initial node 1 Final node 11 Defs subject 1 pattern 1 NOTFOUND 2 iSub 2 10 rtnIndex 2 5 8 ispat 2 5 8 subjectLen 2 patternLen 2 iPat 5 9 Uses subject 2 4,10 4,5 7,8 7,9 pattern 2 4,10 4,5 7,8 7,9 iSub 3,11 3,4 4,10 4,5 5 10 7,8 7,9 patternLen 3,11 3,4 6,10 6,7 subjectLen 3,11 3,4 isPat 3,11 3,4 iPat 6,10 6,7 7,8 7,9 9 rtnIndex 11 NOTFOUND 8

More Example Def for subject Node 1 Use for subject Node 2 def(1)={subject, pattern} 1 2 3 4 5 6 8 9 7 10 11 use(3,11)=use(3,4)={iSub, patternLen, subjectLen, isPat} def(5)={rtnIndex, isPat, iPat} use(5)={iSub} use(4,10)=use(4,5)={subject, iSub, pattern} use(6,10)=use(6,7)={iPat, patternLen} break use(11)={rtnIndex} def(2)={NOTFOUND, iSub, rtnIndex, isPat, subjectLen, patternLen} use(2)={subject, pattern} use(7,8)=use(7,9)={subject, pattern, iSub, iPat} def(9)={iPat} use(9)={iPat} def(8)={rtnIndex, isPat} use(8)={NOTFOUND} def(10)={iSub} use(10)={iSub} Def for subject Node 1 Use for subject Node 2 Edges (4,10), (4,5), (7,8), (7,9) ADC for subject TR = {[1,2]} Test path = {[1,2,3,11]} Def for pattern Use for pattern ADC for pattern To enter Data flow graph coverage web app Edges 1 2 2 3 3 4 4 5 4 10 3 11 5 6 6 7 6 10 7 8 7 9 8 10 10 3 9 6 Initial node 1 Final node 11 Defs subject 1 pattern 1 NOTFOUND 2 iSub 2 10 rtnIndex 2 5 8 ispat 2 5 8 subjectLen 2 patternLen 2 iPat 5 9 Uses subject 2 4,10 4,5 7,8 7,9 pattern 2 4,10 4,5 7,8 7,9 iSub 3,11 3,4 4,10 4,5 5 10 7,8 7,9 patternLen 3,11 3,4 6,10 6,7 subjectLen 3,11 3,4 isPat 3,11 3,4 iPat 6,10 6,7 7,8 7,9 9 rtnIndex 11 NOTFOUND 8

More Example Def for NOTFOUND Node 2 Use for NOTFOUND Node 8 def(1)={subject, pattern} 1 2 3 4 5 6 8 9 7 10 11 use(3,11)=use(3,4)={iSub, patternLen, subjectLen, isPat} def(5)={rtnIndex, isPat, iPat} use(5)={iSub} use(4,10)=use(4,5)={subject, iSub, pattern} use(6,10)=use(6,7)={iPat, patternLen} break use(11)={rtnIndex} def(2)={NOTFOUND, iSub, rtnIndex, isPat, subjectLen, patternLen} use(2)={subject, pattern} use(7,8)=use(7,9)={subject, pattern, iSub, iPat} def(9)={iPat} use(9)={iPat} def(8)={rtnIndex, isPat} use(8)={NOTFOUND} def(10)={iSub} use(10)={iSub} Def for NOTFOUND Node 2 Use for NOTFOUND Node 8 ADC for NOTFOUND TR = {[1,2,3,4,5,6,7,8]} Test paths = {[1,2,3,4,5,6,7,8, 10,3,11]}

Example: ADC Variable Test paths (All-Defs Coverage) subject [1,2,3,11] pattern NOTFOUND [1,2,3,4,5,6,7,8,10,3,11] iSub [1,2,3,4,10,3,11] [1,2,3,4,10,3,4,10,3,11] rtnIndex [1,2,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] ispat No path or No path needed subjectLen [1,2,3,4,10,3,11] patternLen iPat [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,7,8,10,3,11] isPat

Example: AUC Variable Test paths (All-Uses Coverage) subject [1,2,3,11] [1,2,3,4,10,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,10,3,11] pattern NOTFOUND [1,2,3,4,5,6,7,8,10,3,11] iSub [1,2,3,4,5,6,10,3,11] [1,2,3,4,10,3,11] [1,2,3,4,10,3,4,5,6,10,3,11] [1,2,3,4,10,3,4,10,3,11] [1,2,3,11] [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,10,3,11] [1,2,3,4,10,3,11] [1,2,3,4,10,3,4,5,6,7,8,10,3,11] [1,2,3,4,10,3,4,5,6,7,9,6,10,3,11] rtnIndex [1,2,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] ispat No path or No path needed subjectLen [1,2,3,11] [1,2,3,4,10,3,11] patternLen [1,2,3,11] [1,2,3,4,10,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] iPat [1,2,3,4,5,6,7,9,6,10,3,11] [1,2,3,4,5,6,7,9,6,7,9,6,10,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,7,8,10,3,11] isPat

Example: AUC Variable Test paths (All-DU-Paths Coverage) subject [1,2,3,11] [1,2,3,4,10,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,9,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] pattern [1,2,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,10,3,11] NOTFOUND [1,2,3,4,5,6,7,8,10,3,11] iSub [1,2,3,4,10,3,11] [1,2,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,10,3,11] [1,2,3,4,10,3,4,10,3,11] [1,2,3,4,10,3,11] [1,2,3,4,10,3,4,5,6,10,3,11] [1,2,3,4,10,3,4,5,6,7,8,10,3,11] [1,2,3,4,10,3,4,5,6,7,9,6,10,3,11] rtnIndex [1,2,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,8,10,3,11] ispat No path or No path needed subjectLen [1,2,3,4,10,3,11] [1,2,3,11] patternLen [1,2,3,4,10,3,11] [1,2,3,11] [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,10,3,11] iPat [1,2,3,4,5,6,7,8,10,3,11] [1,2,3,4,5,6,10,3,11] [1,2,3,4,5,6,7,9,6,10,3,11] [1,2,3,4,5,6,7,9,6,7,8,10,3,11] [1,2,3,4,5,6,7,9,6,7,9,6,10,3,11] isPat

Graph Coverage Criteria Subsumption Simple Round Trip Coverage SRTC Node Coverage NC Edge Coverage EC Edge-Pair Coverage EPC Prime Path Coverage PPC Complete Path Coverage CPC Complete Round Trip Coverage CRTC All-DU-Paths Coverage ADUP All-uses Coverage AUC All-defs Coverage ADC