Testing Object-Oriented Software Test Coverage Criteria

Slides:



Advertisements
Similar presentations
Data Flow Coverage. Reading assignment L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, "A Formal Evaluation of Data Flow Path Selection.
Advertisements

Module 8 “Polymorphism and Inheritance”. Outline Understanding Inheritance Inheritance Diagrams Constructors in Derived Classes Type Compatibility Polymorphism.
Graph Coverage for Design Elements 1.  Use of data abstraction and object oriented software has increased importance on modularity and reuse.  Therefore.
OBJECT ORIENTED PROGRAMMING M Taimoor Khan
1 Software Testing and Quality Assurance Lecture 12 - The Testing Perspective (Chapter 2, A Practical Guide to Testing Object-Oriented Software)
OOP in Java Nelson Padua-Perez Chau-Wen Tseng Department of Computer Science University of Maryland, College Park.
Data Abstraction and Object- Oriented Programming CS351 – Programming Paradigms.
CS 2511 Fall  Abstraction Abstract class Interfaces  Encapsulation Access Specifiers Data Hiding  Inheritance  Polymorphism.
Testing OO Programs (7.1, 2.2.2, 2.4.2) Course Software Testing & Verification 2014/15 Wishnu Prasetya.
C++ fundamentals.
Inheritance. © 2004 Pearson Addison-Wesley. All rights reserved 8-2 Inheritance Inheritance is a fundamental object-oriented design technique used to.
Chapter 15 – Inheritance, Virtual Functions, and Polymorphism
Introduction to Software Testing Chapter 2.4 Graph Coverage for Design Elements Paul Ammann & Jeff Offutt
Software testing techniques Testing criteria based on data flow
Introduction to Software Testing Chapter 7.1 Engineering Criteria for Technologies Paul Ammann & Jeff Offutt
Software Testing and Maintenance Lecture 4 Graph Coverage for Design Element Paul Ammann & Jeff Offutt Instructor: Hossein Momeni Mazandaran.
Unified Modeling Language, Version 2.0
CS200 Algorithms and Data StructuresColorado State University Part 4. Advanced Java Topics Instructor: Sangmi Pallickara
Introduction to Software Testing Chapter 8.1 Building Testing Tools –Instrumentation Paul Ammann & Jeff Offutt
Inheritance - Polymorphism ITI 1121 Nour El Kadri.
1 COMP313A Programming Languages Object Oriented Progamming Languages (3)
1 Graph Coverage (4). Reading Assignment P. Ammann and J. Offutt “Introduction to Software Testing” ◦ Section
OOP: Encapsulation,Abstraction & Polymorphism. What is Encapsulation Described as a protective barrier that prevents the code and data being randomly.
Dale Roberts Object Oriented Programming using Java - Introduction Dale Roberts, Lecturer Computer Science, IUPUI Department.
Software Verification Graph Model 2. 2 Graph Coverage Four Structures for Modeling Software Graphs Logic Input Space Syntax Use cases Specs Design Source.
Introduction to Software Testing (2nd edition) Chapter 7.4 Graph Coverage for Design Elements Paul Ammann & Jeff Offutt
Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Java Software Solutions Foundations of Program Design Sixth Edition by Lewis.
Operator Overloading: indexing Useful to create range-checked structures: class four_vect { double stor[4]; // private member, actual contents of vector.
Introduction to Software Testing Chapter 5.3 Integration and Object-Oriented Testing Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 5.3 Integration and Object-Oriented Testing Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 7.1 Engineering Criteria for Technologies Paul Ammann & Jeff Offutt
Testing Inheritance & Polymorphism in OO Software using Formal Specification Presented by : Mahreen Aziz Ahmad (Center for Software Dependability, MAJU)
Testing OO software. State Based Testing State machine: implementation-independent specification (model) of the dynamic behaviour of the system State:
Graph Coverage for Design Elements 1.  Use of data abstraction and object oriented software has increased importance on modularity and reuse.  Therefore.
Chapter 18 Object Database Management Systems. Outline Motivation for object database management Object-oriented principles Architectures for object database.
Java Software Solutions Lewis and Loftus Chapter 8 Copyright 1997 by John Lewis and William Loftus. All rights reserved. 1 Inheritance -- Introduction.
Banaras Hindu University. A Course on Software Reuse by Design Patterns and Frameworks.
Recap Introduction to Inheritance Inheritance in C++ IS-A Relationship Polymorphism in Inheritance Classes in Inheritance Visibility Rules Constructor.
Coupling-based Criteria for Integration Testing Journal of Software Testing, Verification, and Analysis, 8(3): , September 1998, Jenny Jin and Jeff.
Copyright © 2009 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Outline Creating Subclasses Overriding Methods Class Hierarchies Inheritance.
 Description of Inheritance  Base Class Object  Subclass, Subtype, and Substitutability  Forms of Inheritance  Modifiers and Inheritance  The Benefits.
Chapter 2 : Graph Coverage (part 2)
Lecture 10 – Polymorphism Nancy Harris with additional slides Professor Adams from Lewis & Bernstein.
Design issues for Object-Oriented Languages
Modern Programming Tools And Techniques-I
Paul Ammann & Jeff Offutt
Inheritance Chapter 7 Inheritance Basics Programming with Inheritance
Object-Oriented Analysis and Design
Dataflow Testing, Object-Oriented
Software Verification
Course Software Testing & Verification 2016/17 Wishnu Prasetya
Testing Object-Oriented Software Concepts and Definitions
Graph Coverage for Design Elements CS 4501 / 6501 Software Testing
TIM 58 Chapter 8: Class and Method Design
Object Oriented Programming
Designing for Inheritance
Inheritance Basics Programming with Inheritance
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Inheritance, Polymorphism, and Interfaces. Oh My
Chapter 20 Object-Oriented Analysis and Design
More Object-Oriented Programming
Computer Programming with JAVA
Graph Coverage for Design Elements CS 4501 / 6501 Software Testing
OMEN: A Strategy for Testing Object-Oriented Software
Overview of C++ Polymorphism
Review of Previous Lesson
Paul Ammann & Jeff Offutt
Data-flow Based Testing (A&O Ch , , 7. 1) (2nd Ed. Ch 7
Software Testing and QA Theory and Practice (Chapter 5: Data Flow Testing) © Naik & Tripathy 1 Software Testing and Quality Assurance Theory and Practice.
Presentation transcript:

Testing Object-Oriented Software Test Coverage Criteria Jeff Offutt SWE 737 Advanced Software Testing

Testing OO Software Most of these ideas were developed after 2000 Few are widely used Most adapt graph-based testing SWE 737 © Jeff Offutt

OO Testing Outline Example The Yo-Yo Graph and Polymorphism Categories of Inheritance Faults Testing Inheritance, Polymorphism and Dynamic Binding Object-Oriented Testing Criteria SWE 737 © Jeff Offutt

Example DU Pairs and Anomalies Consider what happens when an overriding method has a different def-set than the overridden method B +h () +i () -x A -u -v -w +h() +i() +j() +l() C +j () -y def-use Method Defs Uses A::h () {A::u, A::w} A::i () {A::u} A::j () {A::v} {A::w} A::l() B::h() {B::x} B::i() C::i() C::j() {C::y} A::h() calls j(), B::h() does not DU anomaly DU anomaly def-use SWE 737 © Jeff Offutt

OO Testing Outline Example The Yo-Yo Graph and Polymorphism Categories of Inheritance Faults Testing Inheritance, Polymorphism and Dynamic Binding Object-Oriented Testing Criteria The Yo-Yo Graph and Polymorphism SWE 737 © Jeff Offutt

Polymorphism Headaches (Yo-Yo) Actual type A d() g() h() i() j() l() +d () +g () +h () +i () +j () +l () B h() i() k() C i() j() l() Object is of actual type A A::d () d() calls g(), which calls h(), which calls i(), which calls j() SWE 737 © Jeff Offutt

Polymorphism Headaches (Yo-Yo) g() h() i() j() l() +d () +g () +h () +i () +j () +l () B h() i() k() C i() j() l() A d() g() h() i() j() l() B Actual type +h () +i () +k () B h() i() k() C i() j() l() Object is of actual type B B::d () SWE 737 © Jeff Offutt

Polymorphism Headaches (Yo-Yo) g() h() i() j() l() +d () +g () +h () +i () +j () +l () B h() i() k() C i() j() l() A d() g() h() i() j() l() B +h () +i () +k () B h() i() k() C i() j() l() A d() g() h() i() j() l() B h() i() k() C Actual type +i () +j () +l () C i() j() l() Object is of actual type C, C::d () SWE 737 © Jeff Offutt

OO Testing Outline Example The Yo-Yo Graph and Polymorphism Categories of Inheritance Faults Testing Inheritance, Polymorphism and Dynamic Binding Object-Oriented Testing Criteria Categories of Inheritance Faults SWE 737 © Jeff Offutt

Potential for Faults in OO Programs Complexity is relocated to the connections among components Less static determinism – many faults can now only be detected at runtime Inheritance and Polymorphism yield vertical and dynamic integration Aggregation and use relationships are more complex Designers do not carefully consider visibility of data and methods SWE 737 © Jeff Offutt

Object-oriented Faults Only consider faults that arise as a direct result of OO language features: inheritance polymorphism constructors visibility Language independent (as much as possible) SWE 737 © Jeff Offutt

OO Faults and Anomalies Acronym Fault / Anomaly ITU Inconsistent Type Use SDA State Definition Anomaly SDIH State Definition Inconsistency SDI State Defined Incorrectly IISD Indirect Inconsistent State Definition ACB1 Anomalous Construction Behavior (1) ACB2 Anomalous Construction Behavior (2) IC Incomplete Construction SVA State Visibility Anomaly Examples shown SWE 737 © Jeff Offutt

Inconsistent Type Use (ITU) No overriding (no polymorphism) C extends T, and C adds new methods (extension) An object is used “as a C”, then as a T, then as a C Methods in T can put object in state that is inconsistent for C s.push (“Steffi”); s.push (“Joyce”); s.push (“Andrew”); dumb (s); s.pop(); void dumb (Vector v) { v.removeElementAt (v.size()-1); } Vector -array +insertElementAt() +removeElementAt() // Stack is empty! Stack +pop (): Object +push (): Object call call SWE 737 © Jeff Offutt

State Definition Anomaly (SDA) X extends W, and X overrides some methods The overriding methods in X fail to define some variables that the overridden methods in W defined W v u m() n() X x Y w W::m () defines v and W::n() uses v X::n () uses v Y::m () does not define v For an object of actual type Y, a data flow anomaly exists and results in a fault if m() is called, then n() SWE 737 © Jeff Offutt

State Definition Inconsistency (SDIH) Hiding a variable, possibly accidentally If the descendant’s version of the variable is defined, the ancestor’s version may not be W v u m() n() X x Y Y overrides W’s version of v Y::m() defines Y::v X::n() uses v … getting W’s version of v For an object of actual type Y, a data flow inconsistency may exist and result in a fault if m() is called, then n() overrides (hides) SWE 737 © Jeff Offutt

Anomalous Construction Behavior (ACB1) Constructor of W calls a method f() A child of W, X, overrides f() X::f() uses variables that should be defined by X’s constructor W W() f() X x X() Calls Uses Overrides When an object of type X is constructed, W() is run before X(). When W() calls X::f(), x is used, but has not yet been given a value! SWE 737 © Jeff Offutt

State Visibility Anomaly (SVA) A private variable v is declared in ancestor W, and v is defined by W::m() X extends W and Y extends X Y overrides m(), and calls W::m() to define v W -v m() X Y Overrides, calls W -v m() X Y Overrides, calls Overrides X::m() is added later X::m() does not call W::m() Y:m() can no longer call W::m(), so v is not defined! SWE 737 © Jeff Offutt

OO Testing Outline Example The Yo-Yo Graph and Polymorphism Categories of Inheritance Faults Testing Inheritance, Polymorphism and Dynamic Binding Object-Oriented Testing Criteria Testing Inheritance, Polymorphism and Dynamic Binding SWE 737 © Jeff Offutt

Coupling Sequences Pairs of method calls within body of method under test: Made through a common instance context With respect to a set of state variables that are commonly referenced by both methods Consists of at least one coupling path between the two method calls with respect to a particular state variable Represent potential state space interactions between the called methods with respect to calling method Used to identify points of integration and testing requirements SWE 737 © Jeff Offutt

Types of Def-Use Pairs def use A () intra-procedural data flow (within the same unit) inter-procedural data flow def A () B () use last-def A () first-use B () full coupling def A() use B() A() B() F () object-oriented direct coupling data flow object-oriented indirect coupling data flow def A() use B() M () N() F() A() M() B() N() SWE 737 © Jeff Offutt

Coupling-Based Testing (from Ch 2) Test data and control connections Derived from previous work for procedural programs Based on insight that integration occurs through couplings among software artifacts Caller F x = 14 y = G (x) print (y) print (a) b = 42 return (b) G (a) Callee last-def-before-call first-use-in-callee call site first-use-after-call last-def-before-return SWE 737 © Jeff Offutt

Polymorphic Call Set public void f ( W o ) { … j o.m(); l o.l(); k o.n(); } Set of methods that can potentially execute as result of a method call through a particular instance context pcs (o.m) = {W::m, Y::m, X::m} SWE 737 © Jeff Offutt

Example Coupling Sequence o bound to instance of W h def (o) Client f i o.m() j o.l() k o.n() W -v : -u : +m() +n() +l() Z X -x : Y -w : m () def (W::v) Coupling sequence with respect to W::v l () def (W::u) Coupling sequence with respect to W::u n () use (W::v) use (W::u) SWE 737 © Jeff Offutt

Example Coupling Sequence (2) o bound to instance of Z h def (o) Client f i o.m() j o.l() k o.n() W -v : -u : +m() +n() +l() X -x : Y -w : Z m () def (Z::x) Coupling sequence with respect to Z::x l () def (W::u) n () use (Z::x) SWE 737 © Jeff Offutt

OO Testing Outline Example The Yo-Yo Graph and Polymorphism Categories of Inheritance Faults Testing Inheritance, Polymorphism and Dynamic Binding Object-Oriented Testing Criteria Object-Oriented Testing Criteria SWE 737 © Jeff Offutt

Testing Goals We want to test how a method can interact with instance bound to object o: Interactions occur through the coupling sequences Need to consider the set of interactions that can occur: What types can be bound to o? Which methods can actually execute? (polymorphic call sets) Test all couplings with all type bindings possible SWE 737 © Jeff Offutt

All-Coupling-Sequences All-Coupling-Sequences (ACS) : For every coupling sequence Sj in f(), there is at least one test case t such that there is a coupling path induced by Sj,k that is a sub-path of the execution trace of f(t) At least one coupling path must be executed Does not consider inheritance and polymorphism SWE 737 © Jeff Offutt

All-Poly-Classes All-Poly-Classes (APC) : For every coupling sequence Sj,k in method f(), and for every class in the family of types defined by the context of Sj,k, there is at least one test case t such that when f() is executed using t, there is a path p in the set of coupling paths of Sj,k that is a sub-path of the execution trace of f(t) Includes instance contexts of calls At least one test for every type the object can bind to Test with every possible type substitution SWE 737 © Jeff Offutt

All-Coupling-Defs-Uses All-Coupling-Defs-Uses (ACDU) : For every coupling variable v in each coupling Sj,k of t, there is a coupling path induced by Sj,k such that p is a sub-path of the execution trace of f(t) for at last one test case t Every last definition of a coupling variable reaches every first use Does not consider inheritance and polymorphism SWE 737 © Jeff Offutt

All-Poly-Coupling-Defs-and-Uses All-Poly-Coupling-Defs-and-Uses (APDU) : For every coupling sequence Sj,k in f(), for every class in the family of types defined by the context of Sj,k, for every coupling variable v of Sj,k, for every node m that has a last definition of v and every node n that has a first-use of v, there is at least one test case t such that when f() is executed using t, there is a path p in the coupling paths of Sj,k that is a sub-path of the trace of f() Every last definition of a coupling variable reaches every first use for every type binding Combines previous criteria Handles inheritance and polymorphism Takes definitions and uses of variables into account SWE 737 © Jeff Offutt

OO Coverage Criteria Subsumption All-Coupling-Sequences ACS All-Poly-Coupling Defs-Uses APDU All-Poly-Classes APC All-Coupling-Defs-Uses ACDU SWE 737 © Jeff Offutt

Conclusions A model for understanding and analyzing faults that occur as a result of inheritance and polymorphism Yo-yo graph Defs and Uses of state variables Polymorphic call set Technique for identifying data flow anomalies in class hierarchies A fault model and specific faults that are common in OO software Specific test criteria for detecting such faults SWE 737 © Jeff Offutt

References Testing Object Oriented Software: A Survey, Robert V. Binder, Wiley’s Software Testing, Verification and Reliability, 6(3-4):125- 252, December 1996 Testing Object-oriented Software Systems, Harry M. Sneed, 24th European Conference on Object-Oriented Programming (ECOOP), Maribor, Slovenia, June 2010 (Anastasia) Testing Coupling Relationships in Object-Oriented Programs, Roger T. Alexander, Jeff Offutt, and Andreas Stefik Wiley’s Journal of Software Testing, Verification, and Reliability, 20(4):291-327, December 2010 SWE 737 © Jeff Offutt