Predicate Complete Testing * Thomas Ball * Thomas Ball, A Theory of Predicate-Complete Test Coverage and Generation, Technical Report MSR-TR-2004- 28,

Slides:



Advertisements
Similar presentations
Data-Flow Analysis II CS 671 March 13, CS 671 – Spring Data-Flow Analysis Gather conservative, approximate information about what a program.
Advertisements

Planning based on Model Checking Dept. of Information Systems and Applied CS Bamberg University Seminar Paper Svetlana Balinova.
Static Single Assignment CS 540. Spring Efficient Representations for Reachability Efficiency is measured in terms of the size of the representation.
Lecture #21 Software Model Checking: predicate abstraction Thomas Ball Testing, Verification and Measurement Microsoft Research.
Bebop: A Symbolic Model Checker for Boolean Programs Thomas Ball Sriram K. Rajamani
SOFTWARE TESTING. INTRODUCTION  Software Testing is the process of executing a program or system with the intent of finding errors.  It involves any.
Graph Coverage (2).
1 Temporal Claims A temporal claim is defined in Promela by the syntax: never { … body … } never is a keyword, like proctype. The body is the same as for.
A Theory of Predicate-complete Test Coverage and Generation Thomas Ball Testing, Verification and Measurement Microsoft Research FMCO Symposium November.
August Moscow meeting1August Moscow meeting1August Moscow meeting11 Deductive tools in insertion modeling verification A.Letichevsky.
Lecture 2 White-box Testing and Structural Coverage.
COVERAGE CRITERIA FOR TESTING 1. ill-defined terms in Testing  complete testing  exhaustive testing  full coverage Are poorly defined terms because.
The Application of Graph Criteria: Source Code  It is usually defined with the control flow graph (CFG)  Node coverage is used to execute every statement.
Review of topics Final exam : -May 2nd to May 7 th - Projects due on May 7th.
Model Checking. Used in studying behaviors of reactive systems Typically involves three steps: Create a finite state model (FSM) of the system design.
White Box Testing and Symbolic Execution Written by Michael Beder.
Software Testing Sudipto Ghosh CS 406 Fall 99 November 16, 1999.
Automatically Validating Temporal Safety Properties of Interfaces Thomas Ball and Sriram K. Rajamani Software Productivity Tools, Microsoft Research Presented.
Data Flow Analysis Compiler Design October 5, 2004 These slides live on the Web. I obtained them from Jeff Foster and he said that he obtained.
Abstract Interpretation Part I Mooly Sagiv Textbook: Chapter 4.
From last lecture x := y op z in out F x := y op z (in) = in [ x ! in(y) op in(z) ] where a op b =
Complexity 19-1 Parallel Computation Complexity Andrei Bulatov.
Overview of program analysis Mooly Sagiv html://
Unit Testing CS 414 – Software Engineering I Don Bagert Rose-Hulman Institute of Technology January 16, 2003.
Antoine Girard VAL-AMS Project Meeting April 2007 Behavioral Metrics for Simulation-based Circuit Validation.
1 Software Testing Techniques CIS 375 Bruce R. Maxim UM-Dearborn.
272: Software Engineering Fall 2012 Instructor: Tevfik Bultan Lecture 4: SMT-based Bounded Model Checking of Concurrent Software.
Path testing Path testing is a “design structural testing” in that it is based on detailed design & the source code of the program to be tested. The methodology.
ECE355 Fall 2004Software Reliability1 ECE-355 Tutorial Jie Lian.
Cheng/Dillon-Software Engineering: Formal Methods Model Checking.
Software Systems Verification and Validation Laboratory Assignment 3
Structural Coverage Verilog code is available to help generate tests o Code can be analyzed statically and/or simulated Easier to detect “additive” design.
MIT Foundations of Dataflow Analysis Martin Rinard Laboratory for Computer Science Massachusetts Institute of Technology.
Software Testing – Lecture #2 Thomas Ball with material from M. Young, A. Memon and MSR’s FSE group.
Final Review  Different tests  Boundary Value Testing  Equivalence Class Testing  Decision Table Based Testing  Path Testing  Data Flow & P-Slice.
Coverage Literature of software testing is primarily concerned with various notions of coverage Four basic kinds of coverage: Graph coverage Logic coverage.
CS4311 Spring 2011 Unit Testing Dr. Guoqiang Hu Department of Computer Science UTEP.
1 Software Testing. 2 Path Testing 3 Structural Testing Also known as glass box, structural, clear box and white box testing. A software testing technique.
White-Box Software Testing Methods Speaker: Jerry Gao Ph.D. San Jose State University URL:
1 CS 201 Compiler Construction Introduction. 2 Instructor Information Rajiv Gupta Office: WCH Room Tel: (951) Office.
1 Program Testing (Lecture 14) Prof. R. Mall Dept. of CSE, IIT, Kharagpur.
BASIS PATH TESTING.
1 Outline:  Optimization of Timed Systems  TA-Modeling of Scheduling Tasks  Transformation of TA into Mixed-Integer Programs  Tree Search for TA using.
Convergence of Model Checking & Program Analysis Philippe Giabbanelli CMPT 894 – Spring 2008.
Syntax and Semantics CIS 331 Syntax: the form or structure of the expressions, statements, and program units. Semantics: the meaning of the expressions,
SOFTWARE TESTING. Introduction Software Testing is the process of executing a program or system with the intent of finding errors. It involves any activity.
1. Black Box Testing  Black box testing is also called functional testing  Black box testing ignores the internal mechanism of a system or component.
Data Flow Analysis II AModel Checking and Abstract Interpretation Feb. 2, 2011.
White-Box Testing Techniques I Prepared by Stephen M. Thebaut, Ph.D. University of Florida Software Testing and Verification Lecture 7.
Iterative Dataflow Problems Taken largely from notes of Alex Aiken (UC Berkeley) and Martin Rinard (MIT) Dataflow information used in optimization Several.
White-Box Testing Statement coverage Branch coverage Path coverage
SOFTWARE TESTING LECTURE 9. OBSERVATIONS ABOUT TESTING “ Testing is the process of executing a program with the intention of finding errors. ” – Myers.
1 Software Testing. 2 What is Software Testing ? Testing is a verification and validation activity that is performed by executing program code.
Introduction to Software Testing (2nd edition) Chapter 5 Criteria-Based Test Design Paul Ammann & Jeff Offutt
Software TestIng White box testing.
BASIS PATH TESTING.
White-Box Testing Pfleeger, S. Software Engineering Theory and Practice 2nd Edition. Prentice Hall, Ghezzi, C. et al., Fundamentals of Software Engineering.
Software Engineering (CSI 321)
Graph Coverage Criteria CS 4501 / 6501 Software Testing
Structural testing, Path Testing
Types of Testing Visit to more Learning Resources.
White-Box Testing.
Alternating tree Automata and Parity games
Sudipto Ghosh CS 406 Fall 99 November 16, 1999
White-Box Testing Techniques I
Data Flow Analysis Compiler Design
 = N  N matrix multiplication N = 3 matrix N = 3 matrix N = 3 matrix
White-Box Testing.
Resolution in predicate Logic
Whitebox Testing.
Presentation transcript:

Predicate Complete Testing * Thomas Ball * Thomas Ball, A Theory of Predicate-Complete Test Coverage and Generation, Technical Report MSR-TR , Microsoft Research

Testing Definitions Statement  Every statement is executed by the test set Path  Every control path is executed by the test set Predicate  Every logical predicate is executed by the test set Complete  Some notion that the test set is “exhaustive”

Theoretical Goals Control Flow metrics  Statement, branch, multiple condition, predicate, path, linear code sequence and jump Predicate Complete Testing  Cover all reachable observable states  Bound the number tests  Know when to stop (when do we have a complete test set) Does PCT actually produce tests?

Using Boolean Programs Recall that BP transforms the original program into a program of its Boolean predicates A BP state is some Boolean assignment of all predicates The BP state space is finite (<= m*2^n) Some BP states are observable Some BP states are reachable

Boolean Program

Reachable States

Reachability Graph

Formalizing the state abstraction

Boolean Program II

Upper and Lower bounds for R R – set of reachable states U – upper bound for R via may-transitions Lp – pessimistic lower bound for R via must+ transitions, a may transition, and must- transitions Lo – optimistic lower bound for R, with the assumption that the program does not diverge

Algorithm Using algorithms for computing L and U, |L|==|U| => the abstraction accurately encodes all the paths of R. Generate all paths for L in the graph

Test Generation

Discussion