Fun facts about the cal() program

Slides:



Advertisements
Similar presentations
Introduction to Software Testing Chapter 1
Advertisements

Making Choices in C if/else statement logical operators break and continue statements switch statement the conditional operator.
Introduction to Software Testing Chapter 3.3 Logic Coverage from Source Code Paul Ammann & Jeff Offutt.
Introduction to Software Testing Chapter 3.1, 3.2 Logic Coverage Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 1 Paul Ammann & Jeff Offutt SUMMARY OF PARTS 1 AND 2 FROM LAST WEEK.
Mutant Subsumption Graphs Mutation 2014 March 31, 2014 Bob Kurtz, Paul Ammann, Marcio Delamaro, Jeff Offutt, Lin Deng.
Introduction to Software Testing Chapter 5.2 Program-based Grammars Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 9.4 Model-Based Grammars Paul Ammann & Jeff Offutt
Overview of Software Testing 07/12/2013 WISTPC 2013 Peter Clarke.
Introduction to Software Testing
Introduction to Software Testing Chapter 8.1 Building Testing Tools –Instrumentation Paul Ammann & Jeff Offutt
SWE 637: Test Criteria and Definitions Tao Xie Prepared based on Slides by ©Paul Ammann and Jeff Offutt Revised by Tao Xie.
Testing Testing Techniques to Design Tests. Testing:Example Problem: Find a mode and its frequency given an ordered list (array) of with one or more integer.
Test Drivers and Stubs More Unit Testing Test Drivers and Stubs CEN 5076 Class 11 – 11/14.
Test Coverage CS-300 Fall 2005 Supreeth Venkataraman.
637 – Introduction (Ch 1) Introduction to Software Testing Chapter 1 Jeff Offutt Information & Software Engineering SWE 437 Software Testing
Combinatorial Testing Review “Combinatorial Software Testing”, Kuhn, Kacker, Lei, Hunter, IEEE Computer, Aug Compared “traditional” software test.
Introduction to Software Testing Chapters 1-5 Coverage Summary Paul Ammann & Jeff Offutt
First delivery of the course Software Quality and Testing Katerina Zdravkova, Anastas Mišev
Paul Ammann & Jeff Offutt
A Few Review Questions Dan Fleck Fall System Test Case Enter invalid username in the input box Able to enter text Enter invalid password in the.
Introduction to Software Testing Paul Ammann & Jeff Offutt Updated 24-August 2010.
Mutation Testing G. Rothermel. Fault-Based Testing White-box and black-box testing techniques use coverage of code or requirements as a “proxy” for designing.
1 Input Space Partitioning(2). Reading Assignment P. Ammann and J. Offutt “Introduction to Software Testing” ◦ Chapter 4  Section 4.1  Section
Software Testing Part II March, Fault-based Testing Methodology (white-box) 2 Mutation Testing.
Introduction to Software Testing Chapter 9.2 Program-based Grammars Paul Ammann & Jeff Offutt
Establishing Theoretical Minimal Sets of Mutants ICST 2014 Paul Ammann Joint work with Marcio Eduardo Delamaro Jeff Offutt April 1, 2014.
1 Graph Coverage (3). Reading Assignment P. Ammann and J. Offutt “Introduction to Software Testing” ◦ Section 2.2 ◦ Section
Some facts about the cal() program February 5, 2015 Paul Ammann, SWE 437 with thanks to Bob Kurtz.
Workshop on Integrating Software Testing into Programming Courses (WISTPC14:2) Friday July 18, 2014 Introduction to Software Testing.
1 Using a Fault Hierarchy to Improve the Efficiency of DNF Logic Mutation Testing Gary Kaminski and Paul Ammann ICST 2009.
Foundations of Software Testing Chapter 7: Test Adequacy Measurement and Enhancement Using Mutation Last update: September 3, 2007 These slides are copyrighted.
CS223: Software Engineering Lecture 26: Software Testing.
Introduction to Software Testing (2nd edition) Chapter 5 Criteria-Based Test Design Paul Ammann & Jeff Offutt
A Few Review Questions Dan Fleck Fall System Test Case Enter invalid username in the input box Able to enter text Enter invalid password in the.
Module 7 Halting Problem –Fundamental program behavior problem –A specific unsolvable problem –Diagonalization technique revisited Proof more complex 1.
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 8.1 Logic Coverage
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 9.2 Program-based Grammars
Mutation Testing Moonzoo Kim School of Computing KAIST
Paul Ammann & Jeff Offutt
Input Space Partition Testing CS 4501 / 6501 Software Testing
Analyzing the Validity of Selective Mutation with Dominator Mutants
Introduction to Software Testing Chapter 3.1, 3.2 Logic Coverage
Structural testing, Path Testing
Introduction to Software Testing Chapter 5.1 Syntax-based Testing
UNIT-4 BLACKBOX AND WHITEBOX TESTING
Introduction to Software Testing Chapter 2 Model-Driven Test Design
Introduction to Software Testing Chapter 9.2 Program-based Grammars
Paul Ammann & Jeff Offutt
Introduction to Software Testing Chapter 5.2 Program-based Grammars
A Few Review Questions Dan Fleck Fall 2009.
Introduction to Software Testing (2nd edition) Chapter 4 TDD Example
A Few Review Questions.
SWE 795: Advanced Testing Techniques
Introduction to Software Testing Chapter 5.1 Syntax-based Testing
Mutation Testing Moonzoo Kim School of Computing KAIST
Introduction to Software Testing Chapter 8.1 Logic Coverage
George Mason University
A Few Review Questions Dan Fleck Spring 2009.
Paul Ammann & Jeff Offutt
Paul Ammann & Jeff Offutt
Whitebox Testing.
Paul Ammann & Jeff Offutt
UNIT-4 BLACKBOX AND WHITEBOX TESTING
Introduction to Software Testing Chapter 3.1, 3.2 Logic Coverage
Mutation Testing Faults are introduced into the program by creating many versions of the program called mutants. Each mutant contains a single fault. Test.
Presentation transcript:

Fun facts about the cal() program Paul Ammann, SWE 437 with thanks to Bob Kurtz

The cal() Example cal() calculates the number of days between two dates in the same year Chosen for its well-defined finite input space A selective mutation tool (muJava) generates 173 mutants (AO Chapter 9) 28 are equivalent (hand analysis) Input domain model and pairwise testing (AO Chapter 6) yields 90 tests with ACTS tool (NIST)

cal() code and variations public static int cal(int month1, int day1, int month2, int day2, int year) { int numDays; if (month2 == month1) numDays = day2 - day1; else { int daysIn[] = {0, 31, 0, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}; int m4 = year % 4; int m100 = year % 100; int m400 = year % 400; if ((m4 != 0) || ((m100 == 0) && (m400 != 0))) daysIn[2] = 28; else daysIn[2] = 29; numDays = day2 + (daysIn[month1] - day1); for (int i = month1 + 1; i <= month2-1; i++) numDays = daysIn[i] + numDays; } return (numDays); what about switching June and July? 31, 30 || what about getting the logic wrong? != what about boundary faults?

Subsumption: Which Mutants to Keep Test set T Tests that kill mj Tests that kill mk Tests that kill mi ✔ ✖ ? ? mi → mj mi → mk

cal() Dynamic Subsumption DMSG Nodes 30 Minimal Nodes 7 Equiv 28

cal() True Subsumption TMGS DMSG Nodes 38 30 Minimal Nodes 10 7* Equiv 28 * All are correct, but not fully distinguished

Takeaways for the 437 student This week’s assignment was a warm-up Simple programs can encode complex behavior Surprising few tests can pin down behavior Engineers need help finding these tests What coverage criteria do! (AO Chapters 6-9) ps – don’t stress out on the subsumption graphs given here– you won’t be tested on that