1. Jeff Offutt SWE 637 Software Testing www.cs.gmu.edu/~offutt/
Review for Final – Fall 2018
Jeff Offutt
SWE 637
Software Testing

2. Parameters and logistics
4:30 – 7:15
Same room
17 December, next next Monday
10-December is a reading day
No quiz retakes after 10-December
Closed book / closed notes / closed neighbor
One sheet of definitions allowed
Hand-written, not typed, double-sided is okay
Comprehensive
You may take breaks (not all together)
Spread out – separate tables and one table per person
637 - Final Review
© Jeff Offutt

3. Summary: Part 1—Foundations
Why Test Software? (Ch 1)
Fault, error, failure, program state
Test maturity levels
Cost of late testing
Model-Driven Test Design (Ch 2)
RIPR model
Other terms
Test Automation (Ch 3)
Junit
Components of a test case
Developer Testing (Ch 4)
Agile and TDD
Criteria-Based Test Design (Ch 5)
Test requirements and criteria
637 - Final Review
© Jeff Offutt

4. Cost of Late Testing 60 50 40 30 20 10
Assume $1000 unit cost, per fault, 100 faults
$250K
$360K
Fault origin (%)
Fault detection (%)
Unit cost (X)
$20K
$100K
$13K
$6K
28-Oct-2010, at GTAC, added the animation to demonstrate increasing the number of faults found early, thereby decreasing the number of faults found late, and finally saving money. Lots of it! This animation is fairly complicated … must practice first!!
Requirements
Design
Prog / Unit Test
Integration Test
System Test
Post-Deployment
Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96-HB-002
637 - Final Review
© Jeff Offutt

5. Incorrect Program State
RIPR Model
Reachability
Infection
Propagation
Revealability
Test
Final Program State
Observed Final Program State
Reaches
Fault
Incorrect Final State
Infects
Incorrect Program State
Propagates
The failures can be revealed when the observed final program state has overlap with the incorrect final program state.
The question is: should testers check the entire program state? How to observe the incorrect program state in a cost-effective manner.
Getting the overlap as big as possible and use the cost as small as possible.
Reveals
Test Oracles
637 - Final Review
© Jeff Offutt

6. Model-Driven Test Design–Activities
refined requirements / test specs
Test Design
model / structure
test requirements
DESIGN
ABSTRACTION
LEVEL
IMPLEMENTATION
ABSTRACTION
LEVEL
software artifact
input values
Test Automation
Test Evaluation
pass / fail
test results
Test Execution
test scripts
test cases
637 - Final Review
© Jeff Offutt

7. Criteria Based on Structures
Structures : Four ways to model software
Input Domain Characterization (sets)
A: {0, 1, >1}
B: {600, 700, 800}
C: {swe, cs, isa, infs}
Graphs
Logical Expressions
(not X or not Y) and A and B
if (x > y)
z = x - y;
else
z = 2 * x;
Syntactic Structures (grammars)
637 - Final Review
© Jeff Offutt

8. Summary: Part 2 Coverage Criteria (1 of 4)
Input Space Partitioning – Ch 6
Input Domain Modeling (6.1)
Combination Strategies Criteria (6.2)
Handling Constraints Among Characteristics (6.3)
Extended Example: Deriving an IDM from JavaDoc (6.4)
Be sure to review the coverage definitions in 6.2
Know how to construct partitions and blocks
Know how to choose values
637 - Final Review
© Jeff Offutt

9. ISP Coverage Criteria Subsumption
Each Choice Coverage
ECC
All Combinations Coverage
ACoC
T-Wise Coverage
TWC
Multiple Base Choice Coverage
MBCC
Pair-Wise Coverage
PWC
Base Choice Coverage
BCC
637 - Final Review
© Jeff Offutt

10. Summary: Part 2 Coverage Criteria (2 of 4)
Graph Coverage – Ch 7
Overview (7.1)
Graph Coverage Criteria (7.2)
Graph Coverage for Source Code (7.3)
Graph Coverage for Design Elements (7.4)
Graph Coverage for Specifications (7.5)
Graph Coverage for Use Cases (7.6)
Be sure to review the definitions in 7.2
Know how to generate graphs for various artifacts and apply the criteria to them
Know how to choose values (EC, EPC, PPC, …)
637 - Final Review
© Jeff Offutt

11. Graph Coverage Criteria Subsumption
Simple Round Trip Coverage
SRTC
Node Coverage NC
Edge Coverage EC
Edge-Pair Coverage
EPC
Prime Path Coverage
PPC
Complete Path Coverage
CPC
Complete Round Trip Coverage
CRTC
All-DU-Paths Coverage
ADUP
All-uses Coverage
AUC
All-defs Coverage
ADC
637 - Final Review
© Jeff Offutt

12. Summary: Part 2 Coverage Criteria (3 of 4)
Logic Coverage – Ch 8
Semantic Logic Coverage Criteria (Active) (8.1)
Syntactic Logic Coverage Criteria (Inactive) (8.2)
Structural Logic Coverage of Programs (8.3)
Specification-based Logic Coverage (8.4)
Logic Coverage of FSMs (8.5)
Be sure to review the coverage definitions in 8.1
Know how to apply the coverage criteria to programs, specs, and FSMs
Know how to choose values (CACC, …)
637 - Final Review
© Jeff Offutt

13. Logic Criteria Subsumption
Clause Coverage CC
Predicate Coverage PC
Combinatorial Clause Coverage
COC
Restricted Active Clause Coverage
RACC
Restricted Inactive Clause Coverage
RICC
General Active Clause Coverage
GACC
Correlated Active Clause Coverage
CACC
General Inactive Clause Coverage
GICC
637 - Final Review
© Jeff Offutt

14. Summary: Part 2 Coverage Criteria (4 of 4)
Syntax-based Testing – Ch 9
Syntax-based Coverage Criteria (9.1)
Program-based Grammars (9.2)
Integration and Object-Oriented Testing (9.3)
Concepts, not all the details
Specification-based Grammars (09.4)
Input Space Grammars (9.5)
Understand how to derive tests from BNF
Understand mutation analysis and mutation testing
637 - Final Review
© Jeff Offutt

15. Representing Input Domains
goal domain
specified domain
This region is a rich source of software errors …
implemented domain
637 - Final Review
© Jeff Offutt

16. Preparing & Format Re-read the text and handouts
Study the homework assignments
Redo the quizzes
Try exercises in text
Get a good night’s sleep
Possibly different kinds of questions
Mostly problem solving—similar to quizzes, perhaps bigger
Short answer
Maybe a few T / F, M / C, single answer questions
637 - Final Review
© Jeff Offutt

17. Posting Grades I will post on Blackboard: Quiz 13 grades
Assignment 12 grades
Average quiz grades
Average assignment grades
Participation grades
Final exam score
637 - Final Review
© Jeff Offutt

18. Final Grades
15% participation, 20% assignments, 30% quizzes, 35% final exam
Unbiased grading – compute and sort:
>= 98 : A – Probably only one A+ per class
> 93 : A
> 90 : A-
> 86 : B+
> 80 : B
> 75 : B-
> 65 : C
I will “slide the line” down to an appropriate “break point”
637 - Final Review
© Jeff Offutt

19. What I Want You to Take Away
You in 5 years Me
Forgot everything about testing and high quality software
Remember testing, but develop software the “same old way”
Use these concepts to hand-design more effective tests for your software
Apply automation and criteria when possible
Convince your company to develop a quality mindset
637 - Final Review
© Jeff Offutt