Jeff Offutt SWE 637 Software Testing www.cs.gmu.edu/~offutt/ Review for Final – Fall 2018 Jeff Offutt SWE 637 Software Testing www.cs.gmu.edu/~offutt/

Parameters and logistics 4:30 – 7:15 Same room 17 December, next next Monday 10-December is a reading day No quiz retakes after 10-December Closed book / closed notes / closed neighbor One sheet of definitions allowed Hand-written, not typed, double-sided is okay Comprehensive You may take breaks (not all together) Spread out – separate tables and one table per person 637 - Final Review © Jeff Offutt

Summary: Part 1—Foundations Why Test Software? (Ch 1) Fault, error, failure, program state Test maturity levels Cost of late testing Model-Driven Test Design (Ch 2) RIPR model Other terms Test Automation (Ch 3) Junit Components of a test case Developer Testing (Ch 4) Agile and TDD Criteria-Based Test Design (Ch 5) Test requirements and criteria 637 - Final Review © Jeff Offutt

Cost of Late Testing 60 50 40 30 20 10 Assume $1000 unit cost, per fault, 100 faults $250K $360K Fault origin (%) Fault detection (%) Unit cost (X) $20K $100K $13K $6K 28-Oct-2010, at GTAC, added the animation to demonstrate increasing the number of faults found early, thereby decreasing the number of faults found late, and finally saving money. Lots of it! This animation is fairly complicated … must practice first!! Requirements Design Prog / Unit Test Integration Test System Test Post-Deployment Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96-HB-002 637 - Final Review © Jeff Offutt

Incorrect Program State RIPR Model Reachability Infection Propagation Revealability Test Final Program State Observed Final Program State Reaches Fault Incorrect Final State Infects Incorrect Program State Propagates The failures can be revealed when the observed final program state has overlap with the incorrect final program state. The question is: should testers check the entire program state? How to observe the incorrect program state in a cost-effective manner. Getting the overlap as big as possible and use the cost as small as possible. Reveals Test Oracles 637 - Final Review © Jeff Offutt

Model-Driven Test Design–Activities refined requirements / test specs Test Design model / structure test requirements DESIGN ABSTRACTION LEVEL IMPLEMENTATION ABSTRACTION LEVEL software artifact input values Test Automation Test Evaluation pass / fail test results Test Execution test scripts test cases 637 - Final Review © Jeff Offutt

Criteria Based on Structures Structures : Four ways to model software Input Domain Characterization (sets) A: {0, 1, >1} B: {600, 700, 800} C: {swe, cs, isa, infs} Graphs Logical Expressions (not X or not Y) and A and B if (x > y) z = x - y; else z = 2 * x; Syntactic Structures (grammars) 637 - Final Review © Jeff Offutt

Summary: Part 2 Coverage Criteria (1 of 4) Input Space Partitioning – Ch 6 Input Domain Modeling (6.1) Combination Strategies Criteria (6.2) Handling Constraints Among Characteristics (6.3) Extended Example: Deriving an IDM from JavaDoc (6.4) Be sure to review the coverage definitions in 6.2 Know how to construct partitions and blocks Know how to choose values 637 - Final Review © Jeff Offutt

ISP Coverage Criteria Subsumption Each Choice Coverage ECC All Combinations Coverage ACoC T-Wise Coverage TWC Multiple Base Choice Coverage MBCC Pair-Wise Coverage PWC Base Choice Coverage BCC 637 - Final Review © Jeff Offutt

Summary: Part 2 Coverage Criteria (2 of 4) Graph Coverage – Ch 7 Overview (7.1) Graph Coverage Criteria (7.2) Graph Coverage for Source Code (7.3) Graph Coverage for Design Elements (7.4) Graph Coverage for Specifications (7.5) Graph Coverage for Use Cases (7.6) Be sure to review the definitions in 7.2 Know how to generate graphs for various artifacts and apply the criteria to them Know how to choose values (EC, EPC, PPC, …) 637 - Final Review © Jeff Offutt

Graph Coverage Criteria Subsumption Simple Round Trip Coverage SRTC Node Coverage NC Edge Coverage EC Edge-Pair Coverage EPC Prime Path Coverage PPC Complete Path Coverage CPC Complete Round Trip Coverage CRTC All-DU-Paths Coverage ADUP All-uses Coverage AUC All-defs Coverage ADC 637 - Final Review © Jeff Offutt

Summary: Part 2 Coverage Criteria (3 of 4) Logic Coverage – Ch 8 Semantic Logic Coverage Criteria (Active) (8.1) Syntactic Logic Coverage Criteria (Inactive) (8.2) Structural Logic Coverage of Programs (8.3) Specification-based Logic Coverage (8.4) Logic Coverage of FSMs (8.5) Be sure to review the coverage definitions in 8.1 Know how to apply the coverage criteria to programs, specs, and FSMs Know how to choose values (CACC, …) 637 - Final Review © Jeff Offutt

Logic Criteria Subsumption Clause Coverage CC Predicate Coverage PC Combinatorial Clause Coverage COC Restricted Active Clause Coverage RACC Restricted Inactive Clause Coverage RICC General Active Clause Coverage GACC Correlated Active Clause Coverage CACC General Inactive Clause Coverage GICC 637 - Final Review © Jeff Offutt

Summary: Part 2 Coverage Criteria (4 of 4) Syntax-based Testing – Ch 9 Syntax-based Coverage Criteria (9.1) Program-based Grammars (9.2) Integration and Object-Oriented Testing (9.3) Concepts, not all the details Specification-based Grammars (09.4) Input Space Grammars (9.5) Understand how to derive tests from BNF Understand mutation analysis and mutation testing 637 - Final Review © Jeff Offutt

Representing Input Domains goal domain specified domain This region is a rich source of software errors … implemented domain 637 - Final Review © Jeff Offutt

Preparing & Format Re-read the text and handouts Study the homework assignments Redo the quizzes Try exercises in text Get a good night’s sleep Possibly different kinds of questions Mostly problem solving—similar to quizzes, perhaps bigger Short answer Maybe a few T / F, M / C, single answer questions 637 - Final Review © Jeff Offutt

Posting Grades I will post on Blackboard: Quiz 13 grades Assignment 12 grades Average quiz grades Average assignment grades Participation grades Final exam score 637 - Final Review © Jeff Offutt

Final Grades 15% participation, 20% assignments, 30% quizzes, 35% final exam Unbiased grading – compute and sort: >= 98 : A+ – Probably only one A+ per class > 93 : A > 90 : A- > 86 : B+ > 80 : B > 75 : B- > 65 : C I will “slide the line” down to an appropriate “break point” 637 - Final Review © Jeff Offutt

What I Want You to Take Away You in 5 years Me Forgot everything about testing and high quality software Remember testing, but develop software the “same old way” Use these concepts to hand-design more effective tests for your software Apply automation and criteria when possible Convince your company to develop a quality mindset 637 - Final Review © Jeff Offutt