KEY IDEA Assure that your testing fits the logistics of the project.

Dynamic Quality Paradigm It’s more important to work on Item B. Perfect Awful unnecessary quality unacceptable quality Item A Further improvement would not be a good use of resources. floating line Good enough quality bar Item B Further improvement is necessary.

A Heuristic for Good Enough All conditions must apply. 1. X has sufficient benefits. 2. X has no critical problems. 3. Benefits of X sufficiently outweigh problems. 4. In the present situation, and all things considered, improving X would be more harmful than helpful. Problems Benefits

Good Enough... …with what level of confidence? …to meet ethical obligations? …in what time frame? …compared to what? …for what purpose? …or else what? …for whom? Perspective is Everything

Test Project Dynamics: Context Model

Test Project Dynamics: Context Model

Test Project Dynamics: Context Model

Test Project Dynamics: Givens vs. Choices Motivation Capability CHOI Motivation: What testing does the situation require? Capability: Can we perform that testing in this situation?

Tailoring the Test Process Are GIVENS good enough? Do CHOICES about process exploit the GIVENS and address the MISSION well enough? Is MISSION achieved well enough? How do you know?

MISSION: The most important part Find important problems Assess quality Certify to standard Fulfill process mandates Satisfy stakeholders Assure accountability Advise about QA Advise about testing Advise about quality Maximize efficiency Minimize time Minimize cost The quality of testing depends on which of these possible missions matter and how they relate. Many debates about the goodness of testing are really debates over missions and givens.

Slow down project to improve testability Driving analogy When on the freeway, go 65mph. When on a suburban artery, go 40mph. When in a neighborhood, go 25mph. When in a parking lot, go 5mph. Park very slowly. Change Control Funnel The closer you are to shipping, the more reluctantly you should change code. To minimize retesting, carefully review proposed changes. “You can change what you like, but change may invalidate everything we thought we knew about quality.” More risks, lower speeds. time

“Why didn’t you find that bug?” Highway Patrol analogy: Do you realize how hard it would be to patrol the highways so that all speeders are always caught? Risk-based improvement argument: Our goal is to find important problems fast. Was that bug important? Could we economically have anticipated it? If so, we’ll modify our approach. Testability reframe: We didn’t find it because the developers didn’t make the bug obvious enough for us to notice. The developers should put better bugs into the code. Let’s make a more testable product.

Exercise We may want to purchase DiskMapper. Analyze it and tell us what the testing issues might be.