1. that focus first on areas of risk.
KEY IDEA
Run crisp test cycles
that focus first on areas of risk.

2. Test Cycle: What You Do With a Build
1. Receive the product.
Formal builds
Informal builds
Save old builds.
2. Clean your system.
Completely uninstall earlier builds.
3. Verify testability.
Smoke testing
Suspend test cycle if the product is untestable.

3. Test Cycle: What You Do With a Build
4. Determine what is new or changed.
Change log
5. Determine what has been fixed.
Bug tracking system
6. Test fixes.
Many fixes fail!
Also test nearby functionality.
7. Test new or changed areas.
Exploratory testing.

4. Test Cycle: What You Do With a Build
8. Perform regression testing.
Not performed for an incremental cycle.
Automated vs. manual
Important tests first!
9. Report results.
Coverage
Observations
Bug status (new, existing, reopened, closed)
Assessment of quality
Assessment of testability

5. Typical Quality Growth
The Test Cycle
Ship it!
Typical Quality Growth
Quality
Time

6. Risk Focus: Common and Critical Cases
Core functions: the critical and the popular.
Capabilities: can the functions work at all?
Common situations: popular data and pathways.
Common threats: likely stress and error situations.
User impact: failures that would do a lot of damage.
Most wanted: problems of special interest to someone else on the team.

7. Test Cycle Convergence
Ideal Test Cycle Convergence
Good Enough Quality
Tests failed: known bad
Cycle End
Cycle Start
Tests not run:
Unknown }
Quality
Build
Tests passed: known good
Untestable Quality
Time

8. Rapid Bug Investigation
Identification
Notice a problem.
Recall what you were doing just prior to the problem.
Examine symptoms of the problem w/o disturbing system state.
Consider possibility of tester error.
Investigation
How can the problem be reproduced?
What are the symptoms of the problem?
How severe could the problem be?
What might be causing the problem?
Reality Check
Do we know enough about the problem to report it?
Is it important to investigate this problem right now?
Is this problem, or any variant of it, already known?
How do we know this is really a problem?
Is there someone else who can help us?
Identify
Investigate
Check

9. Exercise
“I’ve modified the Triangle program so that it now renders an image of the triangle. I’m also worried about the limitations of the input field. Please do another test cycle on it.”
-- Your Friendly Programmer

10. Provide a clear report on how completely you tested Triangle
Exercise
Provide a clear report on how completely you tested Triangle

11. An Innocent Question from Management…
“On a scale of 0 to 100, how completely did you test the Triangle program?”
completely untested
completely tested 50
100
Answer with a number.
You can also offer a comment, if you have one.
Any questions?

12. What Could We Report?
Tasks: Things we are doing. Example: “We are trying to reinstall the server so we can do the smoke tests. We have 28 fix verifications to get through.”
Product Coverage: What we have examined. Example: “We have tested printing, performance, and compatibility.”
Product Risk: Problems or potential for problems in the product. Example: “We have found 53 bugs.”
Agreement: What we specifically contracted to do. Example: “We have implemented 90% of the tests on our test plan. As we agreed, we have started testing for printer compatibility.”

13. What Could We Report?
Project: Schedule, documents, resources, people, or anything else that makes it possible for us to test. Example: “Testing is on schedule. But Jaeline goes on vacation next week.”
Mission: The ultimate goal, which may be to ship a product that meets quality criteria, satisfy customers, or some other goal that can be a guiding principle for the testing. Example: “We don’t yet know enough about feature X of the product.”
Client Satisfaction: What our clients think of our work. Example: “The project manager is happy with our work.”

14. Reporting Considerations
Reporter safety: What will they think if I made no progress?
Client: Who am I reporting to and how do I relate to them?
Rules: What rules and traditions are there for reporting, here?
Significance of report: How will my report influence events?
Subject of report: On what am I reporting?
Other agents reporting: How do other reports affect mine?
Medium: How will my report be seen, heard, and touched?
Precision and confidence levels: What distinctions make a difference?
Take responsibility for the communication.