1. March 25, 2004

2. Announcements Friday – recruiting event 12-6 Dean Smith Center Bloomberg LP, New York City full-time positions and summer internships programmers C/C++ and UNIX New section of 190 next fall: Logic and Automated Theorem Proving Prof. Plaisted

3. Final Presentations Logistics 11 presentations @ 20 minutes Need 4 hours Exam scheduled for noon on April 27 Propose that we run from 11:30 – 3:30 Problems?

4. Reminder: Data to Collect Your expertise level for (3 point scale) Language and platform Technology Time spent Education Meetings Code and test Documentation

5. U.S. Students Shun Computer Sciences San Jose Mercury News (03/24/04) CRA’s Taulbee Survey U.S. undergraduate enrollment in CS fields declined by 19 percent in 02-03 newly declared CS majors dropped 23 percent concern about offshore outsourcing lowered career expectations "most of the solutions to the complex problems that companies need to solve haven't been written yet… that's what we should be preparing our students to do"

6. Four Parts of Testing Model Select test cases Execute test cases Measure

7. Model

8. Basic Software Model capabilities environment User interfaces APIs Operating system Files Input Output Storage Processing

9. Test Case Selection

10. From the User Interface: Inputs Error messages Default values Character sets and data types Overflow input buffers Input interactions Repeated inputs

11. From the User Interface: Outputs Concept: What inputs create interesting outputs? REQUIRES DOMAIN EXPERTISE State-based outputs Invalid outputs Changeable outputs Screen refreshes

12. Bubble diagrams: Reverse state exploration Define a failure state What would have happened to get you there? Repeat Find a way to force it down that path Let’s try it: remote access keys

13. Capabilities – Storage and Processing Same input, different initial conditions Too many or too few values in a data structure Alternative ways to modify constraints Invalid operator/operand combinations Recursive functions Overflow/underflow computations Feature interaction

14. How to select specific cases Data based Boundary conditions Equivalence classes Control based State transitions

15. Let’s try it Word PowerPoint calculator

16. Execution

17. Execution tools Automation – often primarily scripts Critical for regression testing GUI tools are abundant, but marginal

18. Measurement

19. Test Coverage Metrics Statement coverage basic block coverage Decision coverage Each Boolean expression Condition coverage Each entity in Boolean expressions Path coverage loops Advantages of different models?

20. Estimating how many bugs are left Historical data Capture-recapture model from biology

21. References Whittaker, How to Break Software Kaner, The Impossibility of Complete Testing at www.kaner.com (Articles)www.kaner.com

22. Your Test Plan and Testing Unit System User