1. March 16, 2004

2. Calendar Next week: Thursday meeting instead of Tuesday (May 25) Web update later today

3. Semester Plan Finish up key topics Testing Maintenance Security and privacy Industry environment Case studies (guest speakers) But to keep you coming, start each class with discussion of deliverables and what I’m looking for

4. Other info I owe you Presentation grades are sitting on my dining room table I’ll email them tonight Project grading “algorithm” Coming soon

5. Exam Reminder 75% of 10% = 7.5% 75 points: 1 point =.1% Will be glad to talk with anyone about the test Brief answers will be posted (linked from exam) Mean 58, median 61

6. Exam Discussion Lessons that I learned Reminders about test taking Quick discussion of topics

7. Software Quality and Testing

8. Why do we care? Therac-25 (1985) Multiple space fiascos (1990s) Millenium bug (2000) Microsoft attacks (2003) NIST: cost to US, $59 billion

9. Quality and testing “Errors should be found and fixed as close to their place of origin as possible.” Fagan “Trying to improve quality by increasing testing is like trying to lose weight by weighing yourself more often.” McConnell

10. Life Testing Used regularly in hardware Addresses “normal use” n specimens put to test Test until r failures have been observed Choose n and r to obtain the desired statistical errors As r and n increase, statistical errors decrease Expected time in test = mu 0 (r / n) Where mu 0 = mean failure time

11. Butler and Finelli “The Infeasibility of Experimental Quantification of Life-Critical Software Reliability” In order to establish that the probability of failure of software is less than 10 -9 in 10 hours, testing required with one computer is greater than 1 million years

12. Tools for Improving Quality Formal specification Self-checking (paranoid) code Program verification and validation Testing Deploy with capabilities to repair

13. Types of Testing: Purpose Conformance testing Usability testing Performance testing Acceptance testing Reliability testing …

14. Other classifications Scope Unit, component, system, regression, … Access to code Black box vs. white box (Note that black box testing still assumes knowledge of coding and development in general)

15. What are you trying to test? Most common actions? Most likely problem areas? Risk-based testing

16. Risks Identify criteria of concern: availability, quality, performance, … Risk of it not being met likelihood consequences If I’m testing code for a grocery store, what is the impact of the code not being highly available?

17. Risk Heuristics (just a few) New features New technology Overworked developers Regression Dependencies Complexity Bug history Language specific bugs Environment changes Late changes Slipped in “pet” features Ambiguity Changing requirements Bad publicity Liability Learning curve Criticality Popularity

18. Who should test code? Unit test is always done by developer Different views on system test Build to test Test-driven development Completely independent Acceptance testing Middle-of-the-road

19. Four Parts of Testing Model Select test cases Execute test cases Measure

20. Model

21. Basic Software Model capabilities environment User interfaces APIs Operating system Files Input Output Storage Processing

22. References Therac-25: http://courses.cs.vt.edu/~cs3604/lib/Therac_ 25/Therac_1.html http://courses.cs.vt.edu/~cs3604/lib/Therac_ 25/Therac_1.html Patriot missile: http://www.fas.org/spp/starwars/gao/im9202 6.htm Ariane 5: http://www.esa.int/export/esaCP/Pr_33_1996 _p_EN.html http://www.esa.int/export/esaCP/Pr_33_1996 _p_EN.html NIST: http://www.nist.gov/director/prog- ofc/report02-3.pdf