1. 16 October 2013

2. Reminder

3. Types of Testing: Purpose  Functional testing  Usability testing  Conformance testing  Performance testing  Acceptance testing  Reliability testing ……

4. Other classifications  Scope Unit, component, system, regression, …  Access to code Black box vs. white box (Note that black box testing still assumes knowledge of coding and development in general)

5. What are you trying to test?  Most common actions?  Most likely problem areas?  Risk-based testing

6. Risks  Identify criteria of concern: availability, quality, performance, …  Risk of it not being met likelihood consequences  If I’m testing code for a grocery store, what is the impact of the code not being highly available?

7. Four Parts of Testing  Model  Select test cases  Execute test cases  Measure

8. Questions to Ask for Each Test  How will this test find a defect?  What kind of defect?  How powerful is this test against that type of defect? Are there more powerful ones?

9. Basic Software Model capabilities environment User interfaces APIs Operating system Files Input Output Storage Processing

10. Test Case Selection  Environments What happens if a file changes out from under you? Consider all error cases from system calls ○ (e.g., you can’t get memory) Test on different platforms: software and hardware Test on different versions and with different languages  Capabilities Inputs (boundary conditions, equivalence classes) Outputs (can I generate a bad output?) States (reverse state exploration) Processing

11. From the User Interface: Inputs  Error messages  Default values  Character sets and data types  Overflow input buffers  Input interactions  Repeated inputs  How easy is it to find bugs in Word?

12. Test Coverage Metrics  Statement coverage basic block coverage  Decision coverage Each Boolean expression  Condition coverage Each entity in Boolean expressions  Path coverage loops  Advantages of different models?

13. Unit Test  What do they do? Regression testing framework Incrementally build test suites Typically language specific  What is available? JUnit most well known SUnit was the first (Smalltalk) xxUnit where xx represents every language known to CS ○ (even Wikipedia lists 75 languages)

14. How important is unit test?  91: Patriot Missile incoming missile missed rounding error  93: Intel Pentium chip incorrect calculations table too small  96: Ariane V video video auto-destruct data conversion – 64 bit float to 16 bit integer  98: Mars Climate Orbiter crashes into Mars imperial units instead of metric

15. Regression Test  Automated  Run with every build  Issues Time Many are GUI based ( sample) sample  Tools Wide range of capabilities and quality Avoid pixel-based tools Random vs. scripted testing  Informal version: rerun tests