1 Principles of testing TESTING BASICS: Basic principles of testing: Testing must be:  thorough  ongoing  developed with design  efficient most effective--independent testing exhaustive test--usually not possible

2 Design for testability (DFT) design for testability (DFT)--what makes software "testable"?  operability: few bugs, incremental test  observability: you can see the results of the test  controllability: you can control state + input to test  decomposability: you can decompose into smaller problems and test each separately  simplicity: you choose the “simplest solution that will work”  stability: same test will give same results each time  understandability: you understand code, inputs, and outputs

3 Types of testing Types of testing:  white box--"internals” (also called "glass box")  black box--"interfaces” (also called "behavioral")  system--”functionality” (can be based on specs, use cases)  application-specific-- GUIs Client/Server Real-time Documentation/help

4 Testing strategies testing strategies: verification--functions correctly implemented validation--we are implementing the correct functions (according to requirements)

5 Spiral design/testing strategy A general design/testing strategy can be described as a "spiral”: requirements  design  code system test  integ. test  unit test (system) (black (white box) box) when is testing complete? One model: "logarithmic Poisson model” f(t)=(1/p)ln(I 0 pt+1) f(t) = cumulative expected failures at time t I 0 = failures per time unit at beginning of testing p = reduction rate in failure intensity START END Requirements/System Tests Design/Integration Tests Code/Unit Tests

6 OO testing strategy OO testing: emphasis is on interfaces use UML tools to support testing strategies and development of test cases --system tests: use cases; quality measurements --black box tests: ER diagrams, object message diagrams, dataflow and state diagrams --white box tests: class and state diagrams, CRC cards

7 Good, Bad, and Successful Tests good test: has a high probability of finding an error ("bad test": not likely to find anything new) successful test: finds a new error most effective testing: by an "independent” third party

8 Black box testing--example Examples: CarGasStation: station P company employee Denotes link that leads to one or more test cases

9 Black box testing guidelines General guidelines: test BOUNDARIES test output also choose "orthogonal” cases if possible

10 Using specifications for system tests System tests should verify that specifications have been met For UML-based strategy: each use case ---> one or more system tests each quality / performance requirement  one or more system tests Additional qualitative or quantitative tests (not from use cases): examples: is system “user-friendly”? are timing requirements met? are available resources sufficient?

11 Using specifications for system tests Example: 1. Place call 2. Receive call 3. Use scheduler Cellular network User Associated sequence diagrams Associated test cases use cases Tests verify use case supported