1. Software Testing Testing Process

2. Agenda Determining the test methodology Planning the tests 2

3. The Testing Process 3 Planning and Design for testing is done in all phases of software development. Determine the test methodology Planning the tests Designing the tests Performing the tests

4. Test Methodology Determination Main issues to handle before deciding the methodology  What Quality Standard is needed?  Which testing strategy should be followed? Quality Standard  Depends mainly on the characteristics of the project  Examples  Hospital Patient Bed Monitor Software  Software to handle feedback  Criteria for selecting the right standard  Nature of the issue  Magnitude (severity) of the damages expected in case of failure 4

5. Software Failure Damage Types Damages to the users 1.Danger to human life 2.Affect organizations functions 3.Affects functioning of the main component 4.Affects essential functions (but replacement is available) 5.Affects functioning of software packages 6.Affects functioning of a main component but does not affect the entire system 7.Inconvenience to the user Damages to the developer  Financial Loss  Non-quantitative damages 5

6. Testing Strategy Issues include  Type of testing to use  Big Bang or Incremental  If Incremental  Top-Down  Bottom-Up  Which part of the testing plan should use White-Box Testing  Which part of the testing plan should use Automated Testing 6

7. Planning the Tests Tests to be planned  Unit tests  Integration tests  System tests Issues to consider before test plan  What to Test?  Which sources to use for test cases?  Who performs the tests?  Where to perform the tests?  When to terminate the tests? 7

8. What to Test? Straightforward Plan  Unit test for all individual units (or modules)  Integration tests for all integrations of the units  One complete test for the system Issues with straightforward Plan  Should we unit test all modules – even if they are reused?  Should we test simple units / modules? Factors to decide  Which modules to be unit tested?  Which integration to be tested?  Assign priority to the test cases. 8

9. Rating for Priority - Factors Two main factors  Damage Severity Level  See the damage types discussed before  Software Risk Level  Module / Application Issues  Magnitude  Complexity  Percentage of Original vs. Reused Software  Programmer Issues  Qualification  Experience  Availability  Self-Evaluation 9

10. Rating for Priority - Formula 10 C = A + B C = (k x A) + (m x B) C = A x B C is the combined priority A is the severity of damage B is the risk severity k and m are constants

11. Sources for Test Cases Two main sources  Real-life test cases  Synthetic test cases Issues  Use one or both sources  How many from each  Characteristics of test cases More on this in the next lecture 11

12. Who performs the Tests? Determined by the planning stage  Integration and unit tests are performed by the development team  System test performed by an independent testing team (internal or external)  More than one testing team for a large project If the organization is small (no testing team available)  Test by another development team  Outsourcing 12

13. Where to perform the Tests? Unit and Integration Test  Development Site System Test Location Choices  Developer Site  Customer Site  Consultant Site If performed on the developer site or consultant site – simulate the environment as on the customer site (for acceptance) 13

14. When to stop testing? Five Routes (Paths) Available Route A: Completed Implementation  Entire test plan is carried out  Perfection Approach  Disregard the budget and timeline Route B: Mathematical Models Application  Estimate the % of undetected errors (rate of error detection)  Stop when we go below the rate  Model may not represent the real-world situation 14

15. When to stop testing? Route C: Error Seeding  Errors are hidden inside the tested software  Stop when % of undetected seeded errors reach a predefined level  Additional work to add / hide the errors  Based on past experience Route D: Dual Independent Testing Team  Two teams implement the testing process independently  Work Assignment  Read and complete the problem on page 226-227 of the Book (Galin) Route E: Resource Exhaustion  Terminate when budget is over or time is finished 15