Software Engineering Modern Approaches Eric Braude and Michael Bernstein 1

© 2010 John Wiley & Sons Ltd. Chapter 25: Introduction to Software Testing 2

Learning Goals of This Chapter Why test early? Why often? When and why do you retest? What is the difference between black box and white box testing? What distinguishes unit testing? How do you document tests? How do you plan for testing? Can you verify a set of tests? How do you know when to stop testing? Where does test input come from? Who is involved in testing? How much effort does it take to test? Requirements analysis Design Implementation Testing Maintenance Planning The Software Development Lifecycle Phase most relevant to this chapter is shown in bold 3

Regression Testing Original code artifact Original code artifact addition Original test 1. Regression test (= original test?) 2. Test of added functionality © 2010 John Wiley & Sons Ltd. 4

Suppose that C is a body of already-tested code in an application A. Suppose that A has been altered with new or changed code N.  If C is known to depend on N Perform regression testing on C  If C is reliably known to be completely independent of N There is no need to regression test C  Otherwise Regression test C Regression Testing: What to Retest A snjfgkjfdkgjkfdjgkf djkgjdfkbvjfdkjbkfd jbkfdjbkjdfklbjdflkbj kldfjbkfdjblkjfdklbjf dklbjklfdjbkldfjbkljd fkbljfdklbjoikhkgkjk DHVDABFDSGKJ GFJTIURjskjgls,al 40rgnsj hhjk ghjhgj ghjkklj;ljko fghjhjgfj g;b,kdfgjfjfjg kfdjk N dfkbljfdklHVDABF DS DHVDABFDSlkgt GKJGIURjskjgl C 5

Black- & White-box Testing Black box … requirements Actual output compared with required output White box Input determined by... Result …design elements Validation of expected behavior © 2010 John Wiley & Sons Ltd. 6

An Assembly’s Soundness Depends on its Parts Reliable? Reliable part? Reliable? © 2010 John Wiley & Sons Ltd. 7

Types of Post-Unit Testing Interface testing Validates functions exposed by modules Integration … … combinations of modules System whole application Usability user satisfaction Regression changes did not create defects in existing code Acceptance customer agreement that contract satisfied Installation works as specified once installed on required platform  Robustness ability to handle anomalies  Performance fast enough; uses acceptable amount of memory 8

Post-Unit Testing Unit ….. Unit ….. Module Application on test bed Application on final platform Build ….. integration testing Interface testing system testing usability testing installation testing acceptance testing Regression testing throughout © 2010 John Wiley & Sons Ltd. 9

Plan for Testing 1. Define “units” vs. non-units for testing 2. Determine what types of testing will be performed 3. Determine extent – do not just “test until time expires” – prioritize, so that important tests definitely performed 4. Document – individual’s personal document set included? – how / when to incorporate all types of testing? – how / when to incorporate in formal documents? – how / when to use tools / test utilities? 5. Determine input sources 6. Decide who will test – individual engineer responsible for some (units)? – how / when inspected by QA? – how / when designed & performed by third parties? 7. Estimate resources – use historical data if available 8. Identify metrics to be collected – Define, gather, use – e.g., time, defect count, type, and source 10

© 2010 John Wiley & Sons Ltd. Stopping Criteria for Testing When tester has not been able to find another defect in 5 (10? 30? 100?) minutes of testing When all nominal, boundary and out-of-bounds test examples show no defect When a given checklist of test types has been completed After completing a series of targeted coverage (e.g., branch coverage for unit testing) When testing runs out of its scheduled time 11