Chapter 8 Testing the Programs 8.1 Software Faults and Failures 1. Introduction  faults: A: definition: the problem caused by error B: cause: X: the software itself Y: causes from customer(uncertain requirement, etc.)

Chapter 8 Testing the Programs  failure: correspondent A: definition: the software X the requirement does with describe B: example: (P332) C: cause of failure: (P332-5 dots ) several concepts and problems (about testing) A: purpose: discover faults, not demonstrate correctness B: fault identification: a process----(P332) C: fault correction: making changes to the system so the faults are removed

Chapter 8 Testing the Programs D: discuss about fault X: software faults is caused by human factors Y: “ bug ” E: purpose of this chapter (P333)

Chapter 8 Testing the Programs 2. Types of faults  reason for classifying faults (P333)  types of faults A: algorithmic fault X: definition (see text) Y: the ways to checking m: desk checking (static) n: give classified input identify faults B: computation and precision faults ----computation error, may result in less-than -acceptable precision

Chapter 8 Testing the Programs C: Documentation Faults ----the documentation does not match what the program actually does ( faults proliferation ) D: Stress or overload Faults ---- when data structures are filled past their specified capacity (Queues, buffers, tables, arrays) E: Capacity or Boundary Faults ---- when the system ’ s performance becomes unacceptable as a system activity reaches its specified limit

Chapter 8 Testing the Programs F: timing faults ---- when coordination of several processes or a carefully defined sequence is broken G: performance faults (example: response time ) H: recovery faults ---- system can ’ t recover from running failure I: hardware and system software faults ---- when the supplied hardware or software do not actually work according to the documented operating conditions and procedures (fail to work according to hardware or software introduction)

Chapter 8 Testing the Programs J: Standards and Procedure Faults ---- the code does not follow the organizational standards and procedures orthogonal defect classification (X) A: focus on: X: faults exists in anywhere in software developing Y: classification is helpful to reduce the number of faults B: definition: one fault belongs to one categories C: two type of defect X: fault of omission Y: fault of commission

Chapter 8 Testing the Programs D: example: fault classification of IBM/HP three descriptors: (siderbar8.1, fig8.1) X: origin Y: type Z: mode fig8.2: faults for one Hewlett-packard division

Chapter 8 Testing the Programs 8.2 Testing Issues 1. Test organization  several stages A: unit test: verifies the component functions (according to the program design ) B: integration test: verifies the system components work together(by system design and program design) C: function test: check function by D: performance test: check performance by E: acceptance test: check the customer’s requirement definition

Chapter 8 Testing the Programs F: installation test: check the system in actual environment  relationship among the testing steps A: relationship: fig8.3 B: this chapter: focus on unit and integration testing C: chapter 9: system test = function test+performance test+acceptance test+ installation test

Chapter 8 Testing the Programs 2. Attitudes toward testing  new programmer ----not accustomed to viewing testing as a discovery process  customer: ----interested in being sure that the system works properly under all conditions right attitude ---- egoless programming

Chapter 8 Testing the Programs 3. Who performs the tests ?  focus on: independent test team  reasons A: avoid conflict (between personal responsibility and the need to discover faults) B: there are many choices to introduce faults C: independent test team can participate in review 4. Views of the test objects  black box : A: the contents are unknown, the testing feed input to the black box and note what output is produced B: advantage: is free of the constraints imposed by

Chapter 8 Testing the Programs B: advantage: is free of the constraints imposed by the internal structure and logic of the the test object C: disadvantage: not always possible to run a complete test in this manner D: example: ax 2 +bx+c=0 a: +, -, 0 b: +, -, =27 (digital combination) c: +, -, 0 others: round-off error incompatible data type

Chapter 8 Testing the Programs E: another example: “ federal income tax ”  white box: A: definition: use the structure of the test object to test in different ways B: advantage: detailed testing for a model C: disadvantage: may be impractical D: example: white box --- fig8.4 I: n. 3X3=9(test cases) J: m.

Chapter 8 Testing the Programs several factors for choice of test case A: the number of possible logical paths B: the nature of the input data C: the amount of computation involved D: the complexity of the algorithms

Chapter 8 Testing the Programs 8.3 Unit Testing steps: A: examine the code B: compile the code C: develop test cases 1. Examining the Code  code review: (by an objective group of experts, review the code and its document for misunderstanding, inconsistencies, and other faults)  two types of code review A: code walkthrough (informal)

Chapter 8 Testing the Programs B: code inspection (formal review) ----the review team check code and document according a prepared list of concerns example — (P343) review data structure, algorithms, comments, interfaces, performance, etc. steps: meet checking meeting success of code reviews A: success in detecting faults B: early finding faults (in fact, review is for all processes)

Chapter 8 Testing the Programs  statistics in review ----Fagan’s result (P344 ) ----table 8.2 typical preparation times and meeting times ----table 8.3 Jone’s result: fault discovery rates (rate=number of faults / number of thousands of lines of code) 2. Testing Program Components  choosing test cases A: test case: input data choosing for testing a program

Chapter 8 Testing the Programs B: the methods of giving test case X: black box: according “ SRS ” Y: white box: according internal logic Z: example: aX 2 +bX+c=0 black box: first method: a: +, -, 0 b, +, -, =27(test cases) c, +, -, 0 second method: a>b>c b>c>a 3(test cases) c>b>a white box: b 2 -4ac: >0, <0, =0. 3(test cases)

Chapter 8 Testing the Programs other method: a,b,c= “ F ” (nonnumeric data) W: criteria for classify the test data (black box) ----(P352 — 1,2,3) U: drawbacks of black box and white box black box: uncertainty white box: pay too much effort but still difficult C: combining black-and white-box testing to generate test data X: method: black box + white box + other method Y: example: black box---- “ input positive value ” (P352 — 7 dots)

Chapter 8 Testing the Programs  test thoroughness (white box) A: three methods: X: statement (coverage) testing Y: branch testing Z: path testing B: example—Fig8.7 statement: x>k result> (1 test case) branch: and (2 test cases) path: paths---(4 test cases)

Chapter 8 Testing the Programs 8.4 Integration Testing A: working system: B: focus on: importance of the way of combining the components X: influence to the order of combining component or coding Y: influence to the choice of test cases Z: influence to the cost and thoroughness of the testing

Chapter 8 Testing the Programs 1. Bottom-Up Integration  meaning: (P356)  example: A: component driver B: fig-8.8, fig-8.9 bottom-up testing procedure: d E dd FG ddd F E B C D G A BC EF D G

Chapter 8 Testing the Programs advantage: A: easy to generate a test case B: be suitable for OO approach  drawback: the major faults in high level units can’t correct as soon as possible

Chapter 8 Testing the Programs 2. Top-Down Integration  meaning: (P358)  example: A: stub B: fig-8.8, fig-8.10 top-down testing procedure: s A ss A s s B C D s A BC EF D G

Chapter 8 Testing the Programs advantage: the upper models have more testing choice, so the major question will be found in early stage  drawback: A: generating test cases can be difficult (P358) B: a very large number of stubs may be required (modified top-down testing—Fig8.11) 3. Big-bang Integration  meaning:unit testing integrate all units in one time  drawback: A: stubs+drivers

Chapter 8 Testing the Programs B: its difficult to find the cause of any failure C: interface faults cannot be distinguished easily 4. Sandwich Integration  meaning: A: three layers: X: target layer Y: upper layer Z: lower layer B: strategy X: upper layer — top-down testing Y: lower layer --- bottom-up testing Z: target layer --- testing(stub+driver) W: integration

Chapter 8 Testing the Programs C: example: Fig8.13  feature: A: full testing for top level B: easy to generate test case for lower level 5. Comparison of Integration Strategies  focus on: (P360)  table8.7—comparison of integration strategies sidebar8.5 + Fig8.15

Chapter 8 Testing the Programs 8.5 Testing Object-Oriented Systems Focus on: OO testing has special characteristics ----should take several additional steps 1. Testing the Code  several problem about OO code testing (P dots)  several aspects about the faults of objects (P dots) message : making special testing 2. Difference between OO and traditional Testing  focus on: retest all methods when we add new subclass in a way of inheritance

Chapter 8 Testing the Programs  difference between OO and traditional testing A: traditional testing system change original test case+new test case (ordinary testing) B: OO testing: retest the overriding subclass, and may use different use cases unit testing ---- easy integration testing ---- extensive Fig show the easier and harder parts C: difference (with traditional testing) Fig four different aspects difficulty in OO testing ( P dots )

Chapter 8 Testing the Programs 8.6 Testing Planning  significance: (P366) help careful test planning design and organize the tests us appropriately  several steps about test plan (P366— “1-6” ) 1. Purpose of Plan  purpose: realize test objectives by a test strategy  test plan: who, why, how, when/schedule demanding: A: know modular hierarchy B: choosing test objectives, define test strategy, generate test cases (to wait the testing executing )

Chapter 8 Testing the Programs 2. Contents of the Plan  test objective and designing test cases A: objective and steps B: choosing representative test cases ( classify test cases)  methods and techniques ----integration methods + review methods + all kinds of reports detailed list of test cases 3. Final Purpose of Test Plan (P367) ----having a complete picture of how and why testing will be performed

Chapter 8 Testing the Programs