Download presentation
Presentation is loading. Please wait.
Published byQuentin Brooks Modified over 9 years ago
1
1 Testing Course notes for CEN 4010
2
2 Outline Introduction: terminology and philosophy Factors that influence testing Testing techniques
3
3 Why Do We Test? We test to find bugs “Testing is the process of executing a program with the intent of finding errors” A good test intends to find bugs “A good test case is one that has a high probability of detecting an as-yet undiscovered error” A test that finds no bug is a failure “A successful test case is one that detects an as-yet undiscovered error”
4
4 Testing vs. Debugging Testing != Debugging While testing may reveal a symptom of an error, it may not uncover the exact cause of the error Debugging is the process of locating the exact cause of an error, and removing that cause
5
5 Our Very Real Problem We can never say for an arbitrary program: This software has no errors This software works only as intended This software is safe Testing proves the presence, not the absence, of bugs Absence of evidence is not evidence of absence
6
6 4 dimensions of the Testing Outputs (i.e. Results) Inputs ValidInvalid Valid Invalid Good Goal (includes exceptions) BUG!!! Should always produce exceptions!
7
7 Where Does Testing Fit?
8
8 When to test ?
9
9 What other Tests Are There?
10
10 Test activities: Analysis Test the Models against the Problem specification “Play computer” through the models Analysis models must accurately describe the problem to be solved, and the boundaries of the problem domain
11
11 Test activities: Design Test the Models against the Solution domain. “Play computer” through the models Incorporating all required “design” classes GUI frameworks Third-party software and class libraries Collection/Container classes Operating environment classes (wrappers) Interface classes to external resources. E.g., RDBMS, network, communications.
12
12 Test Metrics Traditional Metrics: No. of bugs per 10,000 Source Lines of Code IEEE Standard 982.1 McCabe Complexity Measure Halstead Software Science Measures …
13
13 Test Techniques: The “Best of all Possible Worlds” technique: Run it...and make sure it doesn’t crash, then ship it! For the rest of us: Desk Checking Inspections &Walkthroughs Black-box/White-box testing Boundary testing Path testing Code coverage Error message testing Top-down & Bottom-up …
14
14 Desk Checking Simple, labor-intensive Procedure Developer has his/her design or code manually reviewed by another developer “a second pair of eyes” Developer must justify decisions to reviewer. Developer reviews/accepts/rejects reviewer’s recommendations.
15
15 Inspections & Walkthroughs Formal or informal as needed. Well-known technique. Procedure: Group gather to review artifacts (docs to code). Group composed of “role” representatives (e.g. “user”, “QA”, “Standards”, etc.). Rules of conduct apply to participants Responsibility for verification and validation of artifact is removed from the developer.
16
16 Inspections Author narrates, statement by statement, the logic of his program. Reviewers listen to the author, raise questions and ask for clarification in their attempt to uncover errors. Ironically, most of the errors are discovered by the author as he/she “teaches” the program to the group.
17
17 Walkthroughs Programmer does not narrate to the group. One of the group plays the role of “tester”. Tester comes to review armed with a small set of simple test cases to apply to the program. Group “play computer” by performing manual simulation of the system using the test data. Data are important as a vehicle to stimulate discussion. Most errors are uncovered by questioning of the author rather than “execution” of data.
18
18 Inspections & Walkthroughs When should I perform one? Anytime, so long as the product under review is complete (i.e. tangible, understandable, and objective). After each milestone with delivered artifacts (e.g. Requirements Spec., Analysis Models, Design Spec., pseudo- code or flowcharts prepared, program code written, etc.). Yourdon’s milestones [YOUR79]: after the design artifacts are completed after the code is prepared but not compiled after the first compilation after the first “clean” compilation after the first test data set have been executed successfully after the programmer thinks all test cases have been executed successfully
19
19 Inspections & Walkthroughs Who should participate? Author/developer/presenter/producer Moderator/coordinator Scribe/secretary Standards Bearer Maintenance Expert User Representative GUI Expert
20
20 White-Box Testing White-box testing is the testing of the underlying implementation of a piece of software (e.g., source code) without regard to the specification (external description) for that piece of software. The goal of white-box testing of source code is to identify such items as (unintentional) infinite loops, paths through the code which should be allowed, but which cannot be executed, and dead (unreachable) code.
21
21 Black-Box Testing Black-box testing is the testing of a piece of software without regard to its underlying implementation. Specifically, it dictates that test cases for a piece of software are to be generated based solely on an examination of the specification (external description) for that piece of software. The goal of black-box testing is to demonstrate that the software being tested does not adhere to its external specification. (Note that if there is no "external specification" it will be difficult to conduct black-box testing.)
22
22 Test Categories Black Box Tests Boundary-value Analysis Equivalence Partitioning Cause-Effect Graphing Error Guessing Error Message Generation from [MYER79] White Box Tests Statement Coverage Decision/Condition Coverage Multiple-Condition Coverage
23
23 Black Box Tests Boundary-value Analysis Uses test cases “generated on, and immediately around, the boundaries of the input an output for a given piece of software.” Equivalence Partitioning Divide test sets into equivalence partitions: “collections of items which can all be regarded as identical at a given level of abstraction. I.e., a set of data items which will all evoke the same general behavior from a given software module.” Cause-Effect Graphing Generates test sets based on input combinations, specified by a combinatorial logic network or Boolean relations, and the expected outputs from those input combinations.
24
24 Black Box Tests Error Guessing An intuitive and empirical technique of selecting test cases which elicit failures. A.k.a. “a gift”. Error Message Generation Selecting test cases which will elicit the error messages defined in the software under test.
25
25 White Box Tests Statement Coverage a.k.a. “basis path testing” Use test cases which cause each executable statement to be executed at least once. Decision/Condition Coverage All statements are executed at least once, and all binary decisions have a TRUE and FALSE outcome at least once, all exceptions are raised at least once, and all possible interrupts are forced to occur at least once.
26
26 White Box Tests Multiple-Condition Coverage Use test cases such that all possible combinations of condition outcomes in each decision, and all points of entry, are invoked at least once.
27
27 Path Testing tests all possible paths the flowchart on the right has approximately 100 trillion possible paths at 1ms/test, working 24 hrs/day, 365 days/year: it will take 3170 years
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.