CHAPTER 9 - PART 1 Software Testing Strategies

Lesson Outlines Definitions and objectives Software testing strategies Software test classifications White box testing Data processing and calculation correctness tests Correctness tests and path coverage Correctness tests and line coverage McCabe’s cyclomatic complexity metrics Software qualification and reusability testing Advantages and disadvantages of white box testing Black box testing Equivalence classes for output correctness tests Other operation factor testing classes Revision factor testing classes Transition factor testing classes Advantages and disadvantages of black box testing

Introduction In the past, testing (other than some unit testing) was undertaken as the last part of development after the entire app was developed and documented. Later, when discovered problems were discovered too late, unit testing and integration testing became in vogue. Later, the notion of testing has been expanded significantly to include documentation, and hosts of other related software quality assurance activities. We will concentrate on testing as accommodated by running the code.

Introduction As of 1994, some 24% of project development budget allocated to testing; 32% of project management budget slated for testing. Major emphases on increased testing has met resistance and project managers had to reduce. –We cannot test everything!!! We often ‘test’ documentation via standard SQA. But this does not test the functionality of the code. Will concentrate on testing classifications and strategies and classification of requirements types

Introduction Need to be able to: –Explain testing objectives –Discuss the differences between the various testing strategies: advantages and disadvantages –Describe the concepts of both black box and white box testing and their advantages and disadvantages –Discuss path coverage versus line coverage –Describe the various types of black box tests. –Testing has become a huge business and major emphasis in software engineering ! Testing has been described as the process of executing a program with the intention of finding errors.

Software Tests - Definition ■ Software testing is a formal process carried out by a specialized testing team in which a software unit, several integrated software units or an entire software package are examined by running the programs on a computer. All the associated tests are performed according to approved test procedures on approved test cases.

Software Tests - Definition Formal: scheduled in advance; central in the development process; not ad hoc. Specialized Testing Team: independent test team or external consultancy. Unit tests by developers – okay, but that is it. Running the program – a must; nothing static. Approved Test Procedures: use a plan approved. Approved Test Cases: All planned; all tests undertaken.

Software Testing Objectives ■ Direct objectives a.To identify and reveal as many errors as possible in the tested software b.To bring the tested software, after correction of the identified errors and retesting, to an acceptable level of quality. c.To perform the required tests efficiently and effectively, within the limits budgetary and scheduling limitation. ■ Indirect objectives ■ a. To compile a record of software errors for use in error prevention (by corrective and preventive actions)

Software Testing Strategies Incremental testing strategies: -Test incrementally: Unit testing; Integration testing; System testing Bottom-up testing Top-down testing Big bang testing – –Test entire software at one time.

M9 M8 M1M2M3M4M5M6M7 M10 M11 Integration A Integration BIntegration c Stage 2 Stage 4 Stage 3 Stage 1 Explain sequence Bottom-up testing

M9 M8 M1M2 M3M4M5 M6M7 M10 M11 Integration C Integration A Integration B Stage 3 Stage 1 Stage 2 Stage 5 Integration D Stage 4 Stage 6 Top-down testing Explain:

Software Testing Strategies There are many variations to these approaches too. One can do a depth-first approach (vertically sequenced, according to your book) or a horizontally-sequenced approach. This depends on many factors Can also do a sandwich approach too: partially bottom-up and then top-down. When to do this? Perhaps when existing modules are reusable and already exist. Can do both!

Top-down testing of module M8Bottom-up testing of module M8 Module on test M9 Stub of M2 Stub of M1 M8 Module on test Drive of M9 M2M1M8 Module tested in an earlier stage Modules tested in an earlier stage Use of Stubs and Drivers for Incremental Testing

class Tree { private Node root;// only data field in Tree; but key! public void find (int key) { // stub: not showing details of this method here }// end find() public void insert (int id, double dd) { // stub; placeholder }// end insert() pubic void delete (int id) { // stub }// end delete() } // end class Tree. Tree Class – Top Down: Examples of Drivers and Stubs

Top Down: Drivers: ■ In main somewhere: Tree theTree = new Tree(); theTree.find();// main hunks of functionality theTree.insert(); theTree.display(); // program runs in its entirety each time.

class Tree { private Node root;// only data field in Tree; but key! public void find (int key) { // stub: not showing details of this method here System.out.println ( “Got into find()” ); }// end find() public void insert (int id, double dd) { // stub; placeholder System.out.println ( “Got into insert()” ); }// end insert() pubic void delete (int id) { // stub System.out.println (“ Got into delete routine () “ ); }// end delete() } // end class Tree. Tree Class – Example of Stubs – can do “Displays!”

public Node find (int key) {// assumes non-empty tree Node current = root;// start at root while (current.iData != key)// if no match { if (key < current.iData) current = current.leftChild; // recall: current = current.next?? else current = current.rightChild; If (current == null) return null;// not found; boundary condition } // end while return current;// returns reference to node } // end find() Fill in code incrementally…. Develop method…

Tree Class – Bottom Up: For bottom up, you must simulate data passed to a lower level module from above. Use dummy data…and parameters and returns…

Comparison: Bottom-Up versus Top Down Bottom Up – –Main advantage is the relative ease of its performance. –Main disadvantage is the lateness at which the program as a whole can be observed. –Sometimes the pieces may not fit too. Structure may be off. –Sometimes awkward ( غير ملائم ) to pass dummy data and to accept returned data…

Comparison: Bottom-Up versus Top Down Top Down: - –Main advantage is the possibilities if offers to demonstrate the entire program functions shortly after activation of the upper level modules are completed. –Can show early analysis and design flaws –Main disadvantage – often requires complicated programming and relative difficulty of analyzing the results of tests. –Supports top down programming, top down testing, and more. –Easy to add functionality via stubs and drivers.

Comparison: Bottom-Up versus Top Down Lots of debates on preferable strategy. Choice is usually dependent upon the developers choice of a strategy – top down or bottom up. The testing strategy needs to follow the development strategy.

Big Bang Approach vs Incremental Big Bang : In general, not a good approach, unless program is very small and not terribly complicated. –Difficult to identify errors and where they are located. –Simply way too much code / functionality to evaluate at one time. Incremental testing provides a number of advantages –Test on usually small hunks ( كتلة ) of code, like unit or integration tests. –Easier to identify errors than with whole project –Correction is much simpler and requires far fewer resources too. –Find errors much earlier in process. –Prevents migration of errors into later, more complex stages. –But you do have overhead of developing drivers and stubs and drivers for integration testing. –Also, you carry out many testing operations on the same program vice only a single testing operation. Best: generally incremental approach is preferred despite its disadvantages.

Software Test Classifications While quite different (each has strong proponents), there are two accepted classification schemes: –1. Black Box testing –2. White Box testing. Black box Testing: (Functional Testing) – identifies bugs only according to software malfunctioning ( خلل ) as they are revealed in its erroneous outputs. In cases that the outputs are found to be correct, black box testing disregards the internal path of calculations and processing performed. White Box Testing: (Structural) – Examines internal calculation paths in order to identify bugs. Although the term ‘white’ is meant to emphasize the contrast between this method and black box testing, the method’s other name – ‘glass box testing’ better expresses its basic characteristic, that of investigating the correctness of code structure.

Black Box and White Box Definitions Black box testing – IEEE definition 1.Testing that ignores the internal mechanism of the system or component and focuses solely on outputs in response to selected inputs and execution conditions 2.Testing conducted to evaluate the compliance of a system or component with specified functional requirements White box testing – IEEE definition ■Testing that takes into account the internal mechanism of a system or component

Classification According to Requirements White box testing (to me) is more complicated than black box testing. There are many kinds of white box testing: –Path testing, branch testing, equivalence class testing, static analysis, dynamic analysis, etc. Complexity testing, flow graphs, call graphs, etc. –These can be very complicated. –Good for verification. Black box testing – oversimplified – checks to see if the correct outputs are produced for specific inputs. –There is no internal checking. –Simple outputs given inputs –Great for validation testing.