Chapter 10 – Software Testing

Software Testing Definition Purpose Process of finding errors Ultimate review of specifications, design & coding Purpose guarantee all elements of an application work together properly, function as expected, and meet performance criteria

Testing ground rules Testing is the process of executing a program with the intent of finding errors A good test case is one that has a high probability of finding an error A successful test is one that uncovers an error Benefits Uncover errors in software Demonstrates that software functions appear to be working according to specification Demonstrate performance requirements have been met Provide indication of software reliability and quality Testing cannot show the absence of defects

Verification – Validation Verification: to ensure that the software correctly implements a specific function. “Are we building the system right ?” Validation: to ensure that the software correspond to customer requirement. “Are we building the right system ?”

Testing Who tests What is tested Why test How (test cases) Programmers Testers Users What is tested Unit testing Integration testing System testing Validation testing Why test Acceptance Conformance Configuration Performance, stress How (test cases) Intuition Specification based (black box) Code based (white-box) Existing cases (regression) Faults

Testing Strategy We begin by ‘testing-in-the-small’’ and move toward ‘testing-in-the-large’ For conventional software The module (component) is our initial focus Integration of modules follows For OO software our focus when “testing in the small” changes from an individual module (the conventional view) to an OO class that encompasses attributes and operations and implies communication and collaboration Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Unit Test Environment driver test cases Module software engineer stub RESULTS interface local data structures boundary conditions independent paths error handling paths test cases

Unit Test Methods Black-Box Methods White-Box Methods Statement coverage Test cases cause every line of code to be executed Branch coverage Test cases cause every decision point to execute Path coverage Test cases cause every independent code path to be executed Black-Box Methods Equivalence partitioning Divide input values into equivalent groups Boundary Value Analysis Test at boundary conditions

EX. Program Control Graph of computeFine() False True 1 2 3 4 5 6 7 9

Branch Coverage of computeFine() Test Case #1 False True 1 2 3 4 5 6 7 Test Case #2 False True 1 2 3 4 5 6 7 10

Distinct Paths of computeFine() False True 1 2 3 4 5 6 7 Path #2 Path #3 False True 1 2 3 4 5 6 7 Path #4 False True 1 2 3 4 5 6 7 Path #1 False True 1 2 3 4 5 6 7 11

Test Cases for Distinct Paths of computeFine() daysLate printON Path 1 TRUE 1-2-3-4-5-6 7 2 60 FALSE 1-2-4-5--7 3 1-2-3-4-5-7 4 1-2-4-5-6-7 12

Basis Path Testing First, we compute the cyclomatic complexity: number of simple decisions + 1 or number of enclosed areas + 1 In this case, V(G) = 4 E - N + P = 11-8+1 Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Cyclomatic Complexity A number of industry studies have indicated that the higher V(G), the higher the probability or errors. V(G) modules modules in this range are more error prone Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Black-Box Testing requirements output input events Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Black-Box Testing How is functional validity tested? How is system behavior and performance tested? What classes of input will make good test cases? Is the system particularly sensitive to certain input values? How are the boundaries of a data class isolated? What data rates and data volume can the system tolerate? What effect will specific combinations of data have on system operation? Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Black-Box Testing Test cases can not be selected to be ‘optimal’ as the structure of the system is unknown. Equivalence classes: The input domain is divided into a number of classes. The system is then tested with one sample from each class. Boundary values: Selecting test cases at the edge of the class. (Most errors occur at the boundaries of the input-domain)

Equivalence partitioning Partition system inputs and outputs into ‘equivalence sets’ If input is a 5-digit integer between 10,000 and 99,999, equivalence partitions are <10,000, 10,000-99,999 and > 10, 000 Choose test cases at the boundary of these sets

Integration Testing Strategies Options: The “big bang” approach An incremental construction strategy Big bang ทดสอบทั้งหมดทีเดียวพร้อมกันทั้งชุด Incremental ค่อยๆทดสอบไปทีละส่วน โดยใช้ driver-stub มาช่วยเป็นตัวหลอก Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Top Down Integration top module is tested with stubs stubs are replaced one at a time, "depth first" as new modules are integrated, some subset of tests is re-run A B C D E F G Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Bottom-Up Integration drivers are replaced one at a time, "depth first" worker modules are grouped into builds and integrated A B C D E F G cluster Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Regression testing Regression testing is the repetition of previously executed test cases for the purpose of finding defects in software that previously passed the same set of tests. A regression test is applied to a new version to verify that it still perform the same functions as the old version. A set of essential /representative tests from previous version is repeated. Test of functionality most likely affected by the changes. Test of the new/modified module. เป็นการทดสอบระบบเมื่อมีการแก้ไขบางส่วนของโปรแกรม ก็ให้ทำการทดสอบใหม่ทั้งหมดอีกครั้ง เพื่อดูว่าส่วนที่เปลี่ยนแปลงเข้ากันได้กับส่วนที่เหลือหรือไม่ และยังคงทำงานตามที่เคยทำได้ อยู่หรือไม่

Object-Oriented Testing begins by evaluating the correctness and consistency of the OOA and OOD models testing strategy changes the concept of the ‘unit’ broadens due to encapsulation integration focuses on classes and their execution across a ‘thread’ or in the context of a usage scenario validation uses conventional black box methods test case design draws on conventional methods, but also encompasses special features Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

OOT Strategy class testing is the equivalent of unit testing operations within the class are tested the state behavior of the class is examined integration applied three different strategies thread-based testing — integrates the set of classes required to respond to one input or event use-based testing — integrates the set of classes required to respond to one use case cluster testing — integrates the set of classes required to demonstrate one collaboration Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

System test Test according to system specifications (validation). Test for each function/requirement at the interface. Thread testing Test according to user scenarios/use cases.

Acceptance test Purpose: convince the customer that the system correspond to the system requirement. FAT (factory acceptance test) A complete test according to system requirement carried out in the software house. SAT (site acceptance test) A complete test according to system requirement carried out after the system has been installed at the customer.

Operation/performance test Timing test Stress test Volume test Recovery test Availability test Such a test is normally carried out after the system has been put into operation (normally also coincide with the final payment for the system).

Comparison Testing Used only in situations in which the reliability of software is absolutely critical (e.g., human-rated systems) Separate software engineering teams develop independent versions of an application using the same specification Each version can be tested with the same test data to ensure that all provide identical output Then all versions are executed in parallel with real-time comparison of results to ensure consistency Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

Test Only after the completion of the test plan the system and its requirements were fully understood by the software developer.