Dynamic Testing

Overview Dynamic test Test harness Black box and white box test procedures Black box techniques. Class partition technique. Test completeness criteria Statement coverage. Statement and branch coverage Instrumentation

Dynamic testing Dynamic testing is a term used to describe the testing of the dynamic behavior of code. Dynamic analysis refers to the examination of the physical response from the system to variables that are not constant and change with time. The software must actually be compiled and run; Dynamic Testing involves working with the software, giving input values and checking if the output is as expected. These are the Validation activities. Unit Tests, Integration Tests, System Tests and Acceptance Tests are few of the Dynamic Testing methodologies. Dynamic testing means testing based on specific test cases by execution of the test object or running programs.

Static vs. Dynamic Testing Sl.No Static Testing Dynamic Testing 1 Static testing is a form of software testing where the software isn’t actually used. In dynamic testing the software must actually be compiled and run. 2 It is generally not detailed testing, but checks mainly for the sanity of the code, algorithm, or document. It is primarily syntax checking of the code or and manually reading of the code or document to find errors Dynamic analysis refers to the examination of the physical response from the system to variables that are not constant and change with time 3 This type of testing can be used by the developer who wrote the code, in isolation. Code reviews, inspections and walkthroughs are also used. Some of dynamic testing methodologies include unit testing, integration testing, system testing and acceptance testing. 4 This is the verification portion of Verification and Validation Dynamic testing is the validation portion of Verification and Validation. 5 These are verification activites. Code Reviews, inspection and walkthroughs are few of the static testing methodologies. These are the Validation activities. Unit Tests, Integration Tests, System Tests and Acceptance Tests are few of the Dynamic Testing methodologies

Test Harness In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. A basic test harness will contain at least the following elements: application basics, test launching, and result reporting. It may also include a graphical user interface, logging, and test case scripting. The typical objectives of a test harness are to: Automate the testing process. Execute test suites of test cases. Generate associated test reports. A test harness may provide some of the following benefits: Increased productivity due to automation of the testing process. Increased probability that regression testing will occur. Increased quality of software components and application. Ensure that subsequent test runs are exact duplicates of previous ones.

Black box testing Black box testing, also called functional testing and behavioral testing, focuses on determining whether or not a program does what it is supposed to do based on its functional requirements. Black box testing attempts to find errors in the external behavior of the code in the following categories: (1) incorrect or missing functionality; (2) interface errors; (3) errors in data structures used by interfaces; (4) behavior or performance errors; (5) initialization and termination errors. We can determine if the functions appear to work according to specifications. No amount of testing can unequivocally demonstrate the absence of errors and defects in your code.

White Box Testing White-box testing is a verification technique software engineers can use to examine if the code works as expected White-box testing is testing that takes into account the internal mechanism of a system or component (IEEE, 1990). White-box testing is also known as structural testing, clear box testing, and glass box testing Using the white-box testing techniques, a software engineer can design test cases that (1) exercise independent paths within a module or unit; (2) exercise logical decisions on both their true and false side; (3) execute loops at their boundaries and within their operational bounds; (4) exercise internal data structures to ensure their validity White-box testing is used for Unit testing Integration testing Regression testing

White Box vs. Black Box testing 1 Focuses on the functionality of the system Focuses on the structure (Program) of the system 2 Techniques used are : · Equivalence partitioning · Boundary-value analysis · Error guessing · Race conditions · Cause-effect graphing · Syntax testing · State transition testing · Graph matrix Techniques used are: · Basis Path Testing · Flow Graph Notation · Control Structure Testing 1. Condition Testing 2. Data Flow testing · Loop Testing 1. Simple Loops 2. Nested Loops 3. Concatenated Loops 4. Unstructured Loops 3 Tester can be non technical Tester should be technical 4 Helps to identify the vagueness and contradiction in functional specifications Helps to identify the logical and coding issues

Testing Strategies Tests of Customer Requirements every single customer requirement has been tested at least once risky requirements are tested first (is devastating to find a critical flaw right before delivery) Decision Table Testing are used to record complex business rules that must be implemented in the program, and therefore tested Failure (“Dirty”) Test Cases Donald (father of computer science) sends checks to readers who find errors in his books !!! Anticipating the unexpected Test Early and Often executing your test cases as soon as possible is an excellent way of getting concrete feedback about your program. In order to run test cases early, programmers need to integrate the pieces of their code into the code base often.

Testing Strategies Equivalence Partitioning Boundary Value Analysis reduce the number of test cases that need to be developed we don’t want to write several test cases that test the same aspect of our program Equivalence partitioning divides the input domain of a program into classes. For each of these equivalence classes, the set of data should be treated the same by the module under test and should produce the same answer. Test cases should be designed so the inputs lie within these equivalence classes Boundary Value Analysis Boundary value is defined as a data value that corresponds to a minimum or maximum input, internal, or output value specified for a system or component Bugs lurk in corners and congregate at boundaries (Boris Beizer) Programmers often make mistakes on the boundaries of the equivalence classes/input domain

Test Completeness How do we know when software testing is complete? How do you know when anything is complete? You measure it, and then compare that measurement against completion objectives. How do we measure software testing ‘completeness’? Requirements Coverage Structural Coverage Architectural Coverage Requirements, structure and architecture are the three software abstractions used to develop and build software. Together, they roughly describe the ‘size’ of the software. As such, they can be used as a measure of software testing. The completeness criteria are relatively easy to determine (e.g. number of requirements

Testing Completeness Requirements Coverage Structural Coverage Has the software been tested against all requirements for the normal range of use? Has the software been tested against all requirements for abnormal or unexpected usage? Normal Range Coverage of Software Requirements Normal Range Coverage of Software Design Robustness Coverage of Software Requirements Robustness Coverage of Software Design Structural Coverage Has each element of the software been exercised during testing? Code Statements Decisions Conditions and Decisions Architectural Coverage Have the actual control and data links been used during testing? Data Paths Control Links

Coverage Statement Coverage: Branch/Decision Coverage: Path Coverage: In this the test case is executed in such a way that every statement of the code is executed at least once. Branch/Decision Coverage: A test method which aims to ensure that each possible branch from each decision point (e.g. "if" statement) is executed at least once, all reachable code is executed. A branch is the outcome of a decision, so branch coverage simply measures which decision outcomes have been tested. It helps in validating all the branches making sure that no branch leads to abnormal behavior of the application. Path Coverage: In this the test case is executed in such a way that every path is executed at least once. All possible control paths taken, including all loop paths taken zero, once, and multiple (ideally, maximum) items in path coverage technique, the test cases are prepared based on the logical complexity measure of a procedural design. In this type of testing every statement in the program is guaranteed to be executed at least one time.

Instrumentation The used tool inserts counters into the program initialized with 0 During program execution, counters are incremented when they are passed At the end the counters are used to count the number of passing to each point If a counter is 0, the according step has not been reached

http://www.softwaretestinggenius.com/download/HGCH3.pdf http://www.testingreflections.com/node/view/3655 http://agile.csc.ncsu.edu/SEMaterials/BlackBox.pdf http://agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf