is a validation process Software Test is a validation process Executing code from the application is provided with input and the resulting output is compared with the required output. Testing establishes the presence of defects but not their absence Testing Early and Often means the Agile Connection

Testing: Four Main Questions Testing requires significant time to decide what to test, when to do so, and with what data. At which level conducting the testing? Unit Integration System How to choose inputs? using the specifications/use cases/requirements using the code How to identify the expected output? Test the predictions (oracles) How good test cases are? When we can stop the testing activity

Test Phases Acceptance Testing – This checks if the overall system is functioning as required. Unit testing – This is basically testing of a single function, procedure, class. Integration testing – This checks that units tested in isolation work properly when put togehter. System testing –The emphasis is to ensure that the whole system can cope with real data, monitor system performance, test the system’s error handling and recovery routines. Regression Testing – This checks that the system preserves its functionality after maintenance and/or evolution tasks.

Regression Testing: What to Retest Suppose that C is a body of already-tested code in an application A. Suppose that A has been altered with new or changed code N. If C is known to depend on N Perform regression testing on C If C is reliably known to be completely independent of N There is no need to regression test C Otherwise Regression test C A snjfgkjfdkgjkfdjgkfdjkgjdfkbvjfdkjbkfdjbkfdjbkjdfklbjdflkbjkldfjbkfdjblkjfdklbjfdklbjklfdjbkldfjbkljdfkbljfdklbjoikhkgkjkDHVDABFDSGKJGFJTIURjskjgls,al40rgnsj23402- hhjk ghjhgj ghjkklj;ljko fghjhjgfj g;b,kdfgjfjfjg kfdjk C N dfkbljfdklHVDABFDS DHVDABFDSlkgt GKJGIURjskjgl

What are Software Testing Types ? Black box testing : You don't need to know the internal design in detail or have a good knowledge about the code for this test. It's mainly based on functionality and specifications, requirements. White box testing : This test is based on detailed knowledge of the internal design and code. Tests are performed for specific code statements and coding styles.

Black- & White-box Testing Result Input determined by... Actual output compared with required output Black box … requirements White box Validation of expected behavior …design elements

Black-Box Testing Black-Box Testing does not take the manner in which the application was designed or implemented Example: Person «a» rents « the Matrix» on May5 Person «b» rents «Gone with the Wind» on May 6 Person «a» returns «the Matrix» on May 10 This is analogous to building an automobile and then testing it by driving it under various conditions.

White-Box Testing White -box tests are based on design and implementation If the car is built with a new design for its automatic transmission , we would be wise to use this knowledge White box testing is required

Unit Testing versus Post-Unit testing Software applications consist of numerous parts. Therefore the soundness of the whole depends on the soundness of the parts Reliable part? Reliable? Reliable? Reliable? Reliable? Reliable ? Reliable ? Because of the dependence, we test software parts thoroughly before assembling them A process is called as unit testing

Types of Post-Unit Testing Interface testing Validates functions exposed by modules Integration …… combinations of modules System whole application Usability user satisfaction Regression changes did not create defects in existing code Acceptance customer agreement that contract satisfied Installation works as specified once installed on required platform Robustness ability to handle anomalies Performance fast enough; uses acceptable amount of memory

Post-Unit Testing Unit Module Each of post-unit testing activities is performed during the process of building an application Unit ….. Unit Build Application on final platform Application on test bed ….. ….. Module Interface testing ….. system testing installation testing usability testing Unit Build acceptance testing Unit Module integration testing ….. Unit Regression testing throughout

Plan for Testing Estimate resources Identify metrics to be collected Define “units” vs. non-units for testing Determine what types of testing will be performed Determine extent do not just “test until time expires” prioritize, so that important tests definitely performed Document individual’s personal document set included? how / when to incorporate all types of testing? how / when to incorporate in formal documents? how / when to use tools / test utilities? Determine input sources individual engineer responsible for some (units)? How / when inspected by Qality Assurance? How / when designed & performed by third parties? Estimate resources use historical data if available Identify metrics to be collected Define, gather, use e.g., time, defect count, type, and source

Stopping Criteria for Testing When tester has not been able to find another defect in 5 (10? 30? 100?) minutes of testing When all nominal, boundary and out-of-bounds test examples show no defect When a given checklist of test types has been completed After completing a series of targeted coverage (e.g., branch coverage for unit testing) When testing runs out of its scheduled time

The Source of a Unit’s Specification Implementation-time specification Key: “specifies” Example: get DVD title Design document (SDD) Implementation unit (code for method or class) Requirement document (SRS) Domain classes Example: void setDVDName( … ) Example: “It shall be possible to add a DVD to the video store’s inventory” Example: method addDVD( … ) “Preconditions: …. Postconditions: …”

Relating Tests to Requirements & Design (1) D-Requirements 3.2.EC.1.2 Qualities of Encounter characters Every game character has the same set of qualities. Each quality shall be a non- negative floating point number with at least one decimal of precision. . . . (2) Design GameCharacter: An abstract class with attribute name ... Test this class ... ... against this requirement Characters GameCharacter Test this method ... Encounter Characters ... against this requirement EncounterCharacter adjustQuality() Relating Tests to Requirements & Design

Six Types of Testing There are several types of testing that should be done on a large software system. Each type of test has a “specification” Defines the correct behavior the test is examining Incorrect behavior (an observed failure) can be identified There are two issues to think about in these types of testing The opacity of the tester’s view of the code (is it white or black box testing). The scale (is the tester examining a small bit of code or the whole system and its environment

Unit Testing Opacity: White box testing Specification: Low-level design and/or code structure Unit testing is the testing of individual hardware or software units Using white box testing techniques, testers (usually the developers creating the code implementation) verify that the code does what it is intended to do at a very low structural level. The tester will write some test code that will call a method with certain parameters and will ensure that the return value of this method is as expected. Unit testing is generally done within a class or a component

Unit Testing Both white box and black box methods are utilized during unit testing White box unit tests focus on the internal code structure , testing each program statement, every decision point and each independent path Black box methods focus on testing the unit without using its internal structure Equivalence partitioning and boundary value analysis

Integration testing Opacity: Black- and white-box testing Specification: Low- and high-level design Integration test is testing in which software components, hardware components, or both are combined and tested to evaluate the interaction between them Using both black and white box testing techniques, the tester (usually the software developer) verifies that units work together when they are integrated into a larger code base. The components work individually It doesn’t mean that they all work together when assembled or integrated.

Functional and System Testing Opacity: Black-box testing Specification: High-level design, requirements specification Using black box testing techniques, testers examine the high-level design The customer requirements specification to plan the test cases to ensure the code does what it is intended to do.

Functional and System Testing Functional testing involves ensuring that the functionality specified in the requirement specification works. System testing involves putting the new program in many different environments to ensure the program works in typical customer environments Nonfunctional Requirements are tested Stress Testing Performance Testing Usability Testing

Stress Testing Stress testing conducted to evaluate a system or component at or beyond the limits of its specification or requirement If the team is developing software to run cash registers, a non-functional requirement might state that the server can handle up to 30 cash registers looking up prices simultaneously. Stress testing might occur in a room of 30 actual cash registers running automated test transactions repeatedly for 12 hours. There also might be a few more cash registers in the test lab to see if the system can exceed its stated requirements

Performance Testing Performance testing conducted to evaluate the compliance of a system or component with specified performance requirements A performance requirement might state that the price lookup must complete in less than 1 second. Performance testing evaluates whether the system can look up prices in less than 1 second (even if there are 30 cash registers running simultaneously).

Usability Testing Usability testing conducted to evaluate the extent a user can learn to operate, prepare inputs for, and interpret outputs of a system or component.

Acceptance Testing Opacity: Black-box testing Specification: requirements specification After functional and system testing, the product is delivered to a customer The customer runs black box acceptance tests based on their expectations of the functionality. Acceptance testing is formal testing conducted to determine whether or not a system satisfies its acceptance criteria to enable the customer to determine whether or not to accept the system

Regression Testing Opacity: Black- and white-box testing Specification: Any changed documentation, high-level design Throughout all testing cycles, regression test cases are run. Regression testing is selective retesting of a system or component to verify that modifications have not caused unintended effects that the system or component still complies with its specified requirements

Levels of Software Testing

Beta testing Opacity: Black-box testing Specification: None The advantages of running beta tests are Identification of unexpected errors because the beta testers use the software in unexpected ways. A wider population search for errors in a variety of environments different operating systems with a variety of service releases and with a multitude of other applications running Low costs because the beta testers generally get free software but are not compensated.

Beta Testing The disadvantages of beta testing Lack of systematic testing because each user uses the product in any manner they choose. Low quality error reports because the users may not actually report errors or may report errors without enough detail. Much effort is necessary to examine error reports particularly when there are many beta testers.

Regression Testing Throughout all testing cycles, regression test cases are run. Regression testing is selective retesting of a system or component to verify modifications have not caused unintended effects and that the system or component still complies with its specified requirements.

Acceptance Testing versus Unit Testing The motivation of unit testing is finding faults. The motivation of acceptance testing is demonstrating working functionalities. Written and executed during the development. Written before the development and executed after. Written using a unit testing framework. Written using an acceptance testing framework (also unit testing framework). Starting point: new capability (to add a new module/function or class/method). Starting point: User stories, User needs, Use Cases, Textual Requirements, … Used to find faults in individual modules or units (individual programs, functions, procedures, web pages, menus, classes, …) of source code. Used for documentation (low level) Used to verify that the implementation is complete and correct. Used for Integration, System, and regression testing. Used to indicate the progress in the development phase. (Usually as %). Used as a contract. Used for documentation (high level) (extreme programming) When unit tests pass, write another test that fails. (extreme programming) When acceptance tests pass, stop coding. The job is done. Written by developers. Written by Customer and Analyst. Unit Tests Acceptance Tests

Iterative Software Development “At different points in the process” Iterative Software Development Write and execute unit tests Execute acceptance tests Write acceptance tests increment + system Prioritized functionalities “Written before” “Executed after the development”

Unit Test Methods White-Box Methods Statement coverage Branch coverage Test cases cause every line of code to be executed Branch coverage Test cases cause every decision point to execute Path coverage Test cases cause every independent code path to be executed Black-Box Methods Equivalence partitioning Divide input values into equivalent groups Boundary Value Analysis Test at boundary conditions