Software Testing Life Cycle Designed and Compiled by: Udayakumar Sree Senior Test Engineer

STLC - Definition The course of software being tested in a well-planned way is known as Software test life cycle. Contract Signing Requirement Analysis Test Planning Development Execution Defect Reporting Retest Defects Product Delivery

STLC – Stages Involved Contract Signing: Process - The project is signed with client for testing the software Documents involved: SRS Test Deliverables Test Metrics etc.

STLC – Stages Involved Requirement Analysis: Process: Analyzing software for design and implementation methods and testable aspects are recorded Documents involved: Requirement Specification documents Functional Specification documents Design Specification documents (use cases, etc) Use case Documents Test Trace-ability Matrix for identifying Test Coverage

STLC – Stages Involved Test Planning: Test Scope, Test Environment Process: To plan, how the testing process should flow Test Process Flow Test Scope, Test Environment Different Test phase and Test Methodologies Manual and Automation Testing Defect Mgmt, Configuration Mgmt, Risk Mgmt. Etc Evaluation & identification – Test, Defect tracking tools Documents Involved: Master Test Plan, Test Scenario, SCM

STLC – Stages Involved Test Development Process: Test Traceability Matrix and Test coverage Test Scenarios Identification & Test Case preparation Test data and Test scripts preparation Test case reviews and Approval Base lining under Configuration Management Documents Involved: Test Plan, RTM Test cases

STLC – Stages Involved Test Execution: Process: Executing Test cases Testing Test Scripts Capture, review and analyze Test Results Raising the defects and tracking for its closure Documents Involved: Test Cases Test Execution report Bug report Requirement traceability matrix

STLC – Stages Involved Defect Reporting Process: Defect logging Assigning defect and fixing Retesting Defect closing Documents involved: Test report Bug Report

STLC – Stages Involved Product Delivery Process: After the product had undergone several tests, the acceptance test is done by the user/client i.e. UAT, wherein the use cases were executed and the product is accepted to go on live Test Metrics and process Improvements made Build release Receiving acceptance Documents involved Test summary reports UAT Test Plan, UAT Test cases

Test Plan

Test Plan – What? Derived from Test Approach, Requirements, Project Plan, Functional Spec., and Design Spec Details out project-specific Test Approach Lists general (high level) Test Case areas Include testing Risk Assessment Include preliminary Test Schedule Lists Resource requirements

Test Plan – Why? Identify Risks and Assumptions up front to reduce surprises later. Communicate objectives to all team members. Foundation for Test Spec, Test Cases, and ultimately the Bugs we find. Failing to plan = planning to fail.

Test Plan – Definition The test strategy identifies multiple test levels, which are going to be performed for the project. Activities at each level must be planned well in advance and it has to be formally documented. Based on the individual plans, the individual test levels are carried out.

Test Plan – Consists of… Unit Testing Tools Required tool to test at unit level Priority of Program units Module-wise priority Naming convention for test cases Status reporting mechanism Regression test approach

Test Plan – Consists of… ETVX Criteria Entry means the entry point to that phase. For example, for unit testing, the coding must be complete and then only one can start unit testing Task is the activity that is performed Validation is the way in which the progress and correctness and compliance are verified for that phase Exit tells the completion criteria of that phase, after the validation is done. For example, the exit criterion for unit testing is all unit test cases must pass

Risk Analysis A risk is a potential for loss or damage to an Organization from materialized threats. Risk Analysis attempts to identify all the risks and then quantify the severity of the risks Risk Identification Software Risks Business Risks Testing Risks Premature Release Risk Risk Methods

Risk Analysis continued… Software Risks Knowledge of the most common risks associated with Software development, and the platform you are working on Business Risks Most common risks associated with the business using the Software Testing Risks Knowledge of the most common risks associated with Software Testing for the platform you are working on, tools being used, and test methods being applied

Risk Analysis continued… Premature Release Risk Ability to determine the risk associated with releasing unsatisfactory or untested Software Products Risk Methods Strategies and approaches for identifying risks or problems associated with implementing and operating information technology, products and process; assessing their likelihood, and initiating strategies to test those risks

Test Execution

Software Testing Fundamentals Testing objectives include Testing is a process of executing a program with the intent of finding an error A good test case is one that has a high probability of finding an as yet undiscovered error A successful test is one that uncovers an as yet undiscovered error Testing should systematically uncover different classes of errors in a minimum amount of time and with a minimum amount of effort. A secondary benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications

When Testing should start? Testing early in the life cycle reduces the errors. Test deliverables are associated with every phase of development. The goal of Software Tester is to find bugs, find them as early as possible, and make them sure they are fixed The number one cause of Software bugs is the Specification The next largest source of bugs is the Design

When to Stop Testing? Some reasons to stop test are: This can be difficult to determine. Many modern software applications are so complex, and run in such as interdependent environment, that complete testing can never be done. Some reasons to stop test are: Deadlines (release deadlines, testing deadlines.) Test cases completed with certain percentages passed Test budget depleted Coverage of code/functionality/requirements reaches a specified point The rate at which Bugs can be found is too small Beta or Alpha Testing period ends The risk in the project is under acceptable limit

Test Execution Testing of an application includes: Unit Testing Integration testing System Testing Acceptance testing These are the functional testing strategies and few other functional, non-functional, performance and other testing methods can also be applied on the software.

Test Execution – Unit testing The unit test plan is the overall plan to carry out the unit test activities. The lead tester prepares it and it will be distributed to the individual testers Basic input/output of the units along with their basic functionality will be tested input units will be tested for the format, alignment, accuracy and the totals The UTP will clearly give the rules of what data types are present in the system, their format and their boundary conditions Testing the screens, files, database etc., are to be given in proper sequence

Test Execution – Integration testing The integration test plan is the overall plan for carrying out the activities in the integration test level This section clearly specifies the kinds of interfaces fall under the scope of testing internal, external interfaces, with request and response is to be explained Two approaches practiced are Top-Down and Bottom-Up integrations Given this correctly, the testing activities will lead to the product, slowly building the product, unit by unit and then integrating them

Test Execution – System testing The system test plan is the overall plan carrying out the system test level activities System testing is based on the requirements All requirements are to be verified in the scope of system testing The requirements can be grouped in terms of the functionality Based on this, there may be priorities also among the functional groups Apart from this what special testing is performed are also stated here

Test Execution – Non-functional testing Non-functional testing includes: Installation testing – Installation environment, practical obstacles etc. Compatibility testing – compatibility with other system software Configuration testing - how well the product works with a broad range of hardware/peripheral equipment configurations as well as on different operating systems and software Security testing - secure from mistaken/accidental users, hackers, and other malevolent attackers Recovery testing - how well a system recovers from crashes, hardware failures, or other catastrophic problems Usability testing - Testing for 'user-friendliness'

Test Execution – Performance testing Performance testing includes: Load Testing – Testing with the intent of determining how well the product handles competition for system resources. The competition may come in the form of network traffic, CPU utilization or memory allocation

Test Execution – Performance testing Stress testing - Testing with the intent of determining how well a product performs when a load is placed on the system resources that nears and then exceeds capacity

Bug/Defect Management

BUG LIFE CYCLE - What is a bug? A software bug is an error, flaw, mistake, failure, or fault in a computer program that prevents it from behaving as intended.

BUG LIFE CYCLE What is a Bug Life Cycle? In software testing, the term life cycle refers to the various stages that a defect/bug assumes over its life.

BUG LIFE CYCLE Stages involved in Bug Life Cycle The different stages involved in a bug life cycle are as follows: Finding Bugs Reporting/ Documentation Fixing Retesting Closing

BUG LIFE CYCLE Stages explained Finding Bugs: Software Tester finds bug while testing. It is then logged and assigned to a programmer to be fixed. Reporting/ Documentation: In software, bugs need to be tracked and managed to Communicate bug for reproducibility, resolution, and regression. Track bug status (open, resolved, closed). Ensure bug is not forgotten, lost or ignored

BUG LIFE CYCLE Stages explained Continued… Fixing: Retesting: Closing: Once the bug is assigned to the developer, he fixes the bug. Once the programmer fixes the code , he assigns it back to the tester and the bugs enters the resolved state. Retesting: The tester then performs a regression test to confirm that the bug is indeed fixed. Closing: If the bug is fixed, then the tester closes the bug. Here the bug then enters its final state, the closed state.

Different status of a Bug The different status of a bug during its life cycle can be summarized as follows: New Deferred Open Reopened Assign Duplicate Test Rejected Verified Closed

Description of Various Status of a bug New: When the bug is posted for the first time, its state will be “NEW”. This means that the bug is not yet approved. Open: After a tester has posted a bug, the lead of the tester approves that the bug is genuine and he changes the state as “OPEN”. Assign: Once the lead changes the state as “OPEN”, he assigns the bug to corresponding developer or developer team. The state of the bug now is changed to “ASSIGN”. Test: Once the developer fixes the bug, he assigns the bug to the testing team for retesting. Before he releases the software with bug fixed, he changes the state of bug to “TEST”. It specifies that the bug has been fixed and is released to testing team.

Description of Various Status of a bug Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “REJECTED”. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “DUPLICATE”.

Description of Various Status of a bug Verified: Once the bug is fixed and the status is changed to “TEST”, the tester tests the bug. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “VERIFIED”. Reopened: If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “REOPENED”. The bug traverses the life cycle once again. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “CLOSED”. This state means that the bug is fixed, tested and approved.

Severity of a Bug It indicates the impact each defect has on the testing efforts or users and administrators of the application under test. This information is used by developers and management as the basis for assigning priority of work on defects.

Priority Levels of a Bug Critical : An item that prevents further testing of the product or function under test can be classified as Critical Bug. No workaround is possible for such bugs. Examples of this include a missing menu option or security permission required to access a function under test. Major / High: A defect that does not function as expected/designed or cause other functionality to fail to meet requirements can be classified as Major Bug. The workaround can be provided for such bugs. Examples of this include inaccurate calculations; the wrong field being updated, etc

Priority Levels of a Bug Average / Medium: The defects which do not conform to standards and conventions can be classified as Medium Bugs. Easy workarounds exists to achieve functionality objectives. Examples include matching visual and text links which lead to different end points. Minor / Low: Cosmetic defects which does not affect the functionality of the system can be classified as Minor Bugs.

Various Bug tracking tools The various bug tracking tools available are: Quality Center® – from HP Bugzilla® - from Mozilla Dev Track® – from TechExcel

Product Delivery

Product Delivery -Test Deliverables Test Trace-ability Matrix Test Plan Testing Strategy Test Cases (for functional testing) Test Scenarios (for non-functional testing) Test Scripts Test Data Test Results Test Summary Report Release Notes Tested Build

Product Delivery - Test Metrics Measuring the correctness of the testing process with measurable is known to be test metrics.

Product Delivery - Test Metrics There are several test metrics identified as part of the overall testing activity in order to track and measure the entire testing process. These test metrics are collected at each phase of the testing life cycle/SDLC, analyzed and appropriate process improvements are determined and implemented. The metrics should be constantly collected and evaluated as a parallel activity together with testing, both for manual and automated testing irrespective of the type of application

Product Delivery - Test Metrics - Classification Project Related Metrics – such as Test Size, # of Test Cases tested per day –Automated (NTTA) # of Test Cases tested per day –Manual (NTTM) # of Test Cases created per day – Manual (TCED) Total number of review defects (RD) Total number of testing defects (TD) etc.

Product Delivery – Test Metrics – Classification Process Related Metrics – such as Schedule Adherence (SA) Effort Variance (EV) Schedule Slippage (SS) Test Cases and Scripts Rework Effort, etc. Customer related Metrics – such as Percentage of defects leaked per release (PDLPR) Percentage of automation per release (PAPR) Application Stability Index (ASI) etc.

Product Delivery – Acceptance testing – UAT The client at their place performs the acceptance testing. It will be very similar to the system test performed by the Software Development Unit There is no specific clue on the way they will carry out the testing, since the client performs this test It will not differ much from the system testing This is just one level of testing done by the client for the overall product and it includes test cases including the unit and integration test level details

Thank You