Software Verification & Validation By: Sunmeet Sethi Bhavin kansara

Outline  Introduction to Verification and Validation  What is Verification  What is Validation  Verification and Validation Techniques * Static Techniques (Inspection) * Dynamic Techniques (Testing)  Independent Verification and Validation  References

Introduction  Verification & Validation : It is the process of checking that a software system meets specifications and that it fulfills its intended purpose  It involves checking the software against its specifications.  It is used to describe ways of achieving quality software.  Verification is done during the project development life cycle where as validation is done after the product is ready.

Introduction  Verification: Are we building the product right? The software should conform to its specification Validation: Are we building the right product? The software should do what the user really requires

Example  Assume we have to develop an adder which should add only two integer and floating point numbers and should generate an error if string is given as an input. What is the verification and validation part in this example?

Example ( Cont.)  Verification * It should take two inputs that should be integer or floating point. * It should generate an error if string is given as input  Validation * It should add and give the accurate result

Verification  Verification is a broad and complex discipline of software engineering whose goal is to assure that a software fully satisfies all the expected requirements.  Verification involves evaluating software during each life cycle phase to ensure that it meets the requirements set forth in the previous phase.  Verification includes Inspection code inspection, design inspection etc.

Terms involved in Verification  Inspection Team of at least 4 people, which formally reviews the documents and work products during various phases of the product development life cycle  Walkthroughs it is same as inspection without formal preparation (presentation and documentation)  Buddy Checks it is used to find out bugs in a work product during verification

Validation  involves testing software or its specification at the end of the development effort to ensure that it meet its requirements. Terms used in Validation * Unit testing * System testing * Integration testing * Functional testing

V & V MUST BE APPLIED AT EACH STAGE IN THE SOFTWARE PROCESS

Benefits of Verification and Validation  To detect and Stop defects at early stages of life cycle  To ensure software quality.  Ensure resulting product is sound and safe to use.

Index Standard Cost Ratio to fix a defect

Verification and Validation Technique  Static Techniques Software Inspections (against source code)  Dynamic Techniques Software Testing (requires executable program)

What is Inspection  An inspection is a formal review of a work product by the work product owner and a team of peers looking for errors, omissions, inconsistencies, and areas of confusion in the work product.  Involve people examining the source representation with the aim of discovering anomalies and defects  Do not require execution of a system so may be used before implementation  May be applied to any representation of the system (requirements, design, test data, etc.)  Very effective technique for discovering errors if done properly

Inspection pre-conditions  A precise specification must be available.  Team members must be familiar with the organisation standards.  Syntactically correct code or other system representations must be available.  An error checklist should be prepared.  Management must accept that inspection will increase costs early in the software process.  Management should not use inspections for staff appraisal ie finding out who makes mistakes.

The Inspection Team – at least 4 members  Author: This is usually the person who originally constructed the work product  Moderator: responsible for ensuring that the inspection procedures are performed through out the entire inspection process.  Reader: Leads the inspection and who reads the code to the team.  Recorder: will document all defects that arise from the inspection meeting.  Inspector: All of the Inspection Team individuals are also considered to play the Inspector role. The Inspector role is responsible for analyzing and detecting defects within the work product.

Inspection Types  Requirement Inspection  Design Inspection  Code Inspection  Test Inspection These all types consists of four stages

The Software Inspection Stages

1. Planning  Period of time in which details for an inspection are decided and necessary arrangements are made.  These usually include * selecting the inspection team * finding a time and venue for the inspection meeting * deciding whether an orientation meeting is needed.

2. Preparation  The objective of this stage is to detect the potential defects in the work product  Usually a checklist is used to suggest potential defects in the work product.  inspectors individually study and examine the work product

3. Inspection Meeting  Objective is to find problems not solve them  Meeting where the work product is examined for defects by the entire inspection team.  The results of this meeting are recorded in a defect list (defect log).

4. Follow-Up  Short meeting between the author and moderator to verify that defects found during the inspection meeting have been corrected  Involves re-inspection, if all defects are not corrected

In summary - Inspection  System overview is presented to the inspection team  Code and associated documents are distributed to inspection team in advance  Inspection takes place and discovered errors are noted  Modifications are made to repair discovered errors  Re-inspection may or may not be required, depending on the density and severity of defect discovered  When we find an error in an inspection, we also know its nature and location. That is not so in testing.

Software Testing  Testing is the process of executing a program with the intent of finding errors  Testing is obviously concerned with errors, faults, failures and incidents. A test is the act of exercising software with test cases with an objective of  Finding failure  Demonstrate correct execution

Objective of Testing  Provide confidence in the system  Identify areas of weakness  Establish the degree of quality  Establish the extent that the requirements have been met, i.e. what the users asked for is what they got not what someone else though they wanted  To provide an understanding of the overall system  To prove it is both usable and operable

A sample testing cycle  Requirements Analysis: Testing should begin in the requirements phase of the software development life cycle  Test Planning: Test Strategy  Test Development: Test Procedures, Test Scenarios, Test Cases to use in testing software  Test Execution: Testers execute the software based on the plans and tests and report any errors found to the development team.  Retesting the Defects

Testing Classification  There are two levels of classification One distinguishes at granularity level  Unit level  Integration level  System level Other classification (mostly for unit level) is based on methodologies  Black box (Functional) Testing  White box (Structural) Testing

Testing Classification Tested Subsystem Code Functional Integration Unit Tested Subsystem Requirements Analysis Document System Design Document Tested Subsystem Test Test Unit Test Unit Test User Manual Requirements Analysis Document Subsystem Code Subsystem Code All tests by developer Functioning System Integrated Subsystems

Unit Testing  Unit testing is a procedure used to validate that individual units of source code are working properly.  Unit testing involves only those characteristics that are vital to the performance of the unit under test. This encourages developers to modify the source code without immediate concerns about how such changes might affect the functioning of other units or the program as a whole.  Once all of the units in a program have been found to be working in the most efficient and error-free manner possible, larger components of the program can be evaluated by means of integration testing.

Integration Testing  It is a software development process, in which program units are combined and tested as groups in multiple ways.  The objective of integration testing is to find bugs related to interfaces between modules as they are integrated together.  There are two major ways of carrying out an integration test bottom-up method top-down method

Top Down Method Strategy  Start with subsystems in top layer of call hierarchy  Include subsystems that are called by the previously tested subsystems  Repeat until all subsystems are included

Top Down Method

Bottom Up Method Strategy  Start with subsystems in lowest layer of call hierarchy  Test subsystems that call the previously tested subsystems  Repeat until all subsystems are included

Bottom Up Method

System Testing  Testing that attempts to discover defects that are properties of the entire system rather than of its individual components.  The purpose of system testing is to identify defects that will only surface when a complete system is assembled. That is, defects that cannot be attributed to individual components or the interaction between two components.  System testing includes testing of performance, security, startup and recovery from failure modes.

Black Box Testing  Also known as functional testing  No knowledge of software design is used  Test are strictly based on requirements and functionality  This test is carried out by someone else other then the designer or coder

Advantages of Black Box Testing  The test is unbiased because the designer and the tester are independent of each other.  The tester does not need knowledge of any specific programming languages.  The test is done from the point of view of the user, not the designer.  Test cases can be designed as soon as the specifications are complete.

Disadvantages of Black Box Testing  The test can be redundant if the software designer has already run a test case.  The test cases are difficult to design.  Testing every possible input stream is unrealistic because it would take a inordinate amount of time; therefore, many program paths will go untested.

White Box Testing  Also known as glass box, structural, clear box and open box testing.  It is a method of testing in which knowledge of software's internal design is used to develop a test  white box testing uses specific knowledge of programming code to examine outputs.

Comparison between Black & White Box Testing Black Box TestingWhite Box Testing concerned only with testing the software product. concerned only with testing the implementation. it cannot guarantee that the complete specification has been implemented it cannot guarantee that all parts of the implementation have been tested black box testing is testing against the specification and will discover faults of omission, indicating that part of the specification has not been fulfilled White box testing is testing against the implementation and will discover faults of commission, indicating that part of the implementation is faulty Less expensive then white box testingMore expensive then black box testing start test planning with a black box test approach as soon as the specification is available White box planning should commence as soon as all black box tests have been successfully passed

Independent Verification and Validation  Independent verification and validation is performed by an organization that is technically, managerially and financially independent of the development organization.

Key Points  Verification shows conformance with the specification whereas validation shows that the program meets the customer’s needs.  Test plans should be drawn up to guide the testing process.  Program inspections are effective in discovering errors.  Static verification involves examination and analysis of the source code for error detection, and can be used to discover anomalies in the source code.

References  Guide to software verification and validation --- ESA Board for Software Standardization and Control (BSSC).  Software Verification and Validation for Practitioners and Managers, Second Edition by Steven R. Rakitin.  Validation, Verification, and Testing of Computer Software, W. Richards Adrion, Martha A. Branstad, John C. Cherniavsky.ACM journal, computing surveys  Software verification and validation : an overview by Dolores Wallace and Roger Fujii, IEEE Software  IEEE guide for software verification and validation plans