Verification and Validation Yonsei University 2 nd Semester, 2014 Sanghyun Park

Topics Covered  Planning Verification And Validation  Software Inspections  Automated Static Analysis

Verification Vs. Validation  Verification  “Are we building the product right?”  The software should conform to its specification  Validation  “Are we building the right product?”  The software should do what the user really requires

V & V Goals  Verification and validation should establish confidence that the software fits for purpose  This does NOT mean completely free of defects  Rather, it must be good enough for its intended use and the type of use will determine the degree of confidence that is needed

V & V Confidence  The level of required confidence depends on the system’s purpose, the expectations of the system users and the current marketing environment for the system  Software function  How critical is the software to an organization?  User expectations  Users may have low expectations of certain kinds of software  Marketing environment  Getting a product to market early may be more important than finding defects in the program

Two V & V Techniques  Software inspections  Analyze and check system representations such as the requirement document, design diagrams, and the program source code  Static techniques as they do not require the system to be executed  Software testing  Involves executing an implementation of the software with test data and examining the outputs of the software and its operational behaviour  Dynamic techniques as it works with an executable representation of the system

Static And Dynamic V & V Formal specification High-level design Requirement specification Detailed design Software inspections Program Prototype Software testing

Types Of Testing  Defect testing  Tests designed to discover system defects  A successful defect test is one which reveals the presence of defects in a system  Validation testing  Intended to show that the software is what the customer wants – that it meets its requirement  A successful test is one that shows that a requirement has been properly implemented

Testing And Debugging  Defect testing and debugging are distinct processes  Defect testing is a process that establishes the existence of defects in a software system  Debugging is a process that locates and corrects these defects the debugging process

V & V Planning  Careful planning is required to get the most out of inspections and testing, and to control the costs of the V & V process  Planning should start early in the development process  Planning should decide on the balance between static and dynamic approaches

Test Plans As A Link Between Development and Testing t

Software Inspections  Involve people examining the source representation with the aim of discovering anomalies and defects  Do not require execution of a system so may be used before implementation  May be applied to any representation of the system (requirements, design, configuration data, etc.)  Very effective way for discovering errors

Inspection Success  Two reasons why inspections are usually more effective than testing for discovering defects:  Many different defects may be discovered in a single inspection. In testing, one defect may mask another so several executions are required  Reviewers reuse domain and programming knowledge so they are likely to have seen the types of error that commonly arise in particular programming languages and in particular types of application

Program Inspections  Intended explicitly for defect DETECTION (not correction)  Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e.g. an uninitialized variable) or non-compliance with standards

Inspection Pre-conditions  A precise specification of the code to be inspected must be available  The members of the inspection team must be familiar with the organizational standards  An up-to-date, syntactically correct version of the code must be available

The Inspection Process

Inspection Checklists  Checklist of common errors should be used to drive the inspection  Checklist varies according to programming language because of the different levels of checking provided by the language compiler  The ‘weaker’ the type checking, the ‘larger’ the checklist  Examples: Initialization, constant naming, loop termination, array bounds, etc.

Inspection Rate  About 500 source code statements per hour during overview stage  About 125 source code statements per hour during individual preparation  From 90 to 125 statements per hour during the meeting  Inspection is therefore an expensive process

Automated Static Analysis  Static program analyzers are software tools which scan the source text of a program and detect possible faults and anomalies  They parse the program text and then detect whether or not statements are well formed, make inferences about the control flow in the program, and in many cases, compute the set of all possible values for program data  Very effective as an aid to inspections. A supplement to but not a replacement for inspections

Automated Static Analysis Checks

LINT Static Analysis