Verification vs. Validation Verification: "Are we building the product right?" The software should conform to its specification.The software should conform to its specification. Validation: "Are we building the right product?" The software should do what the user really requires.The software should do what the user really requires.

The V & V Process Is a whole life-cycle process - V & V must be applied at each stage in the software process. Is a whole life-cycle process - V & V must be applied at each stage in the software process. –Example: Peer document reviews Has two principal objectives Has two principal objectives –The discovery of defects in a system –The assessment of whether or not the system is usable in an operational situation

V & V Goals Verification and validation should establish confidence that the software is fit for its purpose. Verification and validation should establish confidence that the software is fit for its purpose. –This does NOT mean completely free of defects. –Rather, it must be good enough for its intended use. The type of use will determine the degree of confidence that is needed.

V & V Confidence Depends on the system’s purpose, user expectations, and marketing environment Depends on the system’s purpose, user expectations, and marketing environment –System purpose »The level of confidence depends on how critical the software is to an organization (e.g., safety critical). –User expectations »Users may have low expectations of certain kinds of software –Marketing environment »Getting a product to market early may be more important than finding defects in the program

Static vs. Dynamic V & V Code and document inspections - Concerned with the analysis of the static system representation to discover problems (static v & v) Code and document inspections - Concerned with the analysis of the static system representation to discover problems (static v & v) –May be supplement by tool-based document and code analysis Software testing - Concerned with exercising and observing product behaviour (dynamic v & v) Software testing - Concerned with exercising and observing product behaviour (dynamic v & v) –The system is executed with test data and its operational behaviour is observed

Static Testing & Dynamic Testing Techniques

Static and Dynamic Testing

Static Testing Verifying the conformance of a software system and its specification without executing the code (by a computer) Verifying the conformance of a software system and its specification without executing the code (by a computer) Static testing= human testing Static testing= human testing

Static Testing Involves analysis of source text by humans Involves analysis of source text by humans Can be carried out on ANY documents produced as part of the software process Can be carried out on ANY documents produced as part of the software process Usually more cost-effective that testing for defect detection at the unit and module level Usually more cost-effective that testing for defect detection at the unit and module level Allows defect detection to be combined with other quality checks Allows defect detection to be combined with other quality checks

Static Testing More than 60% of program errors can be detected by informal program inspections and walkthrough More than 60% of program errors can be detected by informal program inspections and walkthrough More than 90% of program errors may be detectable using more rigorous mathematical program detection More than 90% of program errors may be detectable using more rigorous mathematical program detection The error detection process is not confused by the existence of previous errors The error detection process is not confused by the existence of previous errors

Static Testing - Types Inspections – Formal Methodology Inspections – Formal Methodology Walkthroughs – Input from the peers Walkthroughs – Input from the peers

Static Testing- Purpose- Inspection Verifies that the software product satisfies it specifications Verifies that the software product satisfies it specifications Verifies that the software product satisfies specified quality attributes Verifies that the software product satisfies specified quality attributes Verifies that the software product conforms to applicable regulations, standards, guidelines, plans and procedures Verifies that the software product conforms to applicable regulations, standards, guidelines, plans and procedures

Identifies deviations from standards and specifications Identifies deviations from standards and specifications Collects software engineering data (e.g. effort and anomaly data) Collects software engineering data (e.g. effort and anomaly data) Uses the collected software engineering data to improve the inspection process itself and its supporting documentation (e.g. checklists) Uses the collected software engineering data to improve the inspection process itself and its supporting documentation (e.g. checklists)

Roles to established for inspection Inspection Leader (also called Moderator) Inspection Leader (also called Moderator) Recorder Recorder Reader Reader Author Author Inspector Inspector * Minimum 3 people should be there to conduct inspection

Rules to be followed Inspection should be carried out by 3-6 participants Inspection should be carried out by 3-6 participants All participants can act as inspector All participants can act as inspector The author should not act as reader or recorder The author should not act as reader or recorder Roles may be shared among the team members Roles may be shared among the team members Individual participant may act in more than one role Individual participant may act in more than one role Individual holding management positions over any member of the inspection team shall not participate in the inspection Individual holding management positions over any member of the inspection team shall not participate in the inspection

Static Testing- Purpose- Walkthrough Is to evaluate a software product, check conformance to standards and specifications. Is to evaluate a software product, check conformance to standards and specifications. Educating / training participants Educating / training participants Find anomalies Find anomalies Improve the software product Improve the software product Consider alternative implementation Consider alternative implementation Exchange of techniques and style variations Exchange of techniques and style variations

Roles to be established for walk- through Walk-through leader Walk-through leader Recorder Recorder Author Author Team member Team member

Rules to be followed A team should have at least two members A team should have at least two members Roles may be shared among the team members Roles may be shared among the team members Walk-through leader or author may serve as recorder Walk-through leader or author may serve as recorder Walk-through leader may be the author Walk-through leader may be the author

Dynamic Testing

Test Strategies Black Box. Design Test cases by Understanding the specifications White Box. Design test cases by understanding the actual code

Testing Strategies Black Box Testing: Black Box Testing:  Designing test cases with the knowledge of inputs, outputs and functional requirements of programs.  Without looking at contents or implementation. Structural Testing/ White Box/ Glass Box Testing Structural Testing/ White Box/ Glass Box Testing  Designing test cases by looking at internal program structure and the code.

Black Box/ White Box- a comparison Black Box White Box Black Box White Box Tests are derived from the functional design specifications Tests require knowledge of the internal program structure and the code Will fail to test “hidden” functions Will fail to detect “ missing” functions Data driven Logic driven testing Required exhaustive input testing to detect all errors Requires exhaustive Path testing to detect all errors

White Box Testing Methodologies o Statement Coverage o Branch/ decision coverage o Condition coverage o Condition / Decision coverage o Multiple Condition coverage

Statement Coverage  Also known as Line Coverage, Segment Coverage, C1 and Basic Block Coverage.  This measure reports whether each executable statement is encountered.  Does not report whether loops reach their termination condition, only whether the loop body was executed.  Statement Coverage is insensitive to logical operators.  Chief advantage is that it can be applied directly to object code and does not require processing source code.  Chief disadvantage is that it is insensitive to certain control structures.

Branch or Decision Coverage Statement coverage does not address all outcomes of decisions Branches like if…else, while, for, do…while are to be evaluated for both true and false Test each decision for a true value and a false value

Condition Coverage Branch Coverage considers entire boolean expression as one and tests for true and false Branch Coverage considers entire boolean expression as one and tests for true and false It ignores branches within boolean expressions occurring due to relational and logical operatives It ignores branches within boolean expressions occurring due to relational and logical operatives All the conditions should be executed at least once for both false and true All the conditions should be executed at least once for both false and true Conditions using relational and logical operators should be checked for all possible outcomes. Conditions using relational and logical operators should be checked for all possible outcomes.

Condition / Decision Coverage  A hybrid measure composed of Condition Coverage and Decision Coverage.  Chief advantage is the simplicity without the shortcoming of its component measures

Path Coverage  Also known as Predicate Coverage.  This measure reports whether each of the possible paths in each function have been followed.  A path is a unique sequence of branches from the function entry to exit.  Since loops contains an unbounded number of paths, Path Coverage only considers a limited number of looping possibilities.  This measure has the advantage of requiring very thorough testing.  Chief Disadvantage is that the number of paths is exponential to the number of branches. Adding one additional branch could double the number of paths to test.  Another disadvantage is that many paths are impossible to exercise due to the relationships of data.

More Coverage's…. Function Coverage Call Coverage Linear Code Sequence and Jump (LCSAJ) Coverage Data Flow Coverage Object Code Branch Coverage Loop Coverage Race Coverage Weak Mutation Coverage