Prepared by Stephen M. Thebaut, Ph.D. University of Florida Course Wrap-Up Software Testing and Verification Prepared by Stephen M. Thebaut, Ph.D. University of Florida

A few loose ends… Exam 2 + solution notes will be posted (under “Practice Exams”) when all registered students are known to have taken it. Exam 2 and course grades will be posted on Sakai when available – probably around April 29.

A few loose ends… (cont’d) On-campus students may pick-up their graded exam by appointment shortly after course grades have been posted. Details will be posted on the course website under “Announcements.” Graded exams for EDGE students will be scanned and posted by UF EDGE shortly after course grades have been posted. Recall that course grades are based SOLELY on the two exam scores. (Please do not ask if there is some way to improve your grade by doing additional work, etc.)

Verification and Validation… …at the unit/component-level of system development, is what this course is mostly about. Adds value...by improving product quality and reducing risks. Three approaches: Machine-based testing Human-based testing Formal correctness proofs

Machine-Based Testing Execution of (“crafted”) test cases Actual and expected results (i.e., program behaviors) are compared. Black-Box testing Partition testing Combinatorial Approaches Boundary Value Analysis Intuition & Experience

Machine-Based Testing (cont’d) White-Box testing Logic coverage Dataflow coverage Path conditions and symbolic evaluation Fault-based testing Integration testing techniques Top-Down Bottom-Up Risk Driven Schedule Driven Function or Thread Driven

Machine-Based Testing (cont’d) Higher-Level testing Usability test Installability test Serviceability test Performance test Stress test Security test Software compatibility test Device and configuration test Recovery test Reliability test

Human-Based Testing Our focus: Reviews / Inspections Can be more effective than (machine-based) testing (as demonstrated in many studies). Applicable to code, design, requirements, testware, publications, etc. “A (key) ingredient…is consistent and vigorous discipline.” On the value of Inspections: some can achieve 100:1 ROI; you can expect 10:1 ROI.

Formal Correctness Proofs “pre-condition” {X≥2} SP2 := 4 while SP2 <= X SP2 := SP2 * 2 end_while { ( i  SP2 = 2i > X) Л (2(i-1)  X) Л (X = X’) } loop “invariant” { ( i  SP2 = 2i) Л (2(i-1)  X) Л (X = X’) } “post-condition”

Formal Correctness Proofs (cont’d) What are the benefits of studying formal verification? Understanding its limitations. Deeper insights into programming and program structures. Criteria for judging both programs and programming languages. The ability to formally verify small (or parts of large) programs.

Formal Correctness Proofs (cont’d) Bottom line: even if you never attempt to “prove” a program correct outside this course, the study of formal verification should change the way you write and read programs.

Formal Correctness Proofs (cont’d) Axiomatic verification: pre- and post-conditions, ROI’s, weak and strong correctness, synthesizing invariants for loops Predicate transforms: potential for automation, collapsing infinite expressions for loops, the connection between wp’s and loop invariants Functional verification: complete and sufficient correctness, the Iteration Recursion Lemma, the Invariant Status Theorem (the connection between program functions and loop invariants)

Formal Correctness Proofs (cont’d) Cleanroom Software Engineering Intended to make formal verification more practical and effective. Based on the notion that defects in software should be avoided rather than detected and repaired. (cf. MVQ...) Uses functional verification – components are not executed or developer-tested!

In summary… V&V techniques have evolved considerably and require specialized knowledge, disciplined creativity, and ingenuity. Software engineers should be familiar with all V&V techniques, and should be able to employ (and assess the effectiveness of) those that are appropriate to their responsibilities. An important challenge in SE is to more effectively integrate the V&V techniques currently in use.

Prepared by Stephen M. Thebaut, Ph.D. University of Florida Course Wrap-Up Software Testing and Verification Prepared by Stephen M. Thebaut, Ph.D. University of Florida