1. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 1 Disciplined Software Engineering Lecture #12 Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 Sponsored by the U.S. Department of Defense Additional material by James P Alstad March 2002 Indicated by italics

2. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 2 Design Verification Design verification is covered in 2 lectures. This lecture addresses the reasons for design verification state machine correctness execution tables proof by induction The next lecture covers trace tables mathematical program verification program proofs

3. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 3 Need for Design Verification - 1 As you work to improve your personal software process, you will likely find it hard to significantly reduce the numbers of design defects you make. By using checklists and taking increased care, you can improve the yield of your code reviews. To improve the yield of design reviews, you need to use disciplined verification methods.

4. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 4 Defects Injected in Coding - Range Program Number 0 50 100 150 200 250 Max Avg. Min

5. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 5 Defects Injected in Design - Range Program Number 0 20 40 60 80 100 Max Avg Min

6. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 6 Need for Design Verification - 2 An orderly approach to design verification is essential because many common design defects are caused by overlooked conditions what seem like unlikely situations become more likely with high-powered computing systems conditions that were not possible with an initial program version may be induced by later modifications

7. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 7 Need for Design Verification - 3 By following a structured design verification procedure, you are more likely to see overlooked conditions identify rarely exposed risks recognize possible future exposures By recording data on your design reviews, you can simplify later design inspections.

8. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 8 Using Design Verification - 1 Design verification methods should be used during design during design reviews during design inspections Your design verification methods should focus on the defect types that have caused you the most trouble in test.

9. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 9 Using Design Verification- 2 The topics covered in this and the next lecture were selected because they address the design defect categories that caused me the most trouble state machine structures loop constructs I would use additional verification methods for the following defect types if they were available pointers interfaces

10. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 10 Design Verification Topics The design verification topics covered in this and the next lecture are verifying state machine correctness execution tables proof by induction trace tables mathematical program verification

11. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 11 State Machine Correctness Any program that changes the state of the computing system is a state machine. A program is a state machine if it can behave differently with identical inputs. Seemingly simple state machines can have sophisticated behavior. It is thus important to verify them carefully. Important assumption: When you do design, you select where to use state machines, and then treat them as such.

12. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 12 State Machine Verification The steps in state machine verification are to check to ensure the state machine has no hidden traps or loops check that the set of all states is complete and orthogonal check that the set of all transitions from every state are complete and orthogonal After verifying state machine correctness (i.e., does the state machine go through correct transitions, etc?), ensure that the program performs the intended application functions (i.e., when in a state, does the program do the right thing, etc?).

13. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 13 Hidden Traps or Loops - 1 To check for hidden traps and loops construct the state template construct a state machine diagram determine if any exit states are unreachable from any other states If any exit states are unreachable from any other state, this is not a proper state machine.

14. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 14 Hidden Traps or Loops - 2 Example: consider an object BSet as follows 2 states: EmptyState, MemberState 4 methods: Push, Pop, Add, Subtract Push adds a member to BSet Pop removes a member from BSet Add adds a member to BSet if it does not already contain an identical member Subtract removes a member from BSet if it contains an identical member Note: this design is perhaps not as cohesive as it might be.

15. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 15 Hidden Traps or Loops - 3 The state template for BSet would be as follows: EmptyState MemberState EmptyState MemberState EmptyState MemberState n = 0 n >= 1 Pop or Subtract Push or Add Pop(n=1) or Subtract(n=1)(D in BSet) Add or Push or Pop(n>1) or Subtract(D not in BSet) or Subtract(n>1)

16. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 16 Hidden Traps and Loops - 4 EmptyState n = 0 MemberState n > 0 Pop(n=1) or Subtract(n=1)(D in BSet) Pop or Subtract Push or Add Add or Push or Pop(n>1) or Subtract(D not in BSet) or Subtract(n>1)

17. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 17 Hidden Traps or Loops - 5 It is thus clear that this state machine has no hidden traps or loops. However, for a more complex state machine nothing may be “clear”. (Example: in a complex state machine, a particular transition out of state S may be necessary in order to reach the final state. But perhaps conditions for that transition cannot arise for any possible input which leads to state S.) In such complex cases, it may be necessary to provide additional argument for the no hidden traps or loops demonstration.

18. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 18 All Possible States - 1 A common problem is to overlook some state machine state. Examples are initial states, empty states, error states, and so forth. Examine all possible conditions of the state parameters to ensure they are either included or truly impossible.

19. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 19 All Possible States - 2 Checking the BSet example: EmptyState has n = 0 MemberState has n > 0 since n cannot be < 0, this covers all cases To ensure that none of the n > 0 states should be distinct, check to see if any of them exhibit different behavior from the others. This requires considering specifications which are prior to the state machine. A check of the state template or state diagram shows that all n>0 states behave identically. Added states are thus unnecessary.

20. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 20 State Orthogonality For the states to be orthogonal, the state machine must not be able to be in 2 states at once. In the example state machine, either n = 0 or n > 0. The states are orthogonal because the machine must be in either the EmptyState (n = 0) or the MemberState (n > 0); it cannot be in both at once.

21. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 21 State Transitions - 1 In a proper state machine, the state transitions must all be complete and orthogonal. For this to be true every state must have a defined next state for every possible input every state must have a unique next state for every possible input

22. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 22 State Transitions - 2 First, check EmptyState in the BSet example for completeness: the conditions are Push, Pop, Add, and Subtract states are defined for each condition, so the EmptyState transitions are complete. The following table shows this: Push Pop Add Subtract MemberState EmptyState MemberState EmptyState

23. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 23 State Transitions - 3 Next, check EmptyState transitions for orthogonality. The transition conditions to EmptyState are Pop and Subtract. The transition conditions to MemberState are Push and Add. Since these next state transition conditions do not overlap, the transition conditions are orthogonal.

24. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 24 State Transitions - 4 For MemberState completeness, the possible cases are D in BSet D not in BSet n = 1n > 1 n = 1 n > 1 Push Pop Add Subtract MemberState EmptyState Since these are all the possible cases, the MemberState transitions are complete.

25. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 25 State Transitions - 5 For MemberState orthogonality, there must be no overlap between the transitions. the transitions from MemberState to EmptyState occur when: Pop(n=1) or Subtract[(n=1) and (D in BSet)] the transitions from MemberState to MemberState occur when: Add + Push + Pop(n>1) + Subtract(D not in BSet) + Subtract(n>1) As is clear from the previous chart, these two cases have no condition in common and are thus orthogonal.

26. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 26 Class Exercise - 1 The SignOn object is to have the following methods Open - initiates a sign-on procedure LogOn - requests a name if the name is correct, PassWord requests the password if the PassWord is correct, SignOn terminates with the value true if there is any error, the program starts again at LogOn for any two errors, SignOn terminates with the value false

27. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 27 Class Exercise - 2 Construct the SignOn state template and state diagram. Check for hidden traps and loops. Check the states for completeness and orthogonality. Check the transitions for completeness and orthogonality.

28. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 28 Class Exercise - State Diagram StandBy Error Trial Open LogOn(No) (false) LogOn(No) LogOn(Yes) PassWord(Yes) (true) PassWord(No) LogOn(Yes) PassWord(Yes) (true) PassWord(No) (false) Start NameOk

29. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 29 Exercise - State Template StandBy StartOpen Start n = 0 n = 1 NameOk Error LogOn(Yes) LogOn(No) NameOk n = 2 ErrorPassWord(No) StandBy (true)Password(Yes) Error n = 3 TrialLogOn(Yes) StandBy (false)LogOn(No) Trial n = 4 StandBy (true)PassWord(Yes) StandBy(false) PassWord(No)

30. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 30 On the Difficulty of Verification The difficulty of verifying completeness and orthogonality of state machines depends mainly on whether conditions are used on transitions. If transitions are caused just by events, then verification will be straightforward. If transitions include conditions (“Pop occurred and n > 1”) then verification can be difficult.

31. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 31 Execution Tables - 1 An execution table is an orderly way to trace program execution. it is a manual check of the program flow it starts with initial conditions a set of variable values is selected each execution step is examined every change in variable values is entered program behavior is checked against the specification

32. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 32 Execution Tables - 2 The advantages of execution tables are they are simple they give reliable proofs The disadvantages of execution tables are they only check one case at a time they are time consuming they are subject to human error

33. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 33 Execution Table Procedure To use an execution table identify the key program variables and enter them at the top of the trace table enter the principal program steps determine and enter the initial conditions trace the variable values through each program step for repeating loops, add additional execution table steps for each additional loop cycle for long loops, group intermediate steps if their results are obvious

34. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 34 Execution Table Example - 1 Cycle 1 ClearSpaces(var Input: string; State: int) # 1 2 3 4 5 6 7 Instructions Condition Input Length State Length = length(Input) ‘ AB ‘ 50 if Length > 0true repeat(until State=3 or Length=0) if Input[Length-1] = ‘ ‘true Length = Length - 1 if State < 2 State=State+1 4 true 1 else State = 3 until State=3 or Length=0 false

35. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 35 Execution Table Example - 2 3 cycles are required before the until condition is satisfied. Just as with a test case, the execution table will only prove the case for this specific variable combination. Carefully check the initial conditions to ensure they are set by the program. Double check the execution table for errors and omissions.

36. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 36 Proof by Induction This method applies to Boolean functions with integer parameters. It states if f(n) is true for n = k and if when n = z where z > k and f(z) is true you can show that f(z+1) is true then f(n) is true for all values of n larger than k

37. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 37 Simplified Proof by Induction Goal: Prove that some statement f(n) which depends on a non-negative integer variable n is true for all values of n. Typical goal statement: This loop terminates and gives the correct result when the body has been executed n times. Proof by induction says that we have reached our goal if we can do two things: Demonstrate f(0); and Demonstrate f(k+1), where we are allowed to assume f(k).

38. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 38 Proof by Induction Example Example: for i=1 to Limit do xyz If you can show that this loop works for Limit = 0 and assuming it is true for some Limit = z, you can then show it is true for Limit = z+1* then you have shown that the loop works for all non-negative values of Limit. *The form of reasoning here: assuming the loop has worked so far, show it works this time.

39. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 39 A Proof by Induction Technique If you have a design element using variable x you verify that it works for x = k (x = 0 frequently the simplest choice) Next assume it works for some larger value x = z try to find a value of z where program behavior would be improper at z + 1 If you cannot, the proof is completed.

40. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 40 Factorial Example - 1 Prove that the following program produces N! Factorial(N) F = 1 for i = 1 to N F = F*i return = F N! (“N factorial”) = 1 * 2 *... * N 0! = 1

41. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 41 Factorial Example - 2 Show that when N=0, F = N! since 0 factorial is 1, this is correct Next, assume that when N = z, F = z! Finally, find any value of z where F(z) = z! and F(z+1) <> (z+1)! Such cases would only occur when the computing number system was exceeded computing capacity was exceeded

42. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 42 Assignment Previous reading: Chapter 10 Appendix B New reading: Chapter 12

43. Copyright © 1994 Carnegie Mellon University, 2002 James P Alstad CSCI511Personal Software Process - Design Verification 1 43 The Messages to Remember from Lecture 12 1. Your design review yield will significantly improve if you use disciplined design review methods. 2. The time spent verifying designs will be more than repaid by the testing time saved. 3. Practice verification techniques and select the most effective for finding the defects you most comonly find in testing.