Cleanroom Method CS 415, Software Engineering II Mark Ardis, Rose-Hulman Institute March 20, 2003

2 Outline 1. Harlan Mills 2. Cleanroom method 3. Industrial use of cleanroom

3 Harlan Mills

4 Mathematics and Programming Roman accounting "to go from programming as an instinctive, intuitive process to a more systematic, constructive process"

5 Cleanroom Method Incremental (spiral) Box structure specification and design Design verification No debugging Statistical testing

6 Box Structures Black boxes: behavior only State Boxes: behavior + state Clear boxes: procedures

7 Black Boxes S 1 S 2...S n R stimulus history  response

8 State Boxes SR stimulus, old state  response, new state State Data

9 Clear Boxes SR stimulus, old state  response, new state State Data Procedures

10 Box Description Language (BDL) Invocation: use Sequence: do B 1 ; B 2 od Alternation: if then B 1 else B 2 fi Iteration: while do B od

11 Box Structure Hierarchy BB SB CB BBBBBB SBSBSB CBCBCB

12 Cartoon of the Day (1/3)

13 Cartoon of the Day (2/3)

14 Cartoon of the Day (3/3)

15 Design Verification Procedures in BDL are checked for correctness with their higher-level descriptions All boxes (and all procedures) describe functions Formal proofs of correctness can be performed (but often informal proofs are done, instead)

16 Verification of Sequence Given a high-level function [f] for statement: do [g]; [h] od Does [g] followed by [h] compute the same function as [f] ? Example: [f](x) = 2 * x + 7 [g](x) = 2 * x [h](x) = x + 7

17 Verification of Selection Given a high-level function [f] for statement: if then[g]else[h]fi 1. Whenever is true, does [g] compute the same function as [f] ? 2. Whenever is false, does [h] compute the same function as [f] ?

18 Verification of Iteration Given a high-level function [f] for statement: while do[g]od 1. Whenever is true, does [g] followed by [f] compute the same function as [f] ? 2. Does the loop always terminate? 3. Whenever is false, does the empty function compute the same function as [f] ?

19 Usage Testing Develop an operational profile of use Generate random tests that fit the probabilities

20 Example Function Usage Probability Distribution Interval Update32%0-31 Delete14%32-45 Query46%46-91 Print8%92-99

21 Test Generation Test Random NumbersTest Cases 129, 11, 47, 52, 26, 94 U, U, Q, Q, U, P 262, 98, 39, 78, 82, 65 Q, P, D, Q, Q, Q 383, 32, 58, 41, 36, 17 Q, D, Q, D, D, U 436, 49, 96, 82, 20, 77 D, Q, P, Q, U, Q

22 Industrial Use Used in a few areas of IBM Used by some military contractors Tried at NASA

23 Software Engineering Laboratory (SEL) Joint program of NASA Goddard Space Center, Computer Sciences Corporation, and the University of Maryland Conduct experiments and case studies on new software technology

24 SEL Experience First trial at University of Maryland controlled experiment (10 experiment teams, 5 control teams FORTRAN 1.5 KLOC 3 case studies at Goddard flight-dynamics ground support systems FORTRAN 40 KLOC, 22 KLOC, 160 KLOC

25 SEL Results – University Experiment Cleanroom teams use fewer computer resources satisfy requirements more successfully make higher percentage of scheduled deliveries

26 SEL Results – Goddard More effort spent in design Better reliability of final product Smaller projects achieve higher productivity, but large project just average

27 Summary Cleanroom may be an effective method for achieving higher reliability Requires some culture change (no debugging) Still being investigated by researchers and practitioners

28 References Victor Basili and Scott Green, "Software process evolution at the SEL", IEEE Software 11(4), 58-66, July 1994.