1. The Cleanroom Approach to Quality Software Development
(or ZeroDefict Software is really possible!)
References:
The Cleanroom Approach to Quality Software Development, Michael Dyer, Wiley 1992,
IEEE Software paper by Linger in March 1994 issue

2. An analogy
Compare sight typing with touch typing (In sight typing, the learner looks at the keys.)
Touch typing takes longer to learn but results in increased productivity with higher quality of typing in the long run.
Sight typing requires the operator to switch between the text and keys and mistakes and lapses are more common.
Compare this to:
The difference between Programming & Debugging and Specification & Refinement.
The latter takes longer to learn but the payoff is increased productivity and quality.

3. Key features of Cleanroom
Serious programming begins after formal specification, there is emphasis on more explicit design and verification from the specification.
There is no programmer testing.
The Cleanroom approach combines mathematical reasoning during specification and design refinement to code with statistical reasoning during test case generation and testing.
The aim is to produce zero defect software (or minimal defect software).

4. Cleanroom Successes - proven in use since the 1980s
The US 1980 Census System
Controlled by 25kloc program which operated its entire 10 months in production with no failures observed.
The IBM Wheelwriter Typewiter Systems (1984)
A 65 kloc program with millions of users and no failure ever detected.
The US Space Shuttle Software
Over 500 kloc while not completely zero defect, has been zero defect in flight.

5. Improved Specification and testing
Cleanroom is the first practical attempt
to place software development under statistical quality control, and
to deliver software with a known and certified meantime to failure (MTTF)
The key practices are:
to use formal specification and verification methods to create software of sufficient quality to forego programmer testing (ie unit test/debug) of code and
to require statistical based testing for evaluating software reliability
The pay off is that Cleanroom statistically based testing with random sampling driven from input probability distributions has been shown to be highly effective at finding errors with high failure rates. (It is better at finding the errors that occur most often.)

6. Cleanroom Process Flow (overview)
Software requirements specification
Incremental software delivery
Software
design and
development
Incremental statistical testing and regression testing
Action
Statistical Control
Process Error
Diagnosis and
Correction
Software reliability measurement
Basis for FEEDBACK
Basis for Level 5 Process Improvement
Incremental approach based on independent specifications allows parallel development if required

7. The Cleanroom Process Model (in more detail)
(stacked boxes indicate successive increments)
Customer requirements
Specification
Functions
Usage
Incremental Development Planning
Usage Specification
Functional Specification
Formal design correctness verification
Statistical test-case generation
Statistical testing
Feedback of improvements
Quality certification model
MTTF Estimates

8. TABLE 1.1 Cleanroom Component Techniques
Technology Cleanroom
Focus Perspective
Baseline Capability Defined process Starting point
Design and inspection Early quality visibility
Software Specification Software quality Focal point
Formal description Software correctness Drive verification
Usage/build data Customer acceptance Drive validation
Software Verification Software quality Software quality
In construction Error prevention Correct designs
In inspection Confirmed correctness Zero defect
No Developer Testing Software acceptance Software productivity
Statistical Testing Customer acceptance Requirements validation
MTTF Prediction Software reliability Certified MTTF
Statistical Process Process improvement Software warranty

9. FIGURE 1.2 Roadmap for Introducing Cleanroom Component TechniquesB A S E L I N P R O C
Formal Specifications
Correctness
Verification
S/w Configuration
Management
No developer testing
Process Control
Continuous
inspection
Statistical
testing
MTTF
measurement

10. Table 1.3 Trends in Software Quality
Trends in decreasing defect Rates based on improving development towards full use of Cleanroom methods
Table Trends in Software Quality
Total Defect Rate Postdelivery Rate
Traditional Development to to 10
Unstructured design
Only testing for detection
Baseline Development to to 4
Structured programming
Formal inspections
Advanced Development to to 1
Correctness verification
Formal specification
Statistical testing
[IBM results cited in Dyer.]

11. Finally more recently reported results from mid90s
IBM Cobol/SF
Size: 85 Kloc of PL/l
Testing Error Rate: errors per kloc
Productivity Rate: Loc/person month
NASA satellite-control project
Size: 40 kloc of FORTRAN
Testing Error Rate: errors per kloc
Productivity Rate: loc/person month
IBM 3090E tape drive
Size: 86 kloc of C
Testing Error Rate: errors/kloc*
(N.B. comparison with Unit Testing)
Erisesson Telecom 0532 Operation System
Size: kloc
Testing Error Rate: error/kloc
[in Linger 94] (sample)

12. Summary of Cleanroom Impacts on the SLC
1. Requirements Specification
Function and Performance but with Usage
Probabilities and Build Strategy
2. Software Design/Implementation
Incremental Software Development but with
Correctness verification not Unit Test
3. Independent Software Test
Integration and Test of Released Increments
but with Representative Statistical Usage Samples
4. Software Acceptance
Demonstrated Function and Performance
but with Certified Software MTTF