Formal Methods: Industrial Use CS 415, Software Engineering II Mark Ardis, Rose-Hulman Institute March 21, 2003

2 Outline  Controversy over formal methods  Where are formal methods used?  4 Stories  IBM CICS project  Tektronix oscilloscope  LOTOS at Bell Labs  VFSM at Bell Labs

3 Controversy Over Formal Methods  DeMillo, Lipton and Perlis "Social Processes and Proofs of Theorems and Programs", CACM, May  Fetzer "Program Verification: The Very Idea," CACM, September  The "Gang of 10"

4 Where are Formal Methods Used?  Safety critical applications  Aviation  Railway transportation  MOD  Other high-integrity systems  Application generators  Hardware design

5 IBM CICS Project  Maintenance of Customer Information Control System (CICS)  Used Z to reverse engineer old code  Found more errors earlier in the lifecycle

6 Maintenance of CICS  Old (> 30 years)  Large (>500 KLOC)  Multiple languages (assembler and special dialect of PL/I)  Many users  Several configurations

7 Restructuring of CICS  Necessary first step before Z could be used  Independent of any method

8 Reverse Engineering  Z specifications derived from:  manuals  developers  code  About half of CICS described in Z (230 KLOC)  Modules added or rewritten later from Z specifications

9 IBM Development Process  Used standard IBM process, including:  design reviews  code inspections  testing  Used standard IBM programming languages, plus guarded command language  Required training of staff in Z

10 IBM Training  Used standard IBM courses, including:  discrete mathematics  software engineering workshop  Augmented with Z courses  4 days for writers  2 days for readers  1 day for managers

11 IBM Results  More time spent in design  Inspections required less preparation, but took longer to conduct  More problems found earlier in design  Fewer problems found in testing  Overall time was 9% less than average  Won Queen's Award for productivity

12 Cartoon of the Day

13 Tektronix  Exploratory project  Discovered useful abstractions  Concentrated on process of specification, not product

14 Tektronix Process  2 researchers (DeLisle and Garlan) investigated general problem area:  talked to engineers  tried to describe existing devices  Discussed trial specifications with engineers

15 Tektronix Results  Original descriptions were operational  Researchers found an abstraction (waveform) that clarified roles of hardware and software engineers  Resulting specification yielded insights about tradeoffs:  user interfaces  sampling methods  hw/sw partitioning

16 Tektronix Lessons  Industrial engineers can understand formal specifications  Abstraction was very valuable in focusing attention on right problem  Specification was a process, not a product

17 LOTOS at Bell Labs  Some formal methods used in switching applications  SDL  Promela  VFSM  Opportunity to try LOTOS in 1991  Language Of Temporal Ordering Sequences  New standard for telecommunication protocols

18 Primitive LOTOS Project  Basic LOTOS difficult to use  too much redundancy  too little redundancy  Primitive LOTOS (PLOTOS)  added declarations  more "C"-like

19 PLOTOS Results  Used on parts of several projects  Tools were popular  Solved the wrong problem  specification was a verb, not a noun  spaceship theory

20 PLOTOS Lessons  Software developers in Naperville are an oral culture  work via meetings  very little abstraction  Need to first move to literary paradigm  domain engineering to capture knowledge in writing  domain specific languages to develop formal notations

21 VFSM at Bell Labs  Manager convinced by a former teacher to try Virtual Finite State Machines (VFSM)  Constructed a compiler to C  Later adapted SPIN for model checking

22 VFSM Results  Used on several projects  Tools were popular  Solved the right problem  compiled to executable code  testing was the most onerous job of development

23 VFSM Lessons  Bottom-up development is more easily accepted than top-down  Free lunches are a powerful force  Revolutionary methods need crusaders

24 Summary  Formal methods provide substantial benefits, but at cost  May be most applicable in established domains  Adoption requires cultural change for many organizations