Evaluation of Safety Critical Software -- David L. Parnas, -- A. John van Schouwen, -- Shu Po Kwan -- June 1990 Presented By Zhuojing Li

Outline Overview Why is Software a Special Concern Why is software used How is software different from other controllers Software testing concerns Software review concerns Modular structure Reliability assessment Conclusions

Overview Can we trust software? The authors present some important questions.

Why is Software a special concern? Software systems do not work well until they have been used and have failed repeatedly Errors are more common, and more troublesome Can be used in safety- critical applications. –Example: Military and civilian aircraft, nuclear plants, medical devices

Why is software used? More logic Easier to change More flexible

How are software controllers like other controllers Similarities: Both can be described as black box mathematical models.

How is software different from other controller technologies More complex More error sensitive –For example: sensitive to small errors –Conventional engineering is tolerant Harder to test Has correlated failures Lack of Professional standards

Software testing concerns  Cannot test software for correctness:  Testing can show the presence of bugs but it can not show that software is free of design errors.  Difficult to make prediction of software reliability and availability:  Impractical to measure the trustworthiness of software  Need for an independent agency

Software review concerns 1 st question: Why is reviewability a particular concern for software? –Each programmer has his/her own style. –Software structure and documentation were dismissed. –There is problem of clarity

Software review concerns 2 nd question: what reviews are needed? Correct intended functions. Maintainable, understandable, well documented structures. Each module to verify the algorithm and data structure design. Code for consistency with the algorithm and data structure design. Test completeness.

Software review concerns 3 rd question: What documentation is required to review the functional requirements? Description should use the mathematics of control systems.

Software review concerns 4 th question: What documentation is required to review the software structure? Three types of documents are required.  Requirement specification  Informal documents: Describing the responsibilities of each modules.  Module specification: Provides black box description of each modules

Software review concerns 5 th question: What documentation is required to review the module’s internal design? Description of the data structures that will be used. Description of the program functions and abstraction functions. Programs that cannot be described on a single page must be presented in a layered way.

Software review concerns 6 th question: What documentation is required to review the code?  Examine the correctness between the algorithms and the actual code  No need to examine the global design of the system

Software review concerns 7 th question: What documentation is required to review the test plan? -Reviewed by specialists, who compare it with requirement specification

Software review concerns 8 th question : Why is configuration management essential for rigorous reviews? –Complexity of software, and tremendous amount of documents.

Software review concerns Solution for 8 th question: Use online documentation instead of paper copies of the documents. Changes in the documentation should be notified by the computer system to all persons who have used the affected document. Online versions must be kept under strict control.

Modular structure Information Hiding Object-oriented programming Separation of Concerns Encapsulation Data Abstraction Principle: increase “ cohesion ” and reduce “ coupling ”

Reliability Assessment for safety–critical software All software failures would be predictable if we fully understand the software

Reliability Assessment for safety–critical software Software Measurements –Reliability: Probability of not encountering a sequence of inputs that leads to failure. –Availability: Fraction of time that the system is running and assumed to be ready to function.

Reliability Assessment for safety–critical software We cannot predict a software failure rate from failure rates for individual lines or subprograms The finite state machine model help us to determine the number of tests.

Reliability Assessment for safety–critical software Use of hypothesis testing: (1-1/h) N = M 1/h: the probability of failure in a test N: randomly selected tests M: the probability that an unacceptable product would pass our test.

Reliability Assessment for safety–critical software Three classes of program: –For memoryless batch program, use a randomly selected set of input data. –For a batch program with memory, a test is selected by choosing both input data and an internal state. –For real time system, use a multidimensional trajectory.

Conclusions Software can be used in certain safety- critical applications, but extreme discipline in design, documentation, testing and review is needed. Operating conditions and requirements should be well understood and fully documented the end