Software Engineering for Safety : A Roadmap Presentation by: Manu D Vij CS 599 Software Engineering for Embedded Systems

Introduction Wide Spread Use of Safety - Critical Systems and their reliance on Software Wide Spread Use of Safety - Critical Systems and their reliance on Software “The Nation depends on fragile software” “The Nation depends on fragile software”

Keys Areas in SE for Safety Hazard Analysis Hazard Analysis Safety requirements specification and analysis Safety requirements specification and analysis Designing for safety Designing for safety Testing Testing Certification and Standards Certification and Standards Resources Resources

Hazard Analysis Core of the development of safe systems Core of the development of safe systems Identification and Analysis: Identification and Analysis: –Criticality –Likelihood of Occurrence Which Hazards to avoid ? Which Hazards to avoid ? Determination of s/w components that can contribute or prevent hazard Determination of s/w components that can contribute or prevent hazard Safety requirements and constraints on design of system Safety requirements and constraints on design of system

Safety Requirements Specification and Analysis Formal Specification: Formal Specification: –Ease and Accuracy –Investigate if safety properties are preserved –Automated check availability Interactive theorem provers, Model checkers Interactive theorem provers, Model checkers Safety  Software Requirements Requirements Safety  Software Requirements Requirements SpecTRM Embedded Systems SpecTRM Embedded Systems

Designing for Safety Design for Safety Design for Safety –Prevention –Detection and Control Design Trade Offs Design Trade Offs –Safety Vs Other features e.g. Fault Tolerance –Issues involved are moral, legal, finance…… Vulnerability to simple design errors Vulnerability to simple design errors –Tendency to neglect small errors –“Small Errors have small consequence”– not in Software Limited use of known design techniques Limited use of known design techniques –Good design techniques are ignored

Testing Critical for safe system in: Critical for safe system in: –Development –Certification Assumptions about: Assumptions about: –Environment –Users –Operations Is it enough ? Is it enough ?

Certification and Standards Certification Certification –More complicated –Less well defined Standards Standards –“What standards are appropriate for large, safety-critical systems composed of subsystems from different domains” –Problems: Lack of guidance in existing standards Lack of guidance in existing standards Poor integration of software issues with system safety Poor integration of software issues with system safety Heavy burden of making a safety case for certification Heavy burden of making a safety case for certification –Recommendations: Classifying and evaluating standards according to products, process and resources Classifying and evaluating standards according to products, process and resources Constructing domain specific standards for products Constructing domain specific standards for products

Resources Books…. Levenson Books…. Levenson Bowen’s website…. Publications, Conferences, RISKforum Bowen’s website…. Publications, Conferences, RISKforum IEEE video IEEE video

Directions for future Work Integration of informal and formal methods Integration of informal and formal methods –Key Areas: Automatic translation of informal notation into formal models Automatic translation of informal notation into formal models Lightweight formal methods Lightweight formal methods Integration of previously distinct formal methods. Integration of previously distinct formal methods.

Fault Trees hazard events represented by nodes hazard events represented by nodes AND/OR gates AND/OR gates domino effect domino effect errors in the requirements phase errors in the requirements phase example taken from: man/des_s99/safety_critical/

Directions…………. Constraints on safe product families and safe reuse Constraints on safe product families and safe reuse –Two key research areas: Safety Analysis of product families Safety Analysis of product families –A Goal Safety reuse of COTS software Safety reuse of COTS software –Two Problems

Directions…….. Testing & Evaluation Testing & Evaluation –Requirements-based testing –Evaluation from multiple sources –Model consistency –Virtual environment simulations Runtime monitoring Runtime monitoring

Directions….. Education: Education: –Scientific rather than methodical courses –Textbooks –Awareness Related Fields Related Fields –Security & Survivability Techniques quite similar Techniques quite similar Security Vs Safety Security Vs Safety –Software Architecture Safety consequence of flexible architectures Safety consequence of flexible architectures Evaluation of architectures for safety critical product families Evaluation of architectures for safety critical product families Partitioning to control hazards enabled by shared resources Partitioning to control hazards enabled by shared resources Architectural solutions to the need for “techniques that augment the robustness of less robust components Architectural solutions to the need for “techniques that augment the robustness of less robust components –Human Factors Engineering Better understanding of usage patterns, and formal specification of operator's mental model can yield more accurate safety requirements and safer maintenance. Better understanding of usage patterns, and formal specification of operator's mental model can yield more accurate safety requirements and safer maintenance. More research needed. More research needed. –Other fields Domain specific design for fault tolerance Domain specific design for fault tolerance Advances in OS, Programming Language Advances in OS, Programming Language

Conclusion.... Safety is system problem Safety is system problem Advancement in the other fields will enhance safety Advancement in the other fields will enhance safety Protecting devices….Are they enough ?? Safety into system! Protecting devices….Are they enough ?? Safety into system! Probabilistic risk assessment is not enough!! Probabilistic risk assessment is not enough!! Advancement is safety analysis is required Advancement is safety analysis is required