1. JOINT SERVICES SOFTWARE SYSTEMS SAFETY HANDBOOK Michael L. Brown, Chairperson JOINT SOFTWARE SYSTEMS SAFETY COMMITTEE (JSSSC) March 2001 presented to the Safety Critical Systems Club

2. INTRODUCTION This presentation provides an overview of the Software Safety Handbook and its Status. The following topics are addressed: H Purpose H Background H Handbook Layout and Contents H Software Systems Safety Processes H Applicability H Project Status H Additional tasks H Recommendations

3. Provide management and engineering “how-to” guidelines to achieve a reasonable level of assurance that software will execute in a system context with an acceptable level of safety risk. Initial process and methodology based on Independent Software Nuclear Safety Analysis process tailored to conventional systems and experience from many programs. Process successfully applied to wide range of systems PURPOSE

4. Background Inconsistent processes – Most incomplete – Many good points but not tied together well Lessons learned – SSSTRP – Took good points from each process – Developed comprehensive systems engineering based software systems safety process

5. Handbook Sections Introduction to System Safety Introduction to the Handbook Executive Summary 4.0 3.0 2.0 1.0 Appendices Software System Safety HPurpose, Scope, Authority, Overview and Handbook Organization HMotivation and Direction for the Program Director and Program Manager HOverview of System Safety and Safety Risk Management HDefinition, References, Supplemental Information, Generic Guidelines, Sample Documents, Lessons Learned, Synopsis of Analysis Techniques and Methods, and Agency Specific Information HPlanning, Task Implementation, Risk Assessment and Acceptance, Configuration Management and Reusable Software HANDBOOK LAYOUT & CONTENTS

6. System and Software Engineers Written for all members of safety team Purpose and scope of handbook Authority for software safety program requirements Organization of handbook

7. Software Safety Engineer How-to guidance on establishing and conducting software systems safety program Process based on DODI-5000.1 and DOD- 5000.2R System Acquisition Process and Requirements, MIL-STD-498/IEEE STD 1498/12207 Software Development Process, MIL- STD-882 Standard Practice for System Safety, and NATO Standardization Agreement requirements

8. Section 4.0 Main part of the document – “How-to” Guidance Complete – However, needs expansion in certain areas (e.g., CDI in safety critical applications) Covers entire software systems safety process from concept to system retirement Refers reader to examples in appendices and reference documents

9. Software Systems Safety Process Program management – Planning through execution Requirements derivation – Generic Requirements from Lessons Learned – System Specific Safety Requirements from system level analyses Requirements verification and validation – Detailed analyses – Safety Testing Safety Assessment

10. SWS Processes Program Phases & Milestones

11. SWS PROCESSES Software Safety Requirements Derivation Develop Generic Safety Critical Software Guidelines & Requirements  Obtain Generic Software Safety Requirements Lists  Tailor Generic Software Safety Requirements/Guidelines List for the specific system and/or subsystem  Categorize and Prioritize Generic Software Requirements/Guidelines Derive Functional Safety Critical Requirements  Develop Safety Critical Functions List  Develop Potential Functional Hazard List Preliminary Hazard List (PHL) Preliminary Hazard Analysis (PHA)  Categorize & Prioritize System Functional Hazards  Determine System Level H/W, S/W and HF Causal Factors  Execute System Level Trade Study  Begin Determining all of the Software Specific Causal Factors  Begin Software & Architectural Detailed Design Trade Study Derive System-Specific Software Safety Critical Requirements Requirements Hazard Analysis/Software Criteria Requirements Analysis  Tag Safety-Critical Software Requirements  Establish Methods for Tracing Software Safety Requirements to Test  Provide Evidence for Each Functional Hazard Mitigated by Comparing to Requirements  Verify Software Developed IAW Applicable Standards and Criteria

12. SWS PROCESSES Software Safety Program Planning - Customer Acquisition Policy OCD/OR/MENS DOP Proposal Safety Policy Generic Requirements Lessons Learned PHL Inputs (Later Milestones) Draft PHA Draft TEMP Draft SEMP Draft ILSP Draft CRLCMP Draft SSS Draft S/SDD Draft SOW/RFP Software Safety Program Planning Procuring Agency - Customer Define Acceptable Level of Risk (HRI, Acceptance Authority, Risk Assessment Methodology) Establish System Safety Program: –Identify and Establish Interfaces to Other Program Disciplines –Identify Contract Requirements –Develop POA&M –Establish Safety Data Library –Establish Hazard Tracking Process –Resource Requirements Determination: »Analyses Required »Tests Required (U) »Resources Required –Develop Safety Statement of Work: (U) »Define Safety Program Requirements (U) »Define Safety Reporting Requirements (U) »Define Safety Review Requirements (U) –Proposal Evaluation »Develop Evaluation Criteria »Evaluate Proposals Input to SOW/SOO Input to RFP Safety Proposal Evaluation Plan Safety POA&M Review Requirements System Safety Management Plan System Safety Program Plan with SwSPP Appendix SSWG Charter and Subgroups Input to SDP Input to TEMP Input to SEMP Input to ILSP Input to PHL Input to PHA Input to CRLCMP Program InitiationSafety Program Management Program Initiation PM, Principal for Safety SE, SWE, SWSE (EVAL) SSWG (Later Milestones) Identify and Establish System Safety Program Tasks and Requirements Purpose: Primary Sub-Processes Inputs (Suppliers) Outputs (Customers) Players Exit ObjectiveEntry Criteria References in Addition to ¶ 4.1.1 Comments: Related Sub-Processes Proceeding ProcessNext Process Service Specific Guidance IEEE 1228 FAR’s System Safety Program Management Plan Proposal Evaluation and Acceptance Established System Safety Program Program Planning and Management System Safety Management Plan development

13. SWS PROCESSES Preliminary Hazard Analysis Preliminary Hazard Analysis - (PHA) Identify System Level Causal Factors (I) Identify Software Level Causal Factors (I) –Apply Analysis Techniques (for Example SFTA’s) (I) –Develop recommendations to minimize software induced hazards (I) Apply HRI and Prioritize Hazards (I) Apply Risk Assessment Criteria (Categorize) Hazards (I) Link hazard causal factors to requirements (I) Develop design recommendations to mitigate hazards (I) RHA/SCRA Inputs PHA Update Input to S/W Design Inputs to S/W Development Plan Input to Preliminary Software Design Analysis SSHA Input to Trade-Off Studies Design Implementation Recommendations HARs Inputs to Software Test Plans –Test Requirements Input to SPRA Reviews Prioritized Hazard List Input to SSS, SRS, S/SDD, IRS (I) Input to ECP, SENs, PIPs (I) SOW/SOO/RFP Risk Assessment Criteria, HRI Draft SSS, S/SDD Lessons Learned: –Analyses on Similar Systems –Incident Reports –Previous Mishap Causes Life Cycle Environment Profile PHL Tailored Generic Software Safety Requirements List Inputs Later Milestones) HARs ECPs Safety Data Library (SDL) Maintenance System Level Trade Studies Software Level Trade Studies Tailor The Generic Software Safety Requirements List Functional Hazard List Development Program Planning/Management SwSWG, Domain Experts IEEE 1498 Derived System Specific Software Safety Critical Requirements Preliminary Software Design Analysis Upon Completion of PHL Completion of hazard categorization, prioritization, and determination of all causal factors (initial drafts) Resolution of identified hazards (completion) Identification, Classification and Tracking of System Level Software Hazards Purpose: Primary Sub-Processes Inputs (Suppliers) Outputs (Customers) Players Exit Objective Entry Criteria References Comments Related Sub-Processes Proceeding ProcessNext Process References in Addition to ¶ 4.1.1

14. Handbook Charts

15. Developed Outline Used Process Charts as basis Covered all items in chart Assigned specific sections to primary authors Assigned secondary authors to each section

16. Authors Selected for particular expertise Sources sought from each service, industry, and academia Coordinating author – Selected for expertise in field – Provided “common voice” through out handbook

17. Authors Michael Brown, NAVSEA Dahlgren John Bozarth, EG&G Janet Gill, NAWC AD Brenda Hyland, NAWC AD Archibald McKinlay – BAH Lenny Russo, CECOM Coordinating Author – Steve Mattern, SEA

18. Task Assignments Based on author’s area of expertise – Prior experience – Interest in topical area Secondary author – Prior experience – Area of Expertise

19. Applicability Process applicable to wide range of military and non-military systems – Weapon Systems – Fire Control and Guidance Systems – Operational Flight Control Programs – Any system containing safety critical software Handbook provides tailoring guidance for wide range of programs

20. Status Submitted draft to community for review and comment October 1997 Received and collated comments Comments adjudicated by committee Developed revisions based on comments and additional input from authors Forward revised draft to reviewers Finalized handbook in November 1999

21. Collating comments Comments are annotated next to applicable paragraph – Source identified – Comment as provided entered – Rationale for acceptance, rejection, or modification provided. Handbook with comments placed on web page

23. PROJECT STATUS - Mar 2001 First Publication - 31 December 1999 – Handbook text complete All topical areas addressed Some areas require additional work – Appendices approximately 75% complete Need to develop additional examples for guidance –Based on previous programs and lessons learned Need to incorporate additional reference documents

24. Funding Sources to date Joint Systems Engineering Steering Group – Funding source: Army Naval Ordnance Center – WSESRB USAF HQ-AFSC – Lt. Col. Alberico Naval Facilities Engineering Command Personal time by authors

25. Additional Tasks Commercial Developed Items, Government off- the-shelf (legacy) items, and Non- Developmental Items (Hardware and Software) in safety critical applications – Selection criteria – Design requirements and guidelines – “How to” guidelines on influencing system and software architecture, design, and implementation – “How-to” analysis and testing guidelines – Configuration Control Requirements

26. Additional Tasks Software Safety Risk Assessment – Relating software causal factors and control categories to Hazard Risk Indices – Relating software metrics to hazard risk assessment – Guidelines for assessing adequacy of safety program efforts

28. Additional Tasks Guidelines for selection and/or implementation of: – Language – Operating Systems – Operating Environments – Middleware – Pros and Cons for each language, OS, OE, etc. – Selection and implementation guidelines and criteria – Analysis and testing guidelines – Caveats and requirements for each language, etc. – Configuration control

29. Additional Tasks “Software like” hardware: FPGAs, PLDs Define safety assessment process and develop guidelines

30. FUTURE OBJECTIVES Hypertext CD-ROM Integrate into DoD Acquisition Deskbook Software Safety WWW Home Page Revisions & updates to handbook On-Site Software Systems Safety Training System Safety Handbook Tool Development

31. Conclusions The JSSSH provides a comprehensive, Systems Engineering based approach to ensuring that software executes safely within the system context. The process is designed for application to a wide range of systems without the need for highly specialized expertise (e.g., formal methods) The JSSSH provides a basis against which to evaluate the thoroughness of Software Systems Safety Programs The JSSSH is a useful guideline for any safety critical system