July 18, 2001 Mission Success Begins With Safety Quality Leadership Forum Software Quality Assurance at GSFC Dr. Linda H. Rosenberg Chief Scientist for Software Assurance Office of Systems Safety and Mission Assurance

July 18, 2001 Mission Success Begins With Safety Discussion Areas NASA Software Activities Software Working Group NASA Software Initiative Implementation Plan GSFC Software Assurance Activities Software Process Improvement – CMMI Software Safety and Reliability Software Quality Metrics IV&V

July 18, 2001 Mission Success Begins With Safety

July 18, 2001 Mission Success Begins With Safety SWG Initial Tasks 1 - Define criteria for use of IV&V on a project 2 - Standards evaluation –Review IEEE for potential NASA use –Review draft of NPG 2820 –Review draft NPD for IV&V 3 - Prepare a plan for improving software process –Implementation of software metrics program –Implementation of process improvement model –Establishment of Center Software Engineering Process Groups (SEPG)

July 18, 2001 Mission Success Begins With Safety SWG Task 3 - Metrics Set of metrics finalized summer 2000 Objectives Provide project managers with usable information Provide agency with information on software trends Provide a measure to assess improvement 2 test projects per Center started Fall 2000 for 1 year (GSFC projects AURA & AQUA) Developing database for metrics entry and analysis

July 18, 2001 Mission Success Begins With Safety NASA Software Initiative Implementation Plan Goal: Advance software engineering practices (development, assurance, and management) to effectively deliver the scientific and technological objectives of NASA. Strategies: 1.Develop and implement Agency-wide and Center plans for continuous software process and product improvement in NASA and Contractor developed software; also establish infrastructure and measurement system 2.Improve safety, reliability, and quality of software products through the integration of sound software engineering principles and standards. 3.Provide input for research based on identified software problem areas and infuse research results 4.Improve software engineering knowledge base in NASA, and implement strategies for attracting, retaining software engineers

July 18, 2001 Mission Success Begins With Safety

July 18, 2001 Mission Success Begins With Safety GSFC Software Development Process Improvement Purpose - improving the processes and practices in use at GSFC using the Capability Maturity Model Integrated (CMMI) levels of maturity (ML) as a measure of progress. Scope - process improvement effort that will be undertaken with the goal of raising GSFC from its current state to a CMMI Defined maturity level (L3). All projects defined by NPG or otherwise identified by GSFC’s Center Director will participate in this effort.

July 18, 2001 Mission Success Begins With Safety Capability Maturity Model Integrated (CMMI) LevelProcess Areas Organization innovation and deployment Causal analysis and resolution Organizational process performance Quantitative project management Requirements development Technical solution Product integration Verification Validation Organizational process focus Organizational process definition Organizational training Integrated project management Risk management Decision analysis and resolution Requirements management Project planning Project monitoring and control Supplier agreement management Measurement and analysis 5 Optimizing 4 Quantitatively Managed 3 Defined 2 Managed 1 Initial Software Development SW Systems SE Software Acquisition SA CMMI For Pilots: Emphasis - SW CMM As appropriate - SE CMM SA CMM GSFC GOAL

July 18, 2001 Mission Success Begins With Safety Pilot Project Selection Project W FLT SWGND SW Instr1 2 Project X FLT SW GND SW Instr1 2 Project Y FLT SW GND SW Instr1 2 Project Z FLT SWGND SW Instr1 2

July 18, 2001 Mission Success Begins With Safety Schedule GSFC Implementation plan to HQ July 2001 Management Oversight Group Member identification July 2001 Initial meeting August 2001 Training in CMMI September (1/2 day or 3 day option) Engineering Process Group Member identification July 2001 Training in CMMI September 2001(3 day course) Training in Risk Management October 2001 Pilot Implementation Pilot identification by October 1, 2001 Pilot study complete October 1, 2002 Evaluation of Pilot and roll out January 1, 2003

July 18, 2001 Mission Success Begins With Safety What is meant by “safety” A system/product is Safe when: There is little to no chance for it to blow up, break, malfunction, or otherwise fail in such a way as to potentially injure someone Something is Critical when there is a potential for: Serious injury or death Serious impact to the bottom line, or Bad publicity, public reputation Vital information is accessible to the wrong folks A system/product is Not Safe when: Someone could die or be seriously injured NASA includes possible destruction of vital equipment as well

July 18, 2001 Mission Success Begins With Safety Standards NASA Standards ( NPG NASA Safety Manual NSTS B Safety Policy and Requirements for Payloads (Shuttle and ISS) NASA-STD ANASA Software Safety Standard NASA-GB-A302 Software Formal Inspections Guidebook NSTS Methodology for Conduct of Space Shuttle Program Hazard Analyses SSP Computer-Based Control System Safety Requirements, ISS Program NPD/NPG 8730 “ NASA IV&V Processes “ IEEE Standards IEEE Information Technology - Software Life Cycle Processes IEEE Recommended Practice for Software Requirements Specifications IEEE Recommended Practice for Software Design Descriptions IEEE Standard for Software Safety Plans Other Standards MIL-STD-882D System Safety Program Requirements (C version January 19, 1993) DO-178B Software Considerations in Airborne Systems and Equipment Certification (Federal Aviation Administration). ISO Guidelines For The Application Of ISO 9001 To The Development, Supply, Installation And Maintenance Of Computer Software

July 18, 2001 Mission Success Begins With Safety Software Reliability The probability that software will not cause the failure of a system for a specified time under specified conditions. The probability is a function of the inputs to and use of the system, as well as a function of the existence of faults in the software. The inputs to the system determine whether existing faults, if any, are encountered. [AIAA] [IEEE 982] IEEE Software Reliability Management : “The process of optimizing the reliability of software through a program that emphasizes software error prevention, fault detection and removal, and the use of measurements to maximize reliability in light of project constraints such as resources, schedule and performance.”

July 18, 2001 Mission Success Begins With Safety Hardware vs. Software Reliability Burn in Useful Life Wear out Integration Useful Life Obsolete & test Hardware Failure RateSoftware Failure Rate Hardware reliability == Software reliability

July 18, 2001 Mission Success Begins With Safety Definitions: Safe vs. Reliable A system is: safe if it doesn’t kill anyone, or the system itself, while either performing its normal operations or, when unable to perform correctly, “fails-safe”. reliable if it performs the required functions within specified parameters/environment and within predicted working timeframe consistently Some consider Software to be very reliable, in that it does just what its programmed to do, over and over and over again. It doesn’t wear out or ‘break’. However, Linda will give you the real picture on Software reliability!

July 18, 2001 Mission Success Begins With Safety Requirements How Do We Assure / Measure Software Quality? Design Coding Testing Can you test the requirements? Are the requirements complete? Are you testing each requirement? How much testing is necessary? How hard is it to fix the components? Can I reuse any components? What is the quality and complexity of the code? How many errors still remain? What is the reliability? When can I stop testing?

July 18, 2001 Mission Success Begins With Safety IV&V Approach Req Design Code Test (Verification & Validation) Unit IntegrationAcceptance Req Design Code Testing Unit Test (Verification & Validation) Integration Acceptance Clean Room Approach Traditional Software Development V&V iV&V Req Design Code Test (Verification & Validation) Unit IntegrationAcceptance IV&V Implementation IV&V