1 The Role of the Revised IEEE Standard Dictionary of Measures of the Software Aspects of Dependability in Software Acquisition Dr. Norman F. Schneidewind.

Slides:



Advertisements
Similar presentations
Develop an Information Strategy Plan
Advertisements

Software Quality Assurance Plan
Lecture # 2 : Process Models
The software process A software process is a set of activities and associated results which lead to the production of a software product. This may involve.
ISNE101 Dr. Ken Cosh. Recap  We’ve been talking about Software…  Application vs System Software  Programming Languages  Vs Natural Languages  Syntax,
©2006 OLC 1 Process Management: The Foundation for Achieving Organizational Excellence Process Management Implementation Worldwide.
Overview Lesson 10,11 - Software Quality Assurance
Software Quality Metrics
SE 555 Software Requirements & Specification Requirements Management.
MSIS 110: Introduction to Computers; Instructor: S. Mathiyalakan1 Systems Design, Implementation, Maintenance, and Review Chapter 13.
1 Introduction to System Engineering G. Nacouzi ME 155B.
Software Verification and Validation (V&V) By Roger U. Fujii Presented by Donovan Faustino.
Capability Maturity Model
Acquiring Information Systems and Applications
QUALITY MANAGEMENT SYSTEM ACCORDING TO ISO
Software Project Management
8/27/20151NeST Controlled. 2 Communication Transportation Education Banking Home Applications.
1 NASA OSMA SAS02 Software Reliability Modeling: Traditional and Non-Parametric Dolores R. Wallace Victor Laing SRS Information Services Software Assurance.
Introduction to ISO New and modified requirements.
1 Building and Maintaining Information Systems. 2 Opening Case: Yahoo! Store Allows small businesses to create their own online store – No programming.
Introduction to Software Quality Assurance (SQA)
Software Project Management Lecture # 8. Outline Chapter 25 – Risk Management  What is Risk Management  Risk Management Strategies  Software Risks.
Introduction to RUP Spring Sharif Univ. of Tech.2 Outlines What is RUP? RUP Phases –Inception –Elaboration –Construction –Transition.
University of Palestine software engineering department Testing of Software Systems Fundamentals of testing instructor: Tasneem Darwish.
CLEANROOM SOFTWARE ENGINEERING.
N By: Md Rezaul Huda Reza n
-Nikhil Bhatia 28 th October What is RUP? Central Elements of RUP Project Lifecycle Phases Six Engineering Disciplines Three Supporting Disciplines.
1SAS 03/ GSFC/SATC- NSWC-DD System and Software Reliability Dolores R. Wallace SRS Technologies Software Assurance Technology Center
1 Using Excel to Implement Software Reliability Models Norman F. Schneidewind Naval Postgraduate School 2822 Racoon Trail, Pebble Beach, California, 93953,
Software Project Management Lecture # 8. Outline Earned Value Analysis (Chapter 24) Topics from Chapter 25.
Unit 8 Syllabus Quality Management : Quality concepts, Software quality assurance, Software Reviews, Formal technical reviews, Statistical Software quality.
OHT 23.1 Galin, SQA from theory to implementation © Pearson Education Limited 2004 The benefits of use of standards The organizations involved in standards.
Chapter 13: Developing and Implementing Effective Accounting Information Systems
PPMT CE-408T Engr. Faisal ur Rehman CED N-W.F.P UET P.
Quality Control Project Management Unit Credit Value : 4 Essential
Service Transition & Planning Service Validation & Testing
How To Build a Testing Project 1 Onyx Gabriel Rodriguez.
1 IEEE P1633\AIAA R013A Recommended Practice on Software Reliability Status Report Norm Schneidewind, Naval Postgraduate School (chair)
An Introduction to Software Engineering. What is Software?
Software Project Management Lecture # 10. Outline Quality Management (chapter 26)  What is quality?  Meaning of Quality in Various Context  Some quality.
Software Engineering Saeed Akhtar The University of Lahore Lecture 8 Originally shared for: mashhoood.webs.com.
© Mahindra Satyam 2009 Decision Analysis and Resolution QMS Training.
University of Sunderland CIFM03Lecture 2 1 Quality Management of IT CIFM03 Lecture 2.
Systems Reliability Growth Modeling and Analysis
Fifth Lecture Hour 9:30 – 10:20 am, September 9, 2001 Framework for a Software Management Process – Life Cycle Phases (Part II, Chapter 5 of Royce’ book)
This class cannot be shared or copied without the written permission of PracticeWorks Systems, LLC.
1 15 quality goals for requirements  Justified  Correct  Complete  Consistent  Unambiguous  Feasible  Abstract  Traceable  Delimited  Interfaced.
Software Product Line Material based on slides and chapter by Linda M. Northrop, SEI.
1 Report on results of Discriminant Analysis experiment. 27 June 2002 Norman F. Schneidewind, PhD Naval Postgraduate School 2822 Racoon Trail Pebble Beach,
A Metrics Program. Advantages of Collecting Software Quality Metrics Objective assessments as to whether quality requirements are being met can be made.
Herriman High Computer Programming 1A Software Development Cycle Things to Know.
Evaluate Phase Pertemuan Matakuliah: A0774/Information Technology Capital Budgeting Tahun: 2009.
Software Requirements Specification Document (SRS)
Unit – I Presentation. Unit – 1 (Introduction to Software Project management) Definition:-  Software project management is the art and science of planning.
Continual Service Improvement Methods & Techniques.
SE513 Software Quality Assurance Lecture12: Software Reliability and Quality Management Standards.
Alex Ezrakhovich Process Approach for an Integrated Management System Change driven.
Organizations of all types and sizes face a range of risks that can affect the achievement of their objectives. Organization's activities Strategic initiatives.
Quality Management System Deliverable Software 9115 revision A Key changes presentation IAQG 9115 Team March 2017.
DoD Template for Application of TLCSM and PBL
CLE Introduction to Agile Software Acquisition
Software Project Configuration Management
Software Quality Control and Quality Assurance: Introduction
Software Risk Management
Security SIG in MTS 05th November 2013 DEG/MTS RISK-BASED SECURITY TESTING Fraunhofer FOKUS.
Initiating systems development
SUPPLIER PARTNERSHIP 2.
Capability Maturity Model
Capability Maturity Model
Presentation transcript:

1 The Role of the Revised IEEE Standard Dictionary of Measures of the Software Aspects of Dependability in Software Acquisition Dr. Norman F. Schneidewind Naval Postgraduate School

2 Outline Introduction Rationale for Revision Criteria for Measure Inclusion Benefits for DoD Program and Project Managers Assessment Prediction Product Dependability / Process Improvement Integration Benefits for the DoD Acquisition Manager New Measure Example Summary

3 Introduction Standard Dictionary of Measures of the Software Aspects of Dependability Dependability –Trustworthiness of a computer system such that reliance can be justifiably placed on the service it delivers. Scope –Specify and classify measures of the software aspects of dependability. –Provide conventions for the application and use of these measures.

4 Introduction Purpose Provide measures that are applicable for continual self-assessment and improvement of the software aspects of dependability The first standard will be a small document with just a few core measures dealing with reliability, maintainability, and availability. This will be followed by a second standard that will address safety, confidentially, and integrity.

5 Rationale for Revision Not revised since Reaffirmed in 1996, but there were significant negative comments. Revised because many of the original measures had undesirable characteristics: –Naivety about the necessary data, personnel capabilities, and training to effectively use the measures. –Did not measure what they purported to measure. –Little field data to back up claims for benefits. –Measures were not widely used.

6 Criteria for Measure Inclusion Have measures helped acquisition personnel, developers, and users achieve their reliability goals? –Some minimum number of recognized uses. –Demonstrated or potential utility of the measure in producing reliable software. Formulated generic measure classes: –Reliability, Availability, and Maintainability.

7 Criteria for Measure Inclusion Determined whether existing measures should be modified, retained, or deleted, based on the criteria. Added any additional information on the measures developed since Clarified definition and implementation conventions. Identified and incorporated, where appropriate, new measures that have appeared since 1988.

8 Benefits for DoD Program and Project Managers Provide technology for DoD program and project managers: Apply measures of dependability: To assess and predict the dependability of software during test and operation.

9 Assessment Make an evaluation of dependability from a historical perspective (e.g., MTTF). For the manager, assessment answers the question: how dependable has my software been in the past? If for example, the software has not met reliability and availability goals, it may be necessary to strengthen process steps, such as inspections and testing.

10 Prediction Forecast the future dependability of the software. Use failure data obtained during test to fit a model for making predictions of reliability during operation and maintenance: –time to next failure, remaining failures, total failures over the life of the software For the manager, prediction answers the questions: How reliable is my software likely to be in the future? If it will not meet reliability goals, what actions are necessary to correct the situation?

11 Product Dependability / Process Improvement Integration For both assessment and prediction, there may be process improvements that could be made to bring the reliability of the software up to the goals established in the specifications. Organizations that have the capability to measure their products and use these measurements to guide process improvement are those with the highest CMM ratings, such as the NASA Space Shuttle avionics software.

12 Benefits for the DoD Acquisition Manager There is much emphasis in DoD on integrating COTS into host systems due to the possible reduction in software development cost compared to developing the software in-house. These components must be reliable, maintainable, and available, and must interoperate with the host system in order for the customer to benefit from the advertised advantages of lower development and maintenance costs. To ensure compliance with these goals, the acquisition manager would specify in COTS contracts the dependability measures of this standard.

13 New Measure Example Risk Factor Regression Model Definitions –CF = d*(exp(e*CI)): –Cumulative requirements issues reliability prediction equation. –Issues: number of possible conflicting requirements. –RF (Risk Factor): attributes of a requirements change that can induce reliability risk.

14 New Measure Example Variables –CF: Cumulative Failures. –CI: Cumulative Issues. Parameters Coefficients of non-linear regression equations: –d, e.

15 Requirements Change Threshold In Figure 1, cumulative failures are plotted against cumulative requirements issues, for both actual and predicted cases. When issues reach 272, actual cumulative failures reach three and climb rapidly thereafter. In this case, a cumulative failure count of three has been identified as a critical value.

16 Reliability as a Function of Requirements Issues

17 Rate of Change of Reliability Because the equation in Figure 1 is an exponential, its derivative is also an exponential and is simply the original function multiplied by a constant. This plot is shown in Figure 2. We should have concern about the negative effect on reliability of issues because of its predicted explosive growth rate.

18 Rate of Change of Reliability as a Function of Requirements Issues

19 New Measure Example Application –Provide warning to software managers of impending reliability problems early in the development cycle -- during requirements analysis – by using risk factors to predict reliability. –Software managers would be to anticipate problems rather than react to them. –More efficient software management would be possible because, with advance warning of reliability problems, management would be able to better schedule and prioritize development process activities.

20 New Measure Example Data Requirements –Cumulative failure count. –Cumulative requirements issues count. Units of Measurement –Dimensionless numbers. Experience –Proposed for NASA-wide adoption. Tools –SMERFS, CASRE, EXCEL, S-PLUS 6.

21 Summary IEEE provides a strategy and technology for DoD managers to ensure the dependability of their software during acquisition, test, operations, and maintenance.