CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 1 SMU CSE 8314 / NTU SE 762-N Software Metrics and Quality Engineering Module 10 Cost of Quality Analysis

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 2 We Will Examine Value from Three Perspectives Quality (Fewer Defects; Customer satisfaction) Low Cost or High Productivity Customer Value Short Cycle Time or Schedule

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 3 The Cost of Quality

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 4 The Cost of Quality Quality costs money But it can save money The issue: how to save more than it costs Quality (Fewer Defects; Customer satisfaction) Low Cost or High Productivity Customer Value Short Cycle Time or Schedule

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 5 Quality Engineering and Value Quality engineering techniques can add value by – reducing the non-value-added costs – reducing the cost of non-value-added tasks, especially the essential ones Showing reductions in cost is essential to obtain long term support of quality improvement initiatives

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 6 Beware... Since quality improvement initiatives are also non-value-added (according to the strict definition), they are often viewed as suspect Both software engineers and software managers tend to understand the concept of a non- value-added task, but may need to be taught more about the role of quality improvement techniques

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 7 Other Views Some prefer to avoid semantic or emotional arguments by categorizing certain tasks as non-value-added essential (or even as value-added), even though they really are not... – quality improvement tasks – many management tasks – tasks done for political reasons The objective is to improve quality, not to be precise about definitions and categories

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 8 Raising Awareness of Quality Issues

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 9 Raising Awareness is a Prerequisite to Cost of Quality Analysis Managers and technical staff must be convinced that – Quality problems are serious – Poor quality costs them money – Quality is worth fixing – Quality can be fixed by proper techniques

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 10 Why Do We Need COQ Analysis An organization may seek to avoid COQ analysis because – It is a non-value-added activity But... Since we are imperfect, we have to invest in order to improve quality The investment is non-value-added The benefit is to reduce the total non-value-added costs

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 11 Quality Assurance vs Quality Engineering In traditional software QA, software engineers are responsible for the value-added tasks and QA engineers are responsible for many non-value- added quality tasks This leads to a negative bias against quality improvement tasks With QE, everyone is responsible for engineering the process to improve

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 12 “Quality is Free” Juran, Crosby What they mean is that, in the long run, low quality costs more than high quality. Philosophical issue: – Quality means suited to a purpose or fit for a specific use – Quality is relative – “Quality” is not free if it exceeds the need Understanding the needs is essential.

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 13 If you Propose to Improve Quality... Managers and technical staff and even customers will ask a series of very logical questions You must anticipate these questions and understand how to answer effectively And sometimes redirect their focus

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 14 The First Question This is what management and technical staff will typically ask when approached with a proposal to improve quality. What does it cost to have quality?

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 15 A Better Question What is the return on investment for improved quality? In other words, what is the payoff?

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 16 Return on Investment This is the question you want them to ask. – Return on Investment is the way to convince management – But it may not always convince the technical staff

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 17 The Second Question This is what technical staff will typically ask when approached with a proposal to improve quality. Why should I spend my time on quality assurance tasks when I have real engineering work to do?

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 18 Technical Staff are Often Influenced by Pride in Their Work We have the best software in the business!

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 19 Pride in Workmanship Technical staff may need to be shown how quality improvement relates to their pride in their work – This sometimes involves cultural factors unique to the organization – And usually requires management support of activities that promote pride in workmanship

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 20 Cost of Hardware Quality Traditional analysis of the cost of hardware quality puts the focus on production or manufacturing: – Failure Rates From Design Defects From Physical Deterioration – Failures lead to Costs Repair Costs -- Return to acceptable performance Scrap Costs -- Discard and Replace Rework Costs -- Do it Over

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 21 Cost of Hardware Quality These costs are measured in terms of labor and materials and deterioration Failure Cost = xx% of deterioration rate Quality Improvement Cost = yy% of production cost

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 22 The Problem with Software Software production costs are minimal – So measuring quality improvement cost as “% of production costs” makes software look terrible Software deterioration is 0 (see next slide) – So some established methods of measuring failure costs are not applicable

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 23 Software Does “Deteriorate”... Software “deterioration” is due to changes in the environment, accidents, data base damage, and addition of new features

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 24 Software Cost is Mostly Development Cost Sources of Quality Costs Typical Hardware Product Deterioration (60.0%) Production (30.0%) Design (10.0%) Design (99.0%) Typical Software Product Production (1.0%) Deterioration (0.0%)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 25 Categorizing Quality-Related Costs 1) Cost of Conformance – Things that improve quality Prevention Costs - Prevent poor quality Appraisal Costs - Detect poor quality 2) Cost of Non-Conformance – The price of failure to achieve quality Internal Failures - Costs before product shipment External Failures - Costs after product shipment Strategy: Invest in conformance to save in non-conformance

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 26 Categorizing Quality-Related Costs Quality Related Costs Cost of Non- Conformance Failure Costs InternalExternal Cost of Conformance Prevention Costs Appraisal Costs

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 27 Root Cause Analysis Predictive Metrics Preventive Measures Invest in Prevention Prevention Costs Evaluation and Appraisal Costs Failure Costs Invest Here to Save Here Tests Reviews Inspections Audits Evaluations Failures in the field Loss of customers Support costs

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 28 Example of Prevention During the Design Process: What process error resulted in this product problem? Inspections Audits Reviews } Defects in Design Root Cause Analysis Process Improvements Defects in Process Is the Product OK? Is the Process OK?

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 29 Factors Affecting Investment in Prevention Productivity of development environment Mandated prevention activities Technologies in use Types of products under development Development environment Budget and schedule Process maturity Knowledge of development team about – Application domain and software engineering

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 30 Effects of Maturity on Costs SEI Maturity Level Cost as a Percent of Development Cost Prevention Appraisal Internal Failures External Failures Total COQ As reported by Knox (see references)

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 31 Analyzing the Cost of Quality A Process is – A collection of tasks, – Carried out in a particular order, – Producing particular artifacts. Each task can be analyzed for its value produced and its costs Alternatively, you can look at the costs and value of each artifact produced by the process

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 32 Model of a Task Task Name Value added by Task quality cost cycle time etc. Inputs Outputs Results of Performing the Task Product Services Information Performance Reliability Testability Etc. What is Needed to Perform the Task Raw Mtls Data Labor Requirements Training Other Resources

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 33 For a task whose purpose is to improve quality: Total Cost of Performing the Task minus Total Savings equals Net Cost of Quality (COQ) Goal: net COQ should be negative Net Cost of Quality

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 34 In the Next Module... We will show how to analyze cost of quality and value-added together

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 35 References Crosby, Philip B. Quality is Free, New York, McGraw-Hill, Deming, W. Edwards, Out of the Crisis, MIT Press, 1986, ISBN: Juran, J. M., Juran on Leadership for Quality: An Executive Handbook, The Free Press, Knox, 1993, Raytheon studies reported by Houston and Keats, Software Quality Matters, vol 5, no 1 (Spring, 1997), U. of Texas SW Quality Institute

CSE SW Metrics and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M10 8/20/2001Slide 36 END OF MODULE 10