CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 1 SMU CSE 8314 / NTU SE 762-N Software Measurement and Quality Engineering Module 16 Six Sigma Principles and Applications

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 2 Contents Six Sigma Principles Six Sigma Applications Summary

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 3 Six Sigma Principles

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 4 When you Manufacture the Product, the Samples Vary

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 5 Upper and Lower Specification Limits Lose Win Lower Spec Limit Upper Spec Limit

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 6 Tight Upper and Lower Specification Limits Lose Win

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 7 Normal Distribution of Values for a Measurable Characteristic Value of Characteristic Frequency of Those Values Low SpecHigh Spec

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 8 The Normal Distribution Curve

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 9 Characterizing the Curve x is the value of an individual product parameter or characteristic (horizontal axis) – For rolling dice, it would be any number from 1 to 12 N is the total number of products produced (population size) – For rolling dice, it would be the number of rolls

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 10 The mean, , is the average: The standard deviation, , explains how much the individual samples vary from the mean: Mean and Standard Deviation   N x  (x-  )  N   

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 11 Mean and Standard Deviation Mean (  ) + 1  - 1  + 2  Value of Product Characteristic (x) Number of Products with This Value

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 12 What is the Standard Deviation? It is a measure of the variability of data In general, standard deviation is proportional to dispersion of the data For normally distributed data: – 68.26% of the data are within 1  of the mean – 95.46% are within 2  of the mean – 99.73% are within 3  of the mean – % are within 4  of the mean – % are within 5  of the mean – % are within 6  of the mean

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 13 Defects and Spec Limits A defect is any data outside the spec limits For normally distributed data: – 68.26% of the data are within 1  of the mean – Thus if the spec limits are ±1 , 31.74% of the data are defective (317,400 per million) – ±2  = 4.54% defective (45,400 per million) – ±3  =.27% defective (2,700 per million) – ±4  =.0063% defective (63 per million) – ±5  = % defective (0.57 per million) – ±6  = % defective (0.002 per million)

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 14 Most Products Have Sigma Level of Between 3 and 4 This seems to be a natural tendency for human processes that have not been intentionally improved for quality Examples: – IRS phone advice (1-2 sigma) – Mail delivery: (3-4 sigma) – Hospital billing errors (3-4 sigma) – But: accidental hospital fatalities are closer to 6 sigma

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 15 Computing Sigma and Mean when you Cannot Measure Every Case You can take a sample from the complete population and use this to make an estimate of the mean and the standard deviation This works better if the sample size is larger

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 16 “Sample” Method for Calculation of Mean and Standard Deviation x is the value of a product parameter (horizontal axis) n is the total number of products measured (population size)

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 17 The Normal Distribution Curve with Relaxed Specification Limits

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 18 Normal Distribution Curve with Tight Specification Limits How can we Meet These Tight Specifications?

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 19 Method I Design for Producability (Relax the Specification Limits)

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 20 Analogies to Software Less complex architecture or design results in fewer coding and maintenance errors More precise or better understood requirements result in fewer design errors Tests developed during requirements phase provide greater probability that system test will correctly test that the requirements are met

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 21 Method II Sorting the Output Premium Products at High Price Normal Products at Normal Price

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 22 Analogies to Software The focus of quality efforts on most used parts of the software – Testing – Inspections – Best software engineers – etc. Apply the best people or most intense efforts to the riskiest parts of the design or the architecture

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 23 Method III Improve the Process (Reduce the Variance)

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 24 Analogies to Software More effective architectures Better designs Better coding practices Better testing practices Detecting and correcting defects at every step of the process Better planning of integration...

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 25 Process Phase Shift All of the above assume that the process continues to produce the same normal distribution, centered about the same mean, from day to day and year to year Or else variance improves as we get more experienced with the process But in actual practice, the mean varies from day to day due to incidental aspects of the environment

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 26 Causes of Variance Cannot Always be Controlled In the case of production, this could be due to such factors as: – Ambient temperature and humidity – State of repair of production equipment – Calibration of equipment – Morale of operators – Variations in characteristics of raw materials, etc

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 27 Variance in Design Processes In the case of design processes, similar factors influence process variation Thought experiment: identify some factors that might cause variance in day-to-day performance of software design and development tasks

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 28 Examples of Phase Shift Shift Right Many Data are Too Large for Spec Limits No Shift Most Data are Within Spec Limits Shift Left Many Data are Too Small for Spec Limits

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 29 Compensation for Phase Shift The same principles apply: If you widen the spec limits, you leave more room for the shift to occur Sometimes called “widening the road” If you reduce the process variance you also leave more room for the shift to occur Sometimes called “narrowing the car”

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 30 Widen the Road or Narrow the Car Note: To accommodate a maximum ±1.5  phase shift, we redefine the goals. A 6  goal becomes a 4.5  goal after the phase shift. Shift Right Most Data are Still Within the Spec Limits No Shift Most Data are Within Spec Limits Shift Left Most Data are Still Within the Spec Limits

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 31 “Six Sigma” is Really 4.5 Sigma 6  for normal case 4.5  for shifted case

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 32 But Remember... The goal is not the number The goal is quality improvement to meet customer requirements and expectations Six sigma methods help you improve, regardless of the number you are at

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 33 How Do We Show That the Shift Is Acceptable Relative to Tolerances? DESIGN TOLERANCE DESIGN MARGIN = PROCESS VARIANCE This is a measure of the overall effect. It is also called the Capability Index We use the term C P for this

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 34 Both Factors Must be Analyzed If you improve the design, it makes the process variations more tolerable If you improve the process, it makes the design less critical to overall success If you improve both, the quality gets significantly better -- significant leaps are possible

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 35 Design Tolerance This is the extent to which the Design allows Variance in Production It is the “Width of the Road” Formula for Design Tolerance is: | Upper Spec Limit - Lower Spec Limit | If both spec limits are the same distance from the mean, then design tolerance is: 2 * | Upper Spec Limit - µ |

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 36 Process Capability (Process Variance) This factor reflects your quality goal. I.e., what is the probability that you will produce something within tolerance. As a somewhat arbitrary choice, the process variation is usually selected to be: ± 3  or, in other words, a range of 6 

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 37 Short Term Process Capability Index or Design Margin | Upper Spec Limit - Lower Spec Limit | C P =  OR | Upper Spec Limit - µ | C P = 

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 38 Some Possible Goals C P = 1 means the spec limits are ± 3  – a 3 sigma process C P = 2 means the spec limits are ± 6  – a 6 sigma process C P > 2 means it is relatively easy to meet the specification limits, because the design margins are wide enough relative to the process capability

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 39 Phase Shift Measurement | Target Mean - Actual Mean | K = phase shift = /2 (USL-LSL) LSL = Lower Spec Limit USL = Upper Spec Limit K = 0 means no shift K = 1 half of the data are out of spec

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 40 C PK C PK = C P * (1-K) C PK < 0 means most samples are bad C PK = 0 means half bad, half good C PK > 1 means most are good This can be used to monitor the actual performance of a process to see if it is producing mostly good products

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 41 What is this Really Telling Us? The spec limits for a process step are merely the outputs of the previous step Thus the goal of each step is to have wide spec limits for the next step (design tolerance) and a low process variance (high quality output from each process step)

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 42 Rolling Throughput The cumulative effect of the individual step variances for the entire process It is analogous to the concept that for each step of the process, we have: – defects coming in (from prior steps), plus defects introduced (in this step), minus defects corrected. By tracking the   C p & C pk values for a process, we can monitor – The overall quality of the result & the sources of the problems

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 43 Defect Accumulation Process Step I = Defects InputO = Defects Output F = Defects Found and Fixed C = Defects Created O = I + C - F

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 44 Computing the Rolling Throughput If step i has probability p i of having a defect The overall probability of defects is: P =  p i Parts or StepsOverall Yield of Defect Free Products 3  4  5  6  193.3%99.4%99.97% % 10010%53.6%97.7%99.966% %77.2%99.66% 150, %

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 45 Application to Software Process Step iStep jStep k At each step, minimize the defects in the output Measure Defects Found Measure Defects Created Set Targets for Defect Levels Allowed at Each Step Perform Causal Analysis to Fix Sources of Defects

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 46 What does Six Sigma Cause One to Do Differently Short Term Impact: – Continuous Process Improvement Know what you need Know what your process can do Measure and characterize the process Long Term Impact: – Process Reinvention You cannot get there without it You can tell where you need it the most

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 47 3 Sigma vs. 6 Sigma Defective 33 66

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 48 Application from Real Life (see “Pasco” in reference list) “U.S. 19 overpasses may get narrow lanes” – But experts fear the 11-foot-wide lanes, proposed to cut costs, also tread on safety. “The state transportation department’s latest plan to fix U.S drivers may start feeling squeezed from a different direction: the sides.... state planners have proposed shaving 1 foot off the standard 12-foot-wide lanes for a series of overpasses.... ‘It’s a bad idea,’ said David Willis, president of the AAA Foundation for Traffic Safety. “There’s a proven relationship between lane width and traffic safety.”

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 49 Graph from Newspaper PSTA Bus 102 inches Grand Marquis 70 inches Fed Ex Truck 92 inches 11 feet (132 inches) Overpass

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 50 Using Six Sigma

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 51 Simple Six Sigma Process Measure Sigma Improve Improved Measures

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 52 Six Steps to Six Sigma (Manufacturing Processes) 1) Identify critical characteristics 2) Identify product elements that influence these 3) Identify process elements (steps) associated with these elements 4) Establish tolerances for these steps 5) Determine actual capabilities of these process steps 6) Fix the process to assure sufficient robustness

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 53 Software Counterparts? Discuss in study groups Possible exam question

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 54 References H arry, Mikel J. and J. Ronald Lawson, Six Sigma Producibility Analysis and Process Characterization, Motorola University Press, Addison-Wesley, ISBN Pasco (Florida) Times, “US 19 overpass may get narrow lanes,” December 16, 1996, pp. 1b, 6b.

CSE SW Measurement and Quality Engineering Copyright © , Dennis J. Frailey, All Rights Reserved CSE8314M16 version 5.09Slide 55 END OF MODULE 16