© 2007 Breakthrough Systems1 Deming’s Red Bead Experiment AICE Quality Conference Tuesday, 06MAR07 Jim Clauson Breakthrough Systems

© 2007 Breakthrough Systems2 Why Are We Here? To consider: what the Red Bead Experiment is, what it has to do with quality and how it can have an impact on your everyday quality activities

© 2007 Breakthrough Systems3 Session Agenda  The Red Bead Experiment  Numeracy  System of Profound Knowledge  Finding Red beads  Impact of psychology  Charting red beads  “What will you do on Monday?”

© 2007 Breakthrough Systems4 Introductions and Expectations If you are sitting with someone you know, please move You will interview, then introduce the person next to you Who they are, something unique or interesting, what industry they represent, and what their expectations are today

© 2007 Breakthrough Systems5 Run Red Bead Experiment Click [here] for the red bead slides.here

© 2007 Breakthrough Systems6 Review: Why Are We Here? To consider: what the Red Bead Experiment is, what it has to do with quality and how it can have an impact on your everyday quality activities

© 2007 Breakthrough Systems7 Beyond Red Beads Red Beads Everywhere, oh my!! Finding those red beads –Red, white and non-red? Quantifying those read beads –Hit by a car –vs- bump a file cabinet Eliminating those red beads – %

© 2007 Breakthrough Systems8 Numeracy an ability to handle numbers and other mathematical concepts in the US, it is somewhat better known as Quantitative Literacy innumeracy is a lack of numeracy

© 2007 Breakthrough Systems9 3 Kinds of Numbers for Management. Facts of life. If we don't make this profit figure, we will go out of business. Planning, prediction and budget. Can be used to compare alternative plans. Arbitrary numerical targets. Generally used to judge workers. Avoid the use of the 3rd kind of number Henry Neave The Deming Dimension

© 2007 Breakthrough Systems10 Data Sanity We can either react to numbers, with explanations of every percent change, with the inherent frustrations, fear, and failure Or We can understand our data, put it to good use, and apply valid management principles The choice is ours.

© 2007 Breakthrough Systems11 Through the Lens of SoPK System of Profound Knowledge Appreciation for a system Knowledge about variation Theory of Knowledge Psychology From The New Economics, Deming

© 2007 Breakthrough Systems12 1 of 4: Appreciation for a System Pay attention to interactions more so than components Knowledge of statistical variation more so than discrete numbers Long term focus more so than short term Cooperation more so than fear, blame and internal competition

© 2007 Breakthrough Systems13 1 of 4: Appreciation for a System - II “94% of the outcome of any organization comes from the processes used, not the people”. “A fault in the interpretation of observations, seen everywhere, is to suppose that every event is attributable to someone (usually the one closest at hand), or is related to some special event. The fact is that most troubles with service and production lie in the system and not the people”.

© 2007 Breakthrough Systems14 2 of 4: Knowledge of Variation You have experienced the Red Bead Experiment The Theory of Variation is at the core of cost savings, Kaizen, 6 sigma… Dr. Deming’s early works focused on statistical variation. He added the rest of the SOPK in the last 10 years of his life. Stable System versus Unstable System

© 2007 Breakthrough Systems15 Variation: Deterministic –vs- Probabilistic Deterministic - linear, cause and effect sequences. If you do this, that will happen. Probabilistic - exact time, location, and effect is random. e.g. Number of Red Beads. Treating a probabilistic result as if it was deterministic will cause problems Past results will not guarantee future results

© 2007 Breakthrough Systems16 Variation: Can we predict? Engineers often predict accidents. Their predictions are uncanny for correctness in detail. They fail in only one way – they can not predict exactly when the accident will happen. - Dr. Deming, Out of the Crisis page 479 Calculations after the fact, using only data available prior to the disaster, showed there was greater than a 10% chance of the Challenger explosion occurring, given the pre-launch temperatures and prior history of O-ring burn through.

© 2007 Breakthrough Systems17 3 of 4: Theory of Knowledge - I Knowledge is based upon prediction Knowledge is built on theory –Chanticleer the barnyard rooster –Actions taken without theory lead to losses Use of data requires prediction There is no true value of a measurement, it depends on methods, context, and use Operational definitions are necessary

© 2007 Breakthrough Systems18 Operational Definitions - I “Clean the table…

© 2007 Breakthrough Systems19 Operational Definitions - II Most arguments about conflicting data come down to the definition of how to count the data Try to be precise in your definitions, but likely something unforeseen will arise The beads were: –red & white or red, white & non-red?

© 2007 Breakthrough Systems20 Deming said… “It’s absolutely vital for business that you settle this method of counting, measuring, definition of faults, mistake, defect, before you do business. It’s too late afterwards” -Dr. W. Edwards Deming

© 2007 Breakthrough Systems21 4 of 4: Psychology Extrinsic versus intrinsic motivation People will use the charts you make - up and down the organization If you do not understand the people & understand psychology, the charts will be ignored Competition, fear, perceptions, loss of control change the data and the chart’s message

© 2007 Breakthrough Systems22 Through the Lens of SoPK System of Profound Knowledge Appreciation for a system Knowledge about variation Theory of Knowledge Psychology Thinking about all 4, we’ll concentrate on identifying and quantifying variation

© 2007 Breakthrough Systems23 Red Beads: Find, Quantify, Reduce

© 2007 Breakthrough Systems24 Take a Step Back: Systems Thinking Process viewed as a system SIPOC The “new” 4M’s Psychology: Suboptimization Remember the “94/6 Rule” Using SPC

© 2007 Breakthrough Systems25 Production Viewed as a System

© 2007 Breakthrough Systems26 SIPOC Supplier Input Process Output Customer

© 2007 Breakthrough Systems27 The “new” 4M’s Old: man, machine, material, method Measurement added Person or people replaces man Equipment replaces machine Supplies is used for material Process is used for method Environment added –Physical and mental

© 2007 Breakthrough Systems28 Group Activity - IV As individuals, draw a SIPOC diagram for your job See if you can identify all of the new 4M’s as inputs to your process Compare and discuss as teams Choose the 3 most interesting to report Save this work, it will be used later

© 2007 Breakthrough Systems29 Psychology: Suboptimization We assume that optimizing a system considers all the sub-parts One unit may be selfish and take an action that makes them look good, but hurts others One process may shift problems down the line to let others have to worry about it

© 2007 Breakthrough Systems30 No Gold Stars Here Awards, bonuses, gold stars can actually have a detrimental impact For a person driven extrinsically, each subsequent reward must be larger in order to have the same impact Creation of winners and losers

© 2007 Breakthrough Systems31 Group Activity - V Game: Win As Much As You Can A Decision Making Exercise, Illustrating the Effects of Human Behaviors and Psychology on Performance Measures [link] to gamelink

© 2007 Breakthrough Systems32 The “94/6%” Rule It is critical to separate system causes from individual causes of variation Deming started at 85% systems and 15% worker and had moved up to 96% and 4% by his death What are the implications of the 96/4% Rule?

© 2007 Breakthrough Systems33 Breather – We OK? What have we addressed? Numeracy System of Profound Knowledge Looking at your work as a system Suboptimization exercise 96/4% Rule We OK?

© 2007 Breakthrough Systems34 Statistical Process Control Control Charting provides knowledge of variation a lens, providing a different way of viewing the world a significantly different view of what is happening than will other methods

© 2007 Breakthrough Systems35 A Basic Control Chart

© 2007 Breakthrough Systems36 SPC Basics Common -vs- special System in control? System predictable? What about psychology?

© 2007 Breakthrough Systems37 Charting the Red Beads In the Red Bead Experiment, we reacted to the random noise from result to result. Rewards, punishments, ranking of the workers, feedback to the workers had no effect on the results of the process. The process was stable and needed to be changed!

© 2007 Breakthrough Systems38 Stable –vs- Unstable Stable processes contain only common cause variations and are predictable Unstable processes contain special cause variations and are not predictable. Only predictable processes may be used to plan effectively

© 2007 Breakthrough Systems39 Cause: Common –vs- special Special Cause Variation: If a statistically significant trend occurs, find the special cause of this trend. Use this information to correct or reinforce these special causes. Common Cause Variation: If no trends exist, you must look at the long run performance of the process and fundamentally change the process in order to improve the process.

© 2007 Breakthrough Systems40 Defining Trending in Charting One point outside the control limits Two out of Three points two standard deviations above/below average Four out of Five points one standard deviation above/below average Seven points in a row all above/below average Ten out of Eleven points in a row all above/below average Seven points in a row all increasing/decreasing

© 2007 Breakthrough Systems41 Trending Example on Control Chart

© 2007 Breakthrough Systems42 Constructing a Control Chart Plot the actual data by month (or whatever time interval you are using) Plot at least 25 points (when available) Calculate a baseline average rate Add 3 standard deviation control limits Incorporate a set of trend rules

© 2007 Breakthrough Systems43 Why 3 Standard Deviations? Dr. Shewhart established 3 standard deviations as an economic balance between failure to detect and false alarms. Economic Control of Quality of Manufactured Product (1931!)

© 2007 Breakthrough Systems44 Just in Case… Many courses incorrectly teach that the control limits cover 99.7% of the normal distribution Not all data are normal, “real data” can cause the rate to be as low as 95% (Dr. Wheeler) The Tchebychev Inequality states up to 11% can be outside three standard deviations

© 2007 Breakthrough Systems45 Breather – We OK? stable –vs- unstable common –vs- special constructing a control chart (or a process behavior chart) 3 Standard Deviation limits

© 2007 Breakthrough Systems46 Choosing Performance Indicators

© 2007 Breakthrough Systems47 Group Activity – VI As individuals, take your SIPOC & 4M exercise, and –Identify a performance indicator on the input and then the output of the process you have diagrammed and explain why you chose it –Share and discuss your choice with your team –Choose 2 or 3 examples from the team to share with the class –Remember: you are hunting red beads

© 2007 Breakthrough Systems48 Activity Debrief What indicators did you choose? Why? What actions would you take if one developed an adverse trend?

© 2007 Breakthrough Systems49 Choosing the Right Measures “Managers who don’t know how to measure what they want settle for wanting what they can measure.” Dr. Russ Ackoff

© 2007 Breakthrough Systems50 Performance Indicator Introduction It is more important how the measure is used than what the measure is Self-fulfilling prophecies can prevent us from gathering any data We are drowning in data, but little knowledge is derived Context and Operational Definitions are crucial

© 2007 Breakthrough Systems51 5 Critical Issues ( 1-3) Managers suffer from overabundance of irrelevant information. Managers don’t know what information they need. Need to look at the decision process to determine this. Even if given the information they need, decision making will not necessarily improve

© 2007 Breakthrough Systems52 5 Critical Issues (4-5) More communication does not necessarily lead to better performance. Information can be used destructively. Managers do need to know how the information system works. Just because it came from a computer doesn’t mean it is right.

© 2007 Breakthrough Systems53 Designing a Management System The information system should be designed as an integral part of the management system Most information systems are designed independently, leading to failure Information systems should serve management, not vice versa

© 2007 Breakthrough Systems54 PI Barriers higher ups will use it as a “hammer” subjected to quotas and targets imposed from above fear (“accountability”) used as a “motivator” actions and explanations as a result of random fluctuations perceived loss of control over portrayal of performance must develop “perfect” indicator the first time use of SPC can minimize these fears

© 2007 Breakthrough Systems55 Ackoff on Performance Indicators We do need to know the context within which the performance indicators will be used Forecasting and living with the forecasted future is important, but what about designing a better future?

© 2007 Breakthrough Systems56 Context of PI Do not look at a chart in a vacuum Reconcile any differences between the data and “gut feeling” Combine experience and the data Lessons from the data should lead to insight in the field, and vice versa

© 2007 Breakthrough Systems57 PI Evolution As a process matures, one may end up evolving the indicators used. For example, if interested in completing actions by commitment dates, one may end up using (as the process matures): Percent of Actions completed by due date in effect at time of completion Percent of Actions completed without missing any due dates during their life Percent of Actions completed by the original due date Average days Actions completed ahead of original due date

© 2007 Breakthrough Systems58 Trying for the “Perfect” PI When committees get together and try to table-top the perfect indicator, paralysis often sets in. Realize all data are flawed, there is no “true value”, indicators can always be “gamed.” Putting the right culture of HOW to use performance indicators in place minimizes adverse impacts. Gain experience with simple indicators, then move on to more complex indicators if needed. With proper analysis, flaws with existing data can be detected and fixed. If you never look at the data, there will never be an incentive to fix the data.

© 2007 Breakthrough Systems59 Just Do It! All data are flawed Make good use of your data Endless conference table discussions won’t cause any data to appear Initial prototype successes will lead to experience, and will further the spread of the use of indicators

© 2007 Breakthrough Systems60 Data Gathering Plan ahead Establish Operational Definitions Check data quality Avoid bias K.I.S.S.

© 2007 Breakthrough Systems61 Data Quality Data should be replicable Operational Definitions are a must Source Data must be defined There is no “true value” of any measure, but a good operational definition can save much trouble in the future ANYONE at ANYTIME in the future should be able to apply the same operational definition to the same source data, and get the same results.

© 2007 Breakthrough Systems62 Choose a Reporting Interval If a trend developed, how long could you go without needing to know it? Longer intervals imply more risk Need sufficient volume of points (25) Costs increase as reporting interval decreases What is current reporting interval?

© 2007 Breakthrough Systems63 Creating a Control Chart

© 2007 Breakthrough Systems64 Creating the Baseline The Baseline on a control chart consists of the average (center) line, a three- standard deviation Upper Control Limit (UCL) and a three-standard deviation Lower Control Limit (LCL). The Baseline allows us to predict the future, and evaluate for trends.

© 2007 Breakthrough Systems65 A Good Baseline A “good” baseline detects future trends with a minimum of false alarms. If a trend is detected, we don’t want it to be due to too few data points in the baseline, causing the baseline to have been inaccurate.

© 2007 Breakthrough Systems66 To get a Good Baseline Do show all data, but change the average and control limit calculations by: Dropping data off of the beginning Dropping data off of the end Dropping individual datum point(s) and circling them

© 2007 Breakthrough Systems67 Dropping 1 st 3 Points

© 2007 Breakthrough Systems68 Establish Expectations “Stable” performance is not necessarily good Management needs to determine if the current stable baseline is “acceptable” or “unacceptable” Recall that the # of red beads was unacceptable, but process was stable and in control

© 2007 Breakthrough Systems69 Monitoring Update charts on the required time interval Check for trends against the trending rules Circle any trends, inform owning management and look for special cause(s) Do not shift a baseline unless there is a trend (baseline proven guilty)

© 2007 Breakthrough Systems70 That’s Charting Performance Indicators in a Nutshell PI as part of an overall management system What makes a good performance indicator Data gathering Establishing a good baseline Establishing Expectations Monitoring

© 2007 Breakthrough Systems71 Review Control Chart of Red Bead Experiment

© 2007 Breakthrough Systems72 Session Summary  The Red Bead Experiment  Numeracy  System of Profound Knowledge  Finding Red beads  Impact of psychology  Charting red beads  “What will you do on Monday?”

© 2007 Breakthrough Systems73 What will you do on Monday? Well???

© 2007 Breakthrough Systems74 Questions?

© 2007 Breakthrough Systems75 Wrap & Roll… Thanks for your time and attention Jim Clauson