1. ASQ Orange Empire Section January 10, 2017
The Importance of Understanding Type I and Type II Error in Statistical Process Control Charts – Continued
Phillip R. Rosenkrantz, Ed.D., P.E. California State Polytechnic University Pomona
ASQ Orange Empire Section
January 10, 2017

2. Goals – Brief Review of Presentation-Part 1 in October 2016
Brief review process control and process capability
Explain Type I and Type II error and give examples
Illustrate how the improper use of decision rules creates excessive Type I error and creates mistrust in SPC
Suggest simple approaches for reducing Type I error in SPC

3. Goals – Continued
Give examples of Type II error for common decision rules and available strategies for reducing Type II error. 
Present some additional charts available to reduce Type II error for certain types of assignable causes
Educate about the common mistakes in industry related to using the wrong control chart. This mistake seems to more often increase Type II error (failure to detect) rather than Type I error (false alarm).
Discuss the proper use of Individuals charts.

4. Assignable vs. Common Cause Variation
Dr. Walter Shewhart developed Statistical Process Control (SPC) during the 1920s. Dr. W. Edwards Deming promoted SPC during WWII and after.
Premise is that there are three types of variation
Common Cause Variation
Assignable (or Special Cause) variation
Tampering (or over-adjusting)
Each of these types of variation require a different approach or type of action.

5. Common Cause vs. Assignable Cause Variation
According to Dr. Deming’s research, more than 85% of problems are the result of “common cause” variation. Management is responsible for the system and it is their responsibility to work on reducing this type of variation. Later research puts the estimate at over 94%.
The work group is responsible for preventing and reducing “assignable cause” variation.
Management needs to understand these concepts.

6. Tampering – The Third Type of Variation
Tampering is over-adjusting the system caused by a lack of understanding of variation.
Sometimes large built in variation is mistaken for a process going “out of calibration” and needing adjustment
Over adjusting actually increases variation by adding more variation each time the process is changed
Tampering is a difficult habit to break because many machine operators consider it their “job” to constantly adjust their machine.
SPC reduces or eliminates unnecessary adjustments.

7. Major Concept #1: Process Capability
The ability of a process to produce within specification limits
Able to produce within specifications – process is “capable”
Not able to produce within specifications – “not capable”
Often quantified with process capability indices
Cp, Pp – Ability to stay within specs if centered
Cpk, Ppk – Ability based on current distribution
There are two very important concepts that are central to this presentation. If a student could remember only two things from this chapter it would be the difference between process capability and process control. Along with that would be some appreciation for how they would be managed.
#1 is the concept of Process Capability
The ability of a process to produce within specification limits
Able to produce within specifications – process is “capable”
Not able to produce within specifications – “not capable”
Often quantified with process capability indices
Cp – Ability to stay within specs if centered
Cpk – Ability based on current distribution

8. Major Concept #2: Process Control
Process Control refers to how stable and consistent the process is.
“In-control” - stable and only experiencing systematic or “common cause” variation.
“Not in-control” – Process is not stable. Mean and variation are changing due to identifiable or “special” causes (usually controllable by those running the operation).
Represents <10% of the problems
#2 is the concept of Process Control
Process Control refers to how stable and consistent the process is.
“In-control” - stable and only experiencing natural or “common cause” variation.
“Not in-control” – Process is not stable. Mean and variation are changing due to identifiable or “special” causes (usually controllable by those running the operation).
Represents <10% of the problems

9. Process Capability What it is Process Control Process Capability
Note - no reference to
specs !
In Control
(Special Causes Eliminated)
TIME (continued below)
Out of Control
(Special Causes Present)
Process Capability
Sometimes a picture is helpful. The top figure illustrates the concept of Process Control. First we acknowledge that the system itself produces a certain amount of variation that is hard to identify and eliminate. Special cause problems exist caused by identifiable sources of variation. They cause quality problems. Eliminating and control these problems brings a process into a state of “control” or stability.
Once a process is in control, the next objective is to reduce common cause (built in) variation so parts are within specification. Reducing this variation may mean changing the system or doing scientific experiments to understand the factors causing this. Note that if you consider all problems to be special cause problems there is a tendency to blame the workers and not the system.
In Control and Capable
(Variation from Common
Causes Reduced)
Lower Spec Limit
Upper Spec Limit
TIME (continued from above)
In Control but not Capable
(Variation from Common Causes
Excessive)

10. Control Charts
Walter Shewhart developed control charts that help management and workers identify common cause and special cause variation
Management’s responsibility to reduce common cause variation
The work group is primarily responsible for controlling special or assignable cause variation
Small samples are taken periodically with statistics (e.g., average, range) plotted on charts and reveal the amount and type of variation. Control limits are traditionally +/- 3 standard deviations from the process average.

11. Sample Statistical Process Control (SPC) Chart

12. Use of Control Charts
When the process remains within control limits with only a random pattern, process variation can be attributed to common cause variation (random variation in the system) and is deemed “in control.” The process is stable and continues.
When the process goes beyond control limits or is non-random, it is assumed that an assignable cause is present and deemed “out of control.” The process is not stable and predictable. Find and eliminate the assignable cause.

13. Decision or Sensitizing Rules
Decision Rules (a.k.a. Sensitizing rules) are used by operators to determine if a pattern of points indicates a process is no longer stable, that is: “out-of-control”.
Some rules are designed to detect changes or shifts in the process center (mean)
Some rules are designed to detect changes in the process variation (standard deviation)
Some rules are designed to detect a non-normal patterns (e.g. trends or cycles)

15. Types of error when you use sampling
Control charts are based on sampling. Sampling is subject to two kinds of error:
Type I error (α): “False Alarm” – The sample indicates the process is “out-of-control” but is not
Type II error (β): “Failure to detect” – The sample indicates the process is stable, but it really is “out-of- control”
In most quality situations the larger concern is avoiding Type II error: “Failure to detect”. However, with SPC probably the larger concern is Type I error: “False alarms”

16. Types of Error Test Says State of Reality H0 True H0 False
No error
Type I error: a
False alarm, producer’s risk
Type II error: b
Failure to detect, consumer’s risk
H0 True
State of
Reality
H0 False

17. Examples Ho: Part is good Ha: Part is bad
Ho: Person did not commit the crime Ha: Person did commit the crime
Ho: The appendix is good Ha: The appendix is bad
Ho: The process is in control Ha: The process in not in control

18. A look at two decision rules and the probability of Type I and Type II errors

19. Rule 1 – Any point outside the 3σ control limits (probability shown for a sequence of 8 points)
False Alarm
Failure
To Detect
Failure
To Detect

20. Rule 4 – A run of 8 points on the same side of the centerline but within the 3σ control limits
False Alarm
Failure
To Detect
Failure
To Detect

21. Overall Type I Error for both rules

22. Cumulative effect of Type I error on a sequence of 8 points as decision rules are added
The probability of a False Alarm
Increases dramatically as decision rules
are added. It does not take too many
false alarms before operators begin
to lose faith in control charts and
start to ignore them.

23. Type I Error - A Common Problem That Makes SPC Ineffective
Too much Type I error eventually renders SPC ineffective. People get tired of chasing false alarms.
Many experts recommend using two decision rules (three at the most) to minimize Type I error. Rules 1 and 4 are commonly used.
Often, upon set up, software installers toggle on all decision rules thinking that is desirable.
If you use SPC software, ask to see which rules are in effect.

24. Type of Error Consequences Main Causes Solutions
Type I
False alarm.
Sampling error with probability α.
SPC indicates an assignable cause is present but none found.
Results are deadly to SPC efforts. Operators quickly lose faith in SPC and cease to search for assignable causes.
Time wasted collecting data
Too many decision rules. Each additional rule adds to overall Type I error.
Using I, MR-charts on non-normal data
Poor measurement capability
Reduce number of decision rules used (recommend two decision rules).
Test for normality.
Gage R&R studies
Type II
Failure to detect.
Sampling error with probability β.
Slow response to presence of some assignable causes.
Not getting full benefit of SPC. Missed opportunity to improve, especially when capability is low.
Some defects passed on prior to detection.
Some temporary defects never detected.
Some loss of confidence.
Using the wrong chart.
Wrong sampling method
Wrong Sampling frequency.
Poor choice of sample size.
Slow detection of small shifts in processes with low Cp, Cpk.
Not using R or MR-charts.
Not recognizing non-normal patterns.
Low Defect Level
Monitoring an attribute instead of a variable.
Using I-chart on Non-normal data.
Use correct chart.
Use better sampling method.
Increase sampling frequency.
Use appropriate sample size.
Add CUSUM, EWMA, or Moving Average Chart.
Switch from I, MR-charts to X-bar, R-charts.
Learn to interpret control chart patterns.

25. Using the Wrong Chart – 1
The underlying probability distribution of the data and nature of the sample size (constant vs. variable) are used to choose the proper chart.
Continuous random variables (things that are measured) with n > 1 have an underlying normal distribution because of the Central Limit Theorem.
Use x-bar and R-charts
Continuous with n = 1
Use Individuals and Moving Range charts. Very sensitive to non-normality. Check for normality.

26. Using the Wrong Chart - 2
Attribute or discrete data (things that are counted)
Number or proportion good or bad (e.g. defective units, scrap rate)
Underlying distribution is the binomial. Use p or np- charts depending on constant or variable sample size.
Number of defects.
Underlying distribution is the Poisson. Use c or u- charts depending on constant or variable sample size.

27. Using the Wrong Chart - 3 Common problems
Using x-bar, R-charts on defect data
Using x-bar, R-charts or Individual, MR-charts on numbers that are averages
Using Individuals charts for count or proportion data
The wrong chart would generate the wrong estimate of the standard deviation resulting in the wrong control limits. Both Type I and Type II error could occur.
If averages are treated as raw data create huge Type II errors. Averages tend toward the mean, not toward limits.

28. Fraction Non-Conforming Data – Use P Chart (e.g. scrap rate)

29. Incorrect use of Individuals Chart
Incorrect use of Individuals Chart. I-Chart does not show out-of-control

30. C Chart for defects shows out-of-control points

31. Incorrect use of Individuals Chart shows no out-of-control points

32. Not using R-charts with x-bar charts
With normally distributed data the mean and standard deviation are statistically independent. Assignable cause could affect the mean (center), standard deviation (spread), or both.
Using only x-bar charts ignores assignable causes that affect variation.
Users need to be educated on why and how to use both charts together.

33. Using Individual (I) and Moving Range (MR) Charts
The Central Limit Theorm does not apply to individuals data so cannot assume normality. Need to check for normality.
MR method to determine standard deviation uses correlated (not independent) data which makes interpretation of the MR chart potentially invalid. Many experts do not trust the MR chart.
This can be a big problem resulting in either Type I or Type II error—depending on the underlying distribution.
Alternatives – Switch to x-bar, R-charts (n > 1) if practical, transform data, be wary of MR-chart patterns.

34. Wrong Sampling Method (a.k.a. Rational Subgrouping) - 1
Method 1 – Snapshot approach. Samples taken at about the same time. Gap of time between samples.
Cheaper
Can miss assignable causes that come and go
Detects shifts in the sample mean faster
Method 2 – Random sample approach. Every unit has an equal chance of being in the sample.
Can be more labor intensive and therefore cost more
Less likely to miss causes that come and go
Less sensitive to detecting shifts in the sample mean

35. Sampling Methods – Default use Method 1

36. Sampling Method Differences

37. Wrong Sampling Method - 2
Type II error if:
An assignable cause comes and goes between samples using Method 1.
A shift is starting to occur using Method II.
To decrease Type II error, the best strategies are:
Switch to Method 1
Increase sample frequency
Increase sample size
All three of the above

38. Introduction to Statistical Quality Control, 6th Edition by Douglas C
Introduction to Statistical Quality Control, 6th Edition by Douglas C. Montgomery.
Copyright (c) 2009  John Wiley & Sons, Inc.

39. Wrong Sampling Frequency
If frequency between sampling is not close enough, an assignable cause can occur and make it through the process and out the door before detected.
Can end up with lots of defective product throughout the system.
Solution – Determine how long until a defective unit would get to a critical point would be expensive to correct. Sample frequency should be shortened to avoid bad product getting to that point.

40. Sampling Frequency – Design to avoid Type II errors

41. Poor Choice of Sample Size
Some companies always use the same sample size for each application of x-bar and R-charts. (e.g., always using n = 5)
Inceasing the sample size can be used to make charts more sensitive to shifts in the mean or increases in variation.
For processes that are very capable you can decrease n and save money.
For processes that are not capable, you need to increase n to protect against small shifts in the mean.

42. Failing to detect small shifts in low capability processes (Cp, Cpk ≤ 1.67)
If Cp or Cpk are low, then small shifts can produce defective units. One weakness of the traditional Shewart control charts is in detecting small shifts in the mean— under 1 or 1.5 standard deviations.
Can increase sample size and frequency to reduce Type II error. Added sampling can be expensive.
Alternative is to add a third chart that is specifically good at quickly detecting small shifts in the mean: Either the EWMA chart, CUSUM chart, or Moving Average chart.

43. Control Limits are based on Process Variation to Check for Stability

44. Process Capability Affects Your Concern for Shifts from the Mean

45. When process is not capable a shift produces defective units

46. Increasing sample size (n) increases sensitivity to shifts

47. Use CUSUM or EWMA charts to detect small shifts faster
A third chart can be added to the traditional x-bar and R- charts for protecting against small shifts in the process mean (1 to 1.5 standard deviations or less). These charts detect a small shift in half the number of points of traditional Shewart charts. Patterns are not useful because data points are correlated.
Cumulative Sum Chart (CUSUM) – It tracks the cumulative sum of the distance from the mean
Exponentially Weighted Moving Average (EWMA) – Uses exponential weighting to predict the next point.

48. Shewart Control Chart for Process Mean. 1 unit shift after Obs. 20

49. CUSUM for Monitoring the Process Mean

50. EWMA Chart for Process Mean

51. Failing to Detect Non-normal Patterns - 1
Some assignable causes result in non-normal patterns that are not easily detected with traditional decision rules. Analogy would be EEG or EKG where trained eye can see patterns that reflect a health problem.
Cycles, trends – changing operators, changing shifts, tool or equipment wear, or temperature or humidity swings, can cause defects over time but go unrecognized on the control charts.
Hugging limits or centerline – Broken measurement equipment or operators fudging data can cause this.

52. The Cyclic Pattern
Chapter 5

53. Failing to Detect Non-normal Patterns - 2
Recommendations
Training on how to recognize patterns
Some sort of historical data base to help people recognize patterns that repeat themselves.
Use of Six Sigma Black Belts to watch how control charts are being used. They can help recognize patterns and train operators.

54. Dealing with Low-Defect Levels
When defect levels or count rates in a process become very low, say under occurrences per million, then there are long periods of time between the occurrence of a nonconforming unit.
Zero defects occur
Control charts (u and c) with statistics consistently plotting at zero are uninformative.
Introduction to Statistical Quality Control,

55. Dealing with Low-Defect Levels
Alternative
Chart the time between successive occurrences of the counts – or time between events control charts.
If defects or counts occur according to a Poisson distribution, then the time between counts occur according to an exponential distribution.
Introduction to Statistical Quality Control,

56. Dealing with Low-Defect Levels
Consideration
Exponential distribution is skewed.
Corresponding control chart very asymmetric.
One possible solution is to transform the exponential random variable to a Weibull random variable using x = y1/3.6 (where y is an exponential random variable) – this Weibull distribution is well-approximated by a normal.
Construct a control chart on x assuming that x follows a normal distribution
Introduction to Statistical Quality Control,

57. Choice Between Attributes and Variables Control Charts
Each has its own advantages and disadvantages
Attributes data is easy to collect and several characteristics may be collected per unit.
Variables data can be more informative since information about the process mean and variance is obtained directly.
Variables control charts can indicate impending trouble and action may be taken before defectives are produced.
Attributes control charts will not react unless the process has already changed.
Introduction to Statistical Quality Control,

58. Managing SPC
Black Belt or Master Black Belt should be able to set up the proper SPC Charts and monitor them.
Issues to address when designing SPC charts:
Proper type of chart to use for the situation
Decision rules being used
Is the process capable or not capable
Sample size and sample frequency
Sampling method
How assignable causes will be resolved
Continous training on how to interpret/use results