1. Using Baseline Data in Quality Problem Solving
10th Annual Galilee Quality Conference
Ort Braude College, Israel
May 24, 2018
Stefan Steiner
Business and Industrial Statistics Research Group (BISRG)
Dept. of Statistics and Actuarial Science
University of Waterloo, Waterloo, Ontario Canada

2. University of Waterloo Location

3. University of Waterloo Mathematics Faculty
7000+ students
250+ full-time professors
5 departments/schools (Applied Math, Combinatorics and Optimization, Pure Math, Statistics and Actuarial Science, School of Computer Science)
200+ courses in mathematics, statistics and computer science
3 buildings – see below
¼ total size of the University

4. Images of Canada

6. Outline Background and context Baseline investigation
crossbar dimension case study
Baseline investigation
goal/purpose, investigation plan
analysis and baseline summaries
Uses of the baseline investigation results

7. Background and Context
Goal is to reduce costs and improve quality by reducing variation
e.g. Crossbar dimension
Use the Statistical Engineering algorithm (or other variation reduction systems like DMAIC in Six Sigma)

8. Statistical Engineering Algorithm
Baseline investigation occurs in first stage
Each stage requires one or more process investigations planned sequentially
Uses both observational plans (early in algorithm) and experimental plans (late)
ASQ Quality Press,
ISBN

9. Problem Baseline Investigation
Goal: estimate performance measure that defines the problem, i.e. quantify the size of the problem
Need to choose:
characteristic(s)
performance measure(s)
sampling scheme (empirical investigation needed)

10. Why Establish a Baseline?
Traditional Purposes
help set the problem goal
allow later validation that the problem has been solved
Help plan and analyze subsequent investigations, by determining
how the output varies over time
the “full extent of variation”

11. Crossbar Dimension Baseline
Plan: sample and measure 6 consecutive parts, once per hour for 8 hours per day over 5 consecutive days
St. dev: 0.45, full extent of variation -0.3 to 2
Output varies hour-to-hour, day-to-day (but NOT much part to part)

12. What Have We Learned?
Baseline performance: output st. dev. = 0.45 and full extent of output variation is -0.3 to 2
set the goal to reduce standard deviation to 0.25
Full extent of output variation is seen over hours and days
Thus, an observational investigation conducted over a short time will not see the full extent of the problem!

13. Statistical Engineering Algorithm
Baseline investigation occurs in first stage
Each stage requires one or more process investigations planned sequentially
Uses both observational plans (early in algorithm) and experimental plans (late)

14. Assessing the Measurement System
Goal: determine whether the measurement system contributes a large amount of variation?
Plan:
measure a number of parts repeatedly over a variety of conditions (10 times each over 2 days).
how many parts? (3)
what parts should be selected? (small, medium and large)
Analysis:
plot the data and compare to the full extent of variation
use baseline estimate of the process st. dev. to improve precision of conclusion about (the relative size of) measurement variation.

15. Assessing the Crossbar Dimension Measurement System
Assume the simple additive model:
(P and M independent) =
Conclusion: the measurement system is adequate and not a large cause of output variation

16. Statistical Engineering Algorithm
Each stage requires one or more investigations planned sequentially
Requires a medium to high volume process
Uses both experimental and observational plans
Employs a variety of approaches to reduce variation
Focuses on a dominant cause

17. Dominant Cause
Assume there is a dominant cause, i.e. apply the Pareto principle to causes,

18. General Principles for Finding a Dominant Cause
Plan
use the method of elimination, families of causes and a sequence of simple observational studies (another talk)
choose the time frame to see full extent of variation
consider cause families consistent with time pattern of variation seen in the baseline investigation (e.g. in crossbar dimension example a dominant cause does not act part to part)
use leverage (compare extremes) where possible
Analysis
check that the full extent of variation was observed
look for large or dominant cause(s)

19. Finding a Dominant Cause of Crossbar Dimension Variation
Plan: measure 5 slowly varying inputs and crossbar dimension on 40 parts haphazardly selected over a two day period
dominant cause has acted, eliminate hydraulic pressure as a possible dominant cause (and 3 other inputs)
barrel temperature is a suspect

20. Verification of Barrel Temperature as a Dominant Cause
Select two levels, at opposite ends of normal range, (75 and 80°C) for barrel temperature
Over a short time, produce 10 molded parts at each barrel temperature (after temp. stabilized)
Barrel temperature is the dominant cause

21. Replication and Random Run Order Were Not Needed in the Verification
The experiment was conducted over a short time
In this example, all possible dominant causes, other than barrel temperature, varied hour-to-hour
Thus, all possible confounders (in the clue generation) are held fixed in the verification experiment
If the dominant cause had acted over the short term we need many runs and random run order to reduce the risk of confounding.

22. Desensitization Solution
The team raised the barrel temperature set point to make the process less sensitive to barrel temp. variation
Lowered average dimension by changing another input (they already knew how to do this)

23. New Problem
With the new barrel temperature set point there was an increased chance of a “burn” defect – scored 1 (no burn) to 4
The team addresses the new problem with another application of the Statistical Engineering algorithm
Further investigation
showed the dominant cause of “burn” acted part to part, and full extent of variation is 1-4
failed to find the specific dominant cause.

24. Crossbar Robustness Experiment
The team conducted a half fraction experiment with four factors (normally fixed inputs) at 2 levels each
Treatment
Order
Injection
speed
pressure
Back
Screw
rpm
Burn
scores
Average 1 4
slow
1000 75
0.3
1, 2, 1, 1, 1
1.2 2 8
fast
0.6
1, 1, 1, 1, 1
1.0 3
1200
1, 2, 2, 2, 2
1.6 5* 5
100
1, 3, 2, 2, 1
2.2 6 7
3, 3, 2, 2, 4
3.4
1, 1, 1, 2, 2
2.0
2, 2, 4, 3, 2
3.2

25. Crossbar Robustness Experiment
A run consisted of five consecutive parts – the dominant cause was expected to act within each run!
Treatment number 5 corresponded to the current process
Treatment order was randomized

26. Robustness Experiment Results
model average score
Treatments 2 and 3 are very promising
Back pressure (factor C) important
Solution: change to low level of back pressure

27. Robustness Experiment Summary
Define a run long enough to see the full extent of output variation in the existing process
A robustness experiment is not feasible unless the dominant cause acts over a short time
Compare performance under all treatments to the baseline

28. Crossbar Dimension Solution Validation
Proposed process changes were implemented
Solution validation investigation measured crossbar dimension and burn score for 300 plastic bases selected over 2 shifts
St. dev. reduced to 0.23 and only 2 burn defects

29. Baseline Investigation Summary
Plan: systematic sampling of 100+ parts
Analysis: determine process performance, full extent of variation, and time family of the output (and thus the dominant cause)
Baseline information can be used to plan and analyze further investigations. It helps us
define an appropriate study population (time frame)
check that the dominant cause has acted in an observational plan
define a run and assess the importance of randomization and replication in an experimental plan

30. Questions / Comments?