1. Gage Repeatability and Reproducibility (R&R) Studies
An Introduction to Measurement System Analysis (MSA)
Introduction
This course is designed to give you a thorough explanation of gage studies, which are a specialized type of measurement system analysis. Gage R&R is a key tool within the Six Sigma methodology.
This class is best taught using your company’s specific examples, allowing for interaction among the students
Feel free to modify and change this course any way you like. Simply click
View  Slide Master
to edit the master slide so that all slides show your company logo and name. You may also teach it exactly as it is. We recommend about 4 hours to teach the course, but time may vary depending on your experience, class exercises, and knowledge of the class.
This course is intended to teach an individual or group with basic math skills and little to no prior knowledge of the content. We do cover content related to control charts and capability analysis, which is available in course format from our website. We recommend that someone with prior experience and training with Gage R&Rs teach the course (usually Black Belt or Master Black Belt). If not being used as a course, valuable information can still be obtained by following through the presentation and reading the notes section.
If you have any questions, please visit or contact us at
The primary reference for this course came from the AIAG manual for Measurement Systems Analysis:

2. Agenda Importance of data What is MSA?
Measurement Error Sources of Variation
Precision (Resolution, Repeatability, Reproducibility)
Accuracy (Bias, Stability, Linearity)
What is Gage R&R?
Variable vs Binary Data
Variable Gage R&R
Criteria for % of Tolerance
Type I and II error
Attribute Agreement Analysis
Criteria for Kappa
Key Points
More Resources
Here is the agenda for the class
You might want to assign times to it for your own benefit, based on your review of this presentation and what detail you plan to cover.

3. Real life MSA – Mortgage Loan
Approval based on:
Credit score
Rental payment history
Previous home ownership
Job status and length
Income to debt ratio
Type of home
Familiarity with applicant and their references
MSA isn’t only used in manufacturing or technical industries (although it is most common in these environments)
You can also experience measurement variation in real life. If you are applying for a mortgage loan, there is a decision being made about your ability to pay the loan, based upon numerous criteria. These criteria include credit score, rental payment history, previous home ownership, job status, income to debt ratio, type of home, familiarity with applicant, references, etc.
You would hope that the process to determine if you are approved or not has been clearly defined and is accurate (MSA is good). If they have a perfect measurement process, then no matter who reviews your application, the same decision will be made. Otherwise, depending on when you submit your application, you might get different outcomes. Was someone at the bank in a bad mood? Did they discriminate based on information in your loan? Did the credit score show erroneous information that others ignored? Did they have access to all the relevant information? All of these questions might influence the final decision, which means their process has too much measurement error.
Mortgage loans contain some data, but like many processes, there isn’t a single numeric outcome. It might be made up of different numbers, that all roll up into a decision that is reliant on a person, and may or may not be numeric. Even processes that have numeric scores often get reviewed by people, who can decide on how much they want to rely on the numbers.
We will talk more about MSA applied to processes that do not have numeric outcomes later on in the course.
What other examples does the class have to share, where they have experienced measurement error in real life or at work?
Examples:
Credit score
Annual performance review at work
Weight scales at work, gym, home
Restaurant bills (same items priced differently)
Speeding tickets from police (your speed vs radar/laser gun)
Applying for job (resume review)
Judging friends and co-workers competency
Physical check-up at doctor’s office

4. Example: Reproducibility
COMPARE AVERAGES OF SAME PART TO EACH OTHER
REPRODUCIBLE
NOT REPRODUCIBLE
PERSON #1
0.0046
0.0057
0.0032
0.0039
0.0050
0.0030
0.0036
0.0056
AVERAGE
PERSON #2
0.0048
0.0050
0.0034
0.0051
0.0037
0.0032
0.0046
0.0044
AVERAGE
PERSON #1
AVERAGE
PERSON #2
AVERAGE
In this example, we have two people measure the same part over and over again. You can see on the right that the average reading for person 1 compared to person 2 differs from each other ( compared to ), which causes reproducibility variation. On the left, their average readings are the same, even though individual readings do not match exactly. Reproducibility is a comparison of the averages, not the individual readings. However, if the individual readings differ, then it will be less likely that the overall averages will be similar.
GOOD
BAD

5. Measurement Error Measured Value Actual Measurement Measurement Error
Precision
Accuracy
Let’s go back to the diagram we showed earlier. Hopefully it makes more sense, now that you understand the different terms.
Repeatability
Reproducibility
Stability
Bias
Linearity
Resolution

6. Variable Gage R&R Example
OPERATOR # OPERATOR # OPERATOR #3
48 total measurements 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
TRIAL 1
TRIAL 2
Let’s take a look at this study visually. Let’s also assume that we are using measurement (variable) data to assess the study (not attribute data).
At the top, we have 3 different operators. Each operator will take measurements on all 8 parts, and they will repeat the measurement once (2 trials). Therefore, each operator will take 16 total measurements. The total study will be 48 measurements.
Each measurement will be gathered randomly as much as possible, to remove the chance that people could remember their answers, or get in a routine that is not typical of the measurement process.
Let’s look at how the calculations will be setup to assess repeatability and reproducibility in the study. 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

7. Measured Value = 5.5 (PASS)
Type II Error
MEASURING A PASS WHEN SHOULD HAVE FAILED – CONSUMER RISK
LSL = 5
USL = 10
Measured Value = 5.5 (PASS)
Actual Value = 4.9 (FAIL)
Here is a visual picture of Type II error. Let’s assume we have an actual value is 4.9, which is a failing result (below 5). If we have a process with 33% of tolerance in the measurement process, then we have some probability that a measurement from the process would be above 5, which would result in a passing outcome (which would be an incorrect decision).
This is also called consumer risk. Instead of rejecting the item and throwing it away, reworking it, rechecking or re-measuring it, we send it to the customer, who then has to deal with the problem when it does meet their needs.
This is the worst case, as we have passed our problems to our customers, and hurt our reputation.
Let’s look at Type I and II errors in another way… X X
33% measurement error

8. Measurement Error vs Uncertainty
LSL
USL
LSL
USL
Gage R&R% of Tolerance 1%
Gage R&R% of Tolerance
10%
LSL
USL
LSL
USL
As you can see, when the Gage R&R % of tolerance increases, the uncertainty area gets wider. This increases the risk of Type I and Type II errors.
This also shows how the variation in the measurements (green curve) can increase or decrease this risk of these false decisions. If the process is more “capable” (green curve is more narrow), then the probability of data points falling within the uncertainty area would be less. This is why measurement error cannot be evaluated completely on its own, we need to understand where the actual measurements are falling in relation to the limits. The closeness of the measurements to the limits is called Capability.
Gage R&R% of Tolerance
30%
Gage R&R% of Tolerance
80%

9. Attribute Agreement Analysis
The overall results show that 38% of the operators agree with each other completely (3 out of 8, obtained from the last column).
Next, we need to determine if this % of agreement is acceptable or not.
3 out of 8 assessments completely agree = 38%

10. Calculating Kappa for Coin Flips
Ex: Flipping a coin 100 times
45 heads and 55 tails
Po = 45/100 = .45
Pe = 50/100 = .50
Kappa = (0.45 – 0.50) / (1 – 0.5)
Kappa = / 0.5 = -0.1
Po – Pe
1 – Pe
The formula for calculating kappa is the difference between the probability of occurrence and the expected probability, divided by 1 minus the expected probability.
If we flip a coin 100 times, we would expect to get 50 heads and 50 tails. However, we won’t get that exactly due to random chance.
Let’s assume we flip the coin 100 times, and get 45 heads. Therefore, our kappa would use Po = 45/100 = 0.45, and Pe = 50/100 = 0.50
Solving the formula, we get -0.1 for kappa. This is close enough to zero, so we assume that what we observed is close enough to chance, which meets our expectations.
If our observed percentage was 90 heads (0.90), then you can walk the class through the exercise on what that would compute for kappa.
(0.90 – 0.50) / (1-0.50) = 0.40 / 0.50 = 0.8, which would get us closer to a kappa value that differs significantly from chance. You would conclude that the coin or the flipper of the coin does not match random chance. This would lead us to believe that the flipper had a special technique to get the same result, or the coin has been modified to land on one side more often than 50% by chance.
Kappa is near zero, so it matches close enough to our expectations

11. Key Points (cont’d)
Clearly mark or control items, but don’t make markings visible to operators (blind study)
Don’t let operators watch each other, so true behavior can be captured (depends on purpose of study)
Use typical items seen in the process
Measure to as finite a number as possible. Do not round
Make detailed observations as the parts are being measured
Treat each measurement as a new item (full setup and break down each time)
Here are some more points about Gage R&Rs.
Clearly mark the items, so you know which ones are which, but also so that the operator does NOT know which one they are measuring. We are trying to prevent them from memorizing the answers for future measurements, which will not simulate real-life conditions. The operators should think that each part is unique (part number is hidden), even though some of the measurements are repeats of the same part.
In addition, don’t let the operators watch each other perform measurements, as they may be influenced by techniques, and change how they were planning to take the measurement.
Don’t round any measurements, write down or capture as many decimal points as possible.
Record detailed observations when items are being measured. This will help determine what inconsistencies exist, and what improvements need to be made.
Treat each measurement as a new item. If the item requires any setup work (load software, calibrate equipment, change fixturing, documentation, etc) it should all be included in each measurement. Do not just re-measure the item, they need to treat each item like it is a brand new item.

12. Contact For more training materials and resources
Capability Analysis
Control Charts
Lean Six Sigma Overview
Root Cause Analysis 5S
Cost of Poor Quality
Templates and Diagrams
Visit Business Performance Improvement at:
As mentioned in the course, we have other training courses you might be interested in, such as:
Capability Analysis
Control Charts
Lean Six Sigma Overview
Root Cause Analysis 5S
Cost of Poor Quality
Templates and Diagrams
Contact us at our website at Biz-PI.com

13. Business Performance Improvement http://www.biz-pi.com
Additional Resources
Business Performance Improvement
Contact us at if you have any questions about this course, or would like some information about our training and/or consulting options