Measurement Systems Analysis Sebastian Fishman April 2019

About the presenter Flir Systems – Engineering Manager – current… (  ) Raytheon Vision Systems - Quality Manager SpaceX - Senior Lean Facilitator Allergan - Senior Project Engineer Qimonda (Infineon Technologies) - Senior Process Engineer Hynix Semiconductors of America - Process Engineer Work Experience Master’s, Applied Physics – University of Oregon, Oregon, USA Bachelor’s, Chemical Engineering – Shenkar College, Ramat Gan, Israel Education Kepner-Tregoe ATS Instructor Certified Lean Six Sigma Master Black Belt – Ohio State University/MoreSteam Lean Six Sigma Black Belt – Virginia Commonwealth University Six Sigma Black Belt – ASQ Certified Quality Engineer – ASQ Certified Quality Manager – ASQ Certified CAPA Investigator – PathWise Certified Medical Device Regulations - AAMI Certifications Senior member Channel Cities Board Member ASQ Member since 2011 ASQ

About Today Measurement System Analysis What is it? Why do we do use it? How is it done? What issues do we encounter About Today

Measurement System Analysis History For many years different automotive companies impose different methods on suppliers about measuring In 1990 Chrysler, Ford and General Motors agreed to a unified platform – via AIAG the 1st Edition of the Measurement System Analysis Handbook was released The manual was developed to meet the needs of the automotive industry, in conjunction with ASQ and ASTM Measurement System Analysis is required by ISO (ISO/TS 16949), automotive and other industries (medical devices, pharma, aerospace, semiconductor) – explicitly or by inference (test method validations)

MSA Processes AAIG (ANOVA, Gauge) EMP (Evaluation the Measurement Process) A Comparison between both methods is not part of this presentation AIAG EMP Estimates EV σpe Standard deviation of the measurement system (repeatability) AV σo Standard deviation between the operators (reproducibility) GRR σe Standard deviation of the combined repeatability and reproducibility PV σp Standard deviation of the variation in the parts used in the study TV σx Standard deviation of the total variation (combining GRR and PV)

Goal: base your decisions on correct data What is MSA? Measurement systems analysis assesses the adequacy of a measurement system for a given application Source: Minitab An experimental and mathematical method of determining how much the variation within the measurement process contributes to overall process variability Source: isixsigma.com Assesses the precision, consistency, and bias of a measurement system Source: JMP.com The purpose of Measurement System Analysis is to qualify a measurement system for use by quantifying its accuracy, precision, and stability Source: MoreSteam.com Measurement System: A collection of instruments and gages, standards, operations, methods, fixtures, software, personnel, environment, and assumptions used to quantify a unit of measure or fix assessment to the feature characteristic being measured; the complete process used to obtain measurement Source: Measurement System Analysis Handbook, third edition, AIAG Goal: base your decisions on correct data

Where is the Data coming from? Measurement System People measuring (Reproducibility) Person variability Person and Part interaction variability Measurement tools (Repeateability) Parts being measured Gauge R&R Gage or Gauge? Gage is the spelling of an obsolescent word meaning a pledge, a challenge, etc. Gauge is the spelling to use when you measure measurement, estimate, or standard. https://writingexplained.org/gage-vs-gauge-difference

What is being evaluated System Stable No time dependency Accurate No Bias No Linearity Precise Repeatable Reproducible

Stability Establish reference material (can be NIST traceable or internal) Establish measurement frequency, data gathering Evaluate performance (trend analysis, Control Limit violation)

Accuracy Bias Linearity No difference between true and actual Non linear change between measured parts and actual Linear correlation between measured parts and actual

Tip On How to Remember: Machines Repeat – People Reproduce Precision Spread or variation of measured values Repeatability Variation due to the measuring device, or the variation observed when the same operator measures the same part repeatedly with the same device Reproducibility Variation due to the measuring system, or the variation observed when different operators measure the same part using the same device Tip On How to Remember: Machines Repeat – People Reproduce

Precision (Gauge RR) Part A Operator 1 Part B Operator 2 Part C x10 Other Combinations 10 parts, measure 2-3 times One operator, multiple instruments Crossed vs Nested (destructive testing) Operator 1 Part B Operator 2 Part C x10

What do we mean?

Conditions for Running Gauge Study Need sufficient stats Rule of thumb is no less than 30 measurements – why? Replication and repetition Break setups to capture the whole process – capture processing impact on measurement Randomization Parts and operators should be randomly order for measuring – avoid loading effects Blind measurement Operator should not know what part is measuring (or what they measured before) – avoid operator influence Conditions for Running Gauge Study

Sources of Errors in Execution Example is not limited, other more detail sources of variation and issues exist

Type of MSA Continues: Measurement (Gauge RR Study) Numbers Uses ANOVA for assessing the statistical significance of the sources of variation Attribute: Appraiser Assessment (Attribute Gauge Study) Assessments (pass/fail, good/bad) Uses Kappa Statistics to assess the agreement level for the appraiser to him/herself and the Reference material (0: no agreement, 1: full agreement) Requires more data points for statistical significance

Process for data gathering Define Objectives Attribute or Gauge Define reference material Nominal parts or known parts (good/bad) Set up Process Randomization Data gathering sheet Ensure measurement instrument has the correct resolution Train operators on the process How to measure/do the activity Data gathering (blind) Process data Plug into Excel, JMP, Minitab or other statistical software Conclusion Is the instrument capable? Are the Operators consistent and correctly analyzing?

Acceptance Guidelines Source Automotive Industry Action Group (AIAG) (2010). Measurement Systems Analysis Reference Manual, 4th edition. Chrysler, Ford, General Motors Supplier Quality Requirements Task Force AIAG also states that the number of distinct categories into which the measurement system divides process output should be greater than or equal to 5 Percentage of process variation Acceptability Less than 10% The measurement system is acceptable. Between 10% and 30% The measurement system is acceptable depending on the application, the cost of the measurement device, cost of repair, or other factors. Greater than 30% The measurement system is not acceptable and should be improved.

Distinct Categories Represents the number of non-overlapping confidence intervals that span the range of product variation The number of distinct categories also represents the number of groups within your process data that your measurement system can discern Example (Distinct Categories) 2 = cannot distinguish the pars 3 = can distinguish a bit (high, medium, large) 4 = can distinguish a bit more 5 = can distinguish sufficiently to come up with a conclusion Categories is a sign of Resolution Ability of the measurement system to detect and faithfully indicate small changes in the characteristic of the measurement result Rule of tens (10 to 1 for measurement capabilities)

Sources http://reliawiki.org/index.php/Measurement_System_An alysis https://www.minitab.com/uploadedFiles/Documents/sam ple-materials/FuelInjectorNozzles-EN.pdf https://www.jmp.com/support/help/14-2/measurement- systems-analysis-platform-options.shtml https://support.minitab.com/en-us/minitab/18/help-and- how-to/quality-and-process-improvement/measurement- system-analysis/supporting-topics/gage-r-r-analyses/is- my-measurement-system-acceptable/ http://www.six-sigma-material.com/MSA.html https://www.itl.nist.gov/div898/handbook/mpc/mpc.htm

Questions?