ITRI Industrial Technology Research Institute 2014 NCSL International Workshop and Symposium 1 Evaluation of Proficiency Testing Results with a Drifting Artifact – An Example of Standard Resistor Chen-Yun Hung Center for Measurement Standards (CMS), Industrial Technology Research Institute (ITRI) Taiwan July 31, 2014

2014 NCSL International Workshop and Symposium 2 Contents Introduction Stability Testing Performance Evaluation Confidentiality Conclusions

2014 NCSL International Workshop and Symposium 3 Introduction Proficiency testing (PT) is an effective way to verify a laboratory’s measurement capability Through PT activities, participants can understand the differences between their measurement capabilities and those of other laboratories PT often leads to improvement in measurement competence and quality control In Taiwan, CMS/ITRI organizes the PT which focuses on calibration laboratories

2014 NCSL International Workshop and Symposium 4 Why CMS/ITRI? Industrial Technology Research Institute  The biggest non-profit organization in Taiwan  Half budget supported by the government  About 6,000 employees in most industrial fields Center for Measurement Standards  CMS operates the National Measurement Laboratory  NML is the NMI in Taiwan (like a small NIST)  Maintain 120 national measurement standards  Quality Engineering Department organizes quality management system in CMS  QED is also a PT Provider operated in accordance with ISO 17043, accredited by TAF (P002)

2014 NCSL International Workshop and Symposium Certificate of accreditation 5 Accreditation NumberP002 P002

2014 NCSL International Workshop and Symposium Introduction (Cont.) This paper will present a PT scheme for standard resistors to demonstrate the evaluation of PT results with a drifting artifact  A PT of standard resistors numbered “PT2012-KF01” was performed from October 2012 to February 2013  A total of 16 laboratories participated  Two standard resistors with nominal values of 1 Ω and 10 kΩ were chosen as PT items 6 Nominal Value ManufacturerModelSerial No. 1 ΩiETSRL-1B kΩiETSRL-10kB

2014 NCSL International Workshop and Symposium The flowchart of the PT scheme 7 Start PT Parameter PT Item Stability Testing PT Plan PT Instruction PT Orientation Sample Monitoring Data Retrieval Data Analysis Final Report End

2014 NCSL International Workshop and Symposium Stability Testing Ensure the PT items will not undergo any significant changes during the PT process If not, the stability should be quantified and regarded as an additional component of the uncertainty The criterion for allowing stability in PT2012-KF01 where u sta is the standard uncertainty of stability testing u lab, min is the minimum standard uncertainty of the participants’ results u ref is the standard uncertainty of the reference laboratory’s result 8 Based on the guidelines for limiting the uncertainty of the assigned value in ISO Based on the guidelines for limiting the uncertainty of the assigned value in ISO 13528

2014 NCSL International Workshop and Symposium Stability Testing (Cont.) The criterion for allowing stability in PT2012-KF01  If the criterion is met The stability uncertainty will not affect the evaluation of the participants’ performance  If the criterion is not met The stability should be regarded as an additional component of the uncertainty associated with the assigned value More than one assigned values would be provided to limit the effect of the uncertainty on the performance evaluation In some cases, PT items would be replaced based on technical expert judgment 9

2014 NCSL International Workshop and Symposium Stability Testing for PT2012-KF01 Two months for stability testing before the PT was initiated 10 Uncertainties Nominal Value u sta u lab, min u ref 1 Ω Ω Ω Ω 10 kΩ kΩ kΩ kΩ 1 Ω 10 kΩ Criterion for Allowing StabilityPASS PASS

2014 NCSL International Workshop and Symposium 11 Assigned Value Calibration values reported by NMI were used as assigned values in this PT scheme  National Measurement Laboratory (NML) PT items were calibrated for three times by NML to ensure their stability  If the differences between the three calibration values are all within the expanded uncertainty Assigned Value: Median of three calibration values  If not Assigned Value: Average of the calibration values

2014 NCSL International Workshop and Symposium 12 Assigned Value for PT2012-KF01 Assigned Value (X): Average of the calibration values  Two assigned values were provided One was the average of the first and second calibration values One was the average of the second and third calibration values Expanded Uncertainty of Assigned Value (U ref )  Combine the reported uncertainty of the reference laboratory and the uncertainty of the difference between two calibration values 1 Ω The differences between the three calibration values were larger than the expanded uncertainty

2014 NCSL International Workshop and Symposium 13 Assigned Value for PT2012-KF01 (Cont.) Assigned Value (X): Median of three calibration values Expanded Uncertainty of Assigned Value (U ref )  The reported expanded uncertainty of the median 10 kΩ The differences between the three calibration values were smaller than the expanded uncertainty GroupIII Nominal ValueXU ref X 1 Ω Ω Ω Ω Ω 10 kΩ kΩ kΩ kΩ kΩ

2014 NCSL International Workshop and Symposium 14 Performance Statistic E n was chosen as the performance statistic where x is the participant’s result X is the assigned value U lab is the expanded uncertainty of the participant’s result U ref is the expanded uncertainty of the reference laboratory’s assigned value Criteria for performance evaluation  |E n |  1, “satisfactory” performance  |E n | > 1, “unsatisfactory” performance

2014 NCSL International Workshop and Symposium PT results of the 1 Ω standard resistor in group I 15 PT Results for PT2012-KF01 |E n | > 1 (x – X) & U lab is large

2014 NCSL International Workshop and Symposium PT results of the 1 Ω standard resistor in group II 16 PT Results for PT2012-KF01 (Cont.) Note: The value of Lab J is not available.

2014 NCSL International Workshop and Symposium PT results of the 10 kΩ standard resistor 17 PT Results for PT2012-KF01 (Cont.) CMS/ITRI only provided the PT results and suggestions Did not judge whether the participants were qualified laboratories CMS/ITRI only provided the PT results and suggestions Did not judge whether the participants were qualified laboratories |E n | > 1

2014 NCSL International Workshop and Symposium Confidentiality Each laboratory is given a code that is mailed to the laboratory in a sealed envelope  Only knows its own laboratory code The confidentiality should especially be considered when the assigned value of a drifting artifact is determined  More than one assigned values are provided The number of assigned values should be controlled to avoid fewer participants in one group  Using linear regression to predict the assigned value on a given day The transfer schedule is not publicly available PT items should be transferred to participants by PT provider 18 ISO/IEC The identity of participants shall be confidential and known only to persons involved in the operation of the PT scheme ISO/IEC The identity of participants shall be confidential and known only to persons involved in the operation of the PT scheme

2014 NCSL International Workshop and Symposium 19 Conclusions CMS/ITRI has performed PTs in Taiwan for more than 10 years with the focus on calibration laboratories Ensuring the stability of artifacts always has been given a high priority CMS/ITRI shares a PT of standard resistors with quantified criterion for stability testing using statistical methods Most of standard resistor calibration laboratories in Taiwan have good measurement competence Although the PT items drift over time, it will not affect the evaluation of the participants’ performance by a well- designed method

ITRI Industrial Technology Research Institute 2014 NCSL International Workshop and Symposium 20 Thanks for your attention Chen-Yun Hung