1. NCSLi 2011 Conference Calibration Software Uncertainty Implementation Philosophies and Practices Speaker/Author : Jorge Martins

2. NCSLi 2011 Conference Laboratory to Document and Approve Calibration procedures Uncertainty analysis Uncertainty budgets Calibration procedures validation plan Calibration procedures validation results Traceability chain for each measurement method

3. NCSLi 2011 Conference Early Developments of Calibration Procedure Software Lack of uncertainty standardization Focus on the needs of a particular organization No international uncertainty standards to follow

4. NCSLi 2011 Conference ISO/IEC 17025 and GUM

5. NCSLi 2011 Conference Requires that to issue an accredited calibration certificate a laboratory must: Estimate expanded uncertainty for each measurement Estimation is compliant with GUM Certificate shall report numerical value of uncertainty for each measurement Implicit references to uncertainty are NOT acceptable ILAC-P14: 12/2010 Policy for uncertainty in calibration

6. NCSLi 2011 Conference

7. Uncertainty Implementation in Software Group Measurement Uncertainty Type of SW INot supported Open source Procedure can be modified by the laboratory IIBudget is supported Open source Procedure can be modified by the laboratory IIIUser defined per point Locked by developer Procedure cannot be modified by the laboratory IVDeveloper defines budget Locked by developer Procedure cannot be modified by the laboratory VDeveloper defines budget Locked by developer Procedure cannot be modified by the laboratory Unc. Budget ISO 17025 Accredited by 3 rd party

8. NCSLi 2011 Conference

11. Document maintenance and validation responsibilities Group IGroup IIGroup IIIGroup IVGroup V Uncertainty analysis (UA) External maintained Not available to user Uncertainty budget (UB) External maintained Internal user available External maintained Internal user available Internal Not available to user Repeatability (type A) collection Not available Original readings user available Not available Original readings user available Internal Not available to user Uncertainty analysis approval User resp. Developer responsibility Dvlp. Resp. 3 rd party IT test accredited Unc. budget calculations validation User resp. Not need SW framework validated User resp. Not need SW framework validated 3 rd party IT test accredited Calibration Procedure validation User resp. Not need SW framework validated Traceability Req. User defined Developer defined

12. NCSLi 2011 Conference Laboratory Calibration Procedure implementation Efforts

13. NCSLi 2011 Conference Major Characteristics IIIIIIIVV User has complete control of measurement process √√XXX Uncertainty validation is independent of measurement procedure √ X √ XX Merged data (measurement and uncertainty) is available to issue calibration certificate X √√√√ Validation of uncertainty engine calculations X √ X √√ Uncertainty budget is not directly synchronized with measurement procedure X √ X √√ Type A data collection in real time X √ X √ 1) Noisy units detection in real time X √ X √ 1) User technically challenging to make changes on the measurement process √ √ X 2) Requires metrologist with SW knowledge √ √ X 2) User defined environmental conditions √ √ XXX User defined Uncertainty analysis constraints and assumptions √ √ XXX

14. NCSLi 2011 Conference Calibration SW Validation Efforts

15. NCSLi 2011 Conference When the software is: Commercially available Documented Locked by the development house The laboratory procedure validation: Trained technicians Operational Capabilities Resources Commercially Available Software

16. NCSLi 2011 Conference Calibration SW Development Process Published Calibration Procedure Measurement Uncertainty Analysis SW Code Implementation SW Test OK? SW Validation SW Validation OK? Y N N Y SW Release Procedure Revision?

17. NCSLi 2011 Conference Calibration SW Development Deliverables SW House Calibration Final Report: Reference to Documented calibration procedure implemented Validation Report summary Uncertainty analysis Summary Detailed SW specifications

18. NCSLi 2011 Conference

19. Conclusion Significant difference in Laboratory:  Validation efforts  Calibration procedure control  Uncertainty analysis control Group I Group II Group III Group IV Group V Estimation Laboratory Approval Efforts 100%90%55%25%20%

20. NCSLi 2011 Conference Questions?