By John Wilson Accreditation and Metrology Services (Pty) LTD. PT/ILC Requirements - R48 and some implementation ideas. (R80 also covered) By John Wilson Accreditation and Metrology Services (Pty) LTD. Accreditation & Metrology Services -John Wilson 2017

What is PT ? Unfortunately - In our case – it is not …. Accreditation & Metrology Services -John Wilson 2017

What is PT? PT stands for Proficiency Testing The Collins dictionary defines Proficiency as an exam which test how proficient or skilled someone is in a particular activity, field of study, language, etc Similarly test (ing) (the noun) is defined as A test is a series of questions that you must answer or actions that you must perform in order to show how much you know about a subject or how well you are able to do something. Collins English Dictionary. Copyright © HarperCollins Publishers Accreditation & Metrology Services -John Wilson 2017

What is an ILC ? ILC is the acronym for Inter Laboratory Comparison. It is basically the same thing as PT and the two words are interchangeable. Traditionally, PT has been used by Analytical and Chemical Metrology Laboratories and ILC by Physical Metrology Laboratories. Accreditation & Metrology Services -John Wilson 2017

What is an ILC ? In medical laboratories, it is sometimes known as EQA (external quality assurance). There is an international move to phase out the use of ILC (words) and just use PT. Accreditation & Metrology Services -John Wilson 2017

What is PT/ILC ? Neither PT or ILC are defined in the VIM (Vocabulaire international de métrologie) More appropriately, there are definitions in ISO/IEC 17025, ISO/IEC 17011, ISO/IEC 17043, ILAC P9, SANAS A01, SANAS PM01, etc. We are today particularly interested in the SANAS R48 and R80 documents and how they effect the laboratory. R48-04 - Proficiency Testing and other comparison programme requirements for Calibration laboratories. R80-03 – Proficiency Testing … for Testing & Medical laboratories and Blood transfusion services. Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? The SANAS documents reflect largely what the International documents require and the requirements are similar to what most accreditation bodies require of a laboratory. The R48 and R80 documents define the detailed requirements for laboratories. (Read them)! It all starts with a plan! Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? Calibration laboratories need to have a plan that reflects the activities done in the last 5 years AND the plan for next 5 years. This plan must cover the listed parameters and the ranges on the accreditation schedule at least once in each 5 year period. Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? The Testing laboratories are required to have a plan which covers their scope over two accreditation cycles, one cycle in arrears and one in the future. These plans differ from initial assessment PT requirements where the lab has to prove it can achieve the range and CMCs requested. Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? All plans must address the risk profile of the laboratory and it’s activities. The new 17025 puts a much larger emphasis on risk analysis. All plans must be addressed during the Laboratory Management meetings for applicability, progress and performance. Details of what must be in the plan are listed in the appropriate R48 or R80 documents. Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? Calibration laboratories are required to participate in the National Audit Program and may use these audits as part of the PT plan. All laboratories must preferably use PT schemes that have independently been shown to comply with the requirements of ISO/IEC 17043. All laboratories shall participate in PT schemes for the items on their schedule including specific tests or methods or where specific instruments have been separately listed. Calibration should also consider including different methods in their plans. Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? Where no formal PT is available, Cal Labs shall indicate other interlaboratory activities. Such an ILC must still comply with the full R48 PT/ILC activity and reporting requirements. Where no formal PT is available, Testing Labs shall indicate suitable alternative means such as reference materials, intralaboratory or other comparisons. Again R80 compliance is required. The lack of commercially available PTs is not considered a valid reason for non participation. Where PTs are not available or practical, suitable alternatives may be proposed by the laboratory to SANAS for consideration. Accreditation & Metrology Services -John Wilson 2017

What do SANAS require? PT reports shall be clear and comprehensive and have the information required by the R48 or R80 documents. (Identification (people and items), responsibilities, evaluation methods and conclusions are key). All laboratories shall review their own performance and investigate all measurement results. If some fail to meet the minimum acceptance criteria then document the root cause analysis and all corrective and preventative actions taken. (i.e. raise a non conformance in the quality system). Accreditation & Metrology Services -John Wilson 2017

Common problems found Assessors have found the following problems during assessments:- Failure to have a comprehensive plan. Failure to keep the plan up to date. Failure to adhere to the plan. Not using commercially accredited schemes when they are available. (False economy). Participation with another lab and each lab draws up it’s own report with different conclusions. (i.e. no protocol done before ILC) Inappropriate choice of artefact and/or reference value/material. Accreditation & Metrology Services -John Wilson 2017

Common problems found Trying to pass off a PT for one device as being adequate as competence to calibrate another (Drying oven PT given for Autoclave). Failing to prepare a measurement protocol before commencing the PT/ILC. (This means different labs measure different points and report differently so evaluation is not possible). The report does not cover the required items listed in the SANAS documents. Accreditation & Metrology Services -John Wilson 2017

Common problems found Inadequate information in the report to validate the findings. (e.g.. Results, En value but no uncertainties recorded). No evaluation of the PT and it’s adequacy. One word reporting. (e.g. Conclusion – “Acceptable” (even when results do not overlap). Accreditation & Metrology Services -John Wilson 2017

Considering Alternatives There is no benefit from evaluating a device which has also been the source of traceability. Intralaboratory comparisons only evaluates one metrologist against another, usually the procedure, equipment, traceability is exactly the same. It does not confirm that the procedure, equipment or traceability is valid. Accreditation & Metrology Services -John Wilson 2017

Considering Alternatives To do a good ILC, it is not a matter of two or more labs doing an intercomparison and then expecting the assessor to do the evaluation and decision making regarding who passes and who fails. Things to consider are:- Decide on an artefact and check if it will be adequate for the required accuracy or UoM Decide how the reference value will be established. How will drift be monitored. Who will write an acceptable protocol. Accreditation & Metrology Services -John Wilson 2017

Considering Alternatives Define how evaluation will be done. (En, Z) Who will do the evaluation. Who will write the report to the required standard defined in the documents. The organising lab cannot usually use the ILC to demonstrate it’s own performance, especially if it provides the reference value and the reference UoM. How will confidentiality be taken care of? Accreditation & Metrology Services -John Wilson 2017

Conclusion PT/ILCs are definitely an integral part of an accredited laboratory’s life. They should add value and confidence to the laboratory (not just satisfy the assessor). They are a wonderful training tool for evaluating new metrologists progress and provides a source of CPD points for the metrologist concerned. Accreditation & Metrology Services -John Wilson 2017

Conclusion Be creative and innovative when planning the PT schedule so that it is interesting and value adding. Accreditation & Metrology Services -John Wilson 2017

Conclusion Thanks to Neville Tayler from SANAS for his valuable information. Thanks to Steve Sidney from the NLA for his contribution based on running the NLA PT/ILC accredited schemes. Thanks to all the assessors who diligently work with the laboratories during assessments to make sure that the PT/ILC plans are reasonable and within the correct risk profile for the laboratory Thanks to you for listening. Accreditation & Metrology Services -John Wilson 2017

PT/ILCs – here to stay! John Wilson Cell : +27 83 652 0770 eMail : jgpwilson@xsinet.co.za Accreditation & Metrology Services -John Wilson 2017