1. TAIEX Workshop on the Accreditation of Medical and Clinical Laboratories (55902) “PARTICIPATION IN INTERLABORATORY COMPARISONS PROGRAMS FOR MEDICAL LABORATORIES” Aliki Stathopoulou Kyiv, 5-6 February 2015

2. ISO 15189 requires that medical laboratories seek confirmation for confidence in their results through participation in suitable interlaboratory comparisons. A regular independent assessment of the technical performance of a laboratory is necessary to assure the validity of measurements or tests, and should be part of an overall quality strategy. INTRODUCTION

3. In this document the following definitions, according to ISO/IEC 17043:2010 Conformity assessment – General requirements for proficiency testing, are applied: Proficiency Testing (PT): evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons. Interlaboratory Comparison: organization, performance and evaluation of measurements or tests on the same or similar items by two or more laboratories in accordance with predetermined conditions. INTRODUCTION

4. One of the elements by which accredited laboratories can demonstrate technical competence is by satisfactory participation in Proficiency Testing (PT) activities where such activities are available and appropriate. A PT scheme is a system for objectively evaluating a laboratory’s performance by the use of external means, and includes regular comparison of a laboratory’s results with those of other laboratories. This is achieved by the PT scheme provider distributing homogeneous and stable PT items to participants for analysis and reporting of the results. Furthermore, participation in PT schemes does not only give information on the performance of the analytical system, but also on other aspects of the management system such as reception/treatment of the sample, treatment of the data, result reporting etc. It is most important that the laboratory sets up a relevant strategy for participation in PT schemes. INTRODUCTION

5. Technical competence can also be demonstrated by successful participation in interlaboratory comparisons that have been organised for purposes other than PT in its strictest sense. For example:  to evaluate the performance characteristics of a method;  to characterise a reference material;  to compare results of two or more laboratories on their own initiative;  to support statements of the equivalence of measurement of NMIs. INTRODUCTION

6. To ensure the appropriateness of participation in PT schemes the following should be considered and understood by interested parties:  the PT scheme in which a laboratory participates should resemble as closely as possible the laboratory’s routine work in terms of measurement samples, analytes and levels; any differences should be noted and accounted for;  laboratories should treat PT items as routine samples, i.e. not give them special treatment;  the evaluation and interpretation of the performance in a PT scheme should take into account the risk associated with the measurement;  unsatisfactory or repeated questionable results must be thoroughly investigated so that the laboratory can understand the reasons for poor performance and correct as necessary; INTRODUCTION (continued on next page)

7.  the performance of a laboratory over several rounds of a PT scheme and analysis of trends is paramount to determining the successfulness of participation;  the PT scheme documentation and protocols must provide clear information in order for all parties to understand how the scheme operates;  the PT provider should be open to discussion amongst interested parties in order to gain a more accurate understanding of the scheme and its operation;  laboratories should view PT participation as an educational tool using the scheme results to give feedback to staff and in the improvement process. INTRODUCTION

8. In selecting the appropriate PT scheme, within an area of technical competence, a laboratory should answer the following questions: 1) What level of PT and frequency do I need? 2) Do any PT schemes exist for the various areas of technical competence? 3) Is the PT scheme relevant? 4) Is the PT provider competent, i.e. does the PT provider operate according to ISO/IEC 17043? 5) Is the selected PT scheme independent of any manufacturing or marketing interests in equipment, reagents or calibrators in its field of operation? SELECTION OF PT SCHEMES

9. In the case where the PT Scheme is organized by a non- accredited provider, the participated laboratory should evaluate it taking into consideration:  the implemented QM System  the confirmed experience in organizing such PT schemes  the data for sample designation and handling  the analysis of the testing data and the subsequent determination of laboratory testing performance  the establishment of the reference values and the relevant uncertainties from testing laboratories, able to demonstrate the traceability of their measurements. The above information will be collected by the participating laboratory and evaluated by the AB. SELECTION OF PT SCHEMES

10. The following aspects should be taken into consideration when a laboratory is establishing its strategy of participation in proficiency testing: a) The laboratory should define its level and frequency of participation after careful analysis of its other QA measures, such as: - regular use of (certified) reference materials; - comparison of analysis by independent techniques; - participation in method development/validation and/or reference material characterisation studies; - use of internal quality control measures; - other inter/intra – laboratory comparisons e.g. analysis on blind samples within the laboratory. Strategy of PT participation (continued on next page)

11. b) The level of risk presented by the laboratory, the sector in which they operate or the methodology they are using. This can be determined, for example, by considering: - number of measurements undertaken; - turnover of technical staff; - experience and knowledge of technical staff; - source of traceability (e.g. availability of reference materials, national standards, etc...); - known stability/instability of the measurement technique; - significance and final use of measurement data (e.g. genetics is an area requiring a high level of assurance). Strategy of PT participation (continued on next page)

12. c) Different types of PT that can be used by laboratories and should be accepted by accreditation bodies, regulatory bodies or customers, include: - PT organised by independent organisations such as accreditation bodies or organisations such as ILAC, EA, APLAC; - ILC organised by a sufficient number of laboratories as a single or continuous exercise; - submission of an internal sample to another or more external laboratories for the purposes of data comparison. Strategy of PT participation (continued on next page)

13. d) It must be recognised that there are sectors where participation in PT may be difficult, due to the technical characteristics of the measurement, the lack of PT schemes, the low number of existing laboratories in the sector, etc. For some fields PT may only be possible or economically feasible for parts of the test undertaken. In these areas the suitability of other QA/QC measures is paramount. e) Any legislative requirements for frequency of type of PT participation. Strategy of PT participation

14. When determining an area of technical competence (sub-discipline) it may be helpful to consider a stepwise approach working up from measurement technique through properties to products. This is because it is more likely that there will be several products and/or properties associated with one measurement technique within a given sub- discipline than vice versa: a) with reference to the measurement technique; b) with reference to the property (parameter) to be measured, determined or identified; c) with reference to products (matrices) to be measured. Strategy of PT participation

15. Once the laboratory has defined its sub-disciplines the “level of participation” can be deemed to have been defined. Accreditation bodies will also need to evaluate the suitability of a laboratories “frequency” of participation, based on level of risk and should expect a minimum frequency of participation for each sub-discipline to be set by the laboratory. Laboratories should be able to justify and, where required, document the technical arguments that have led to the laboratories’ decision on the level and frequency of participation in PT. Strategy of PT participation

16. Accredited testing activities performed by the laboratory: FSH by Chemiluminescence in blood LH by Chemiluminescence in blood Folic acid by Chemiluminescence in blood Calcium by Electrochemistry in blood and urine Potassium by Electrochemistry in blood and urine Cryoglobulins by Electrophoresis in blood Carbamazepine by Immunoassay in blood Ciclosporin by Immunoassay in blood Transferrin by Nephelometry in blood and urine α2 Macroglobulin by Nephelometry in blood and urine ALAT by UV-Visible spectroscopy in blood ASAT by UV-Visible spectroscopy in blood Magnesium by UV-Visible spectroscopy in blood and urine EA-4/18 - guidance on the level and frequency of proficiency testing participation Case Study 5 – Matrix Approach (Clinical Chemistry) (continued on next page)

17. Considerations for determinations of sub-disciplines: In order to determine its sub-disciplines, the laboratory should list all the measurement techniques it uses within its scope, all the properties, which can be individual parameters or groups of equivalent parameters, and all the products, as shown below. Measurement techniques Chemiluminescence Electrochemistry Electrophoresis Immunoassay Nephelometry UV-Visible spectroscopy (continued on next page) EA-4/18 - guidance on the level and frequency of proficiency testing participation Case Study 5 – Matrix Approach (Clinical Chemistry)

18. Properties Drugs (Carbamazepine, Ciclosporin) Electrolytes (Calcium, Potassium, Magnesium) Enzymes (ALAT, ASAT) Hormones (FSH, LH) Specific proteins (Cryoglobulin, Transferrin, α2Macroglobulin) Vitamins (Folic acid) Products Blood Urine (continued on next page) EA-4/18 - guidance on the level and frequency of proficiency testing participation Case Study 5 – Matrix Approach (Clinical Chemistry)

19. List of analyses: From the listed measurement techniques, properties and products, the laboratory should, for each individual parameter, link the parameter to one measurement technique, one property and one product, as shown in the table below. (continued on next page) EA-4/18 - guidance on the level and frequency of proficiency testing participation Case Study 5 – Matrix Approach (Clinical Chemistry)

20. Resulting matrix: From the list of analyses, the laboratory can then establish a matrix, which will highlight the sub-disciplines, as shown below. If the number of products is limited, they can be included in the matrix. If not the evaluation of the products can be treated separately. (continued on next page) EA-4/18 - guidance on the level and frequency of proficiency testing participation Case Study 5 – Matrix Approach (Clinical Chemistry)

21. Resulting sub-disciplines from this exercise: 1.Hormones by Chemiluminescence in blood 2.Vitamins by Chemiluminescence in blood 3.Electrolytes by Electrochemistry in blood and urine 4.Specific proteins by Electrophoresis in blood 5.Drugs by Immunoassay in blood 6.Specific proteins by Nephelometry in blood and urine 7.Electrolytes by UV-Visible spectroscopy in blood and urine 8.Enzymes by UV-Visible spectroscopy in blood Note: although the different products have been combined into one sub-discipline for each detection system in terms of being equivalent from a competency point of view, this does not suggest that they are equivalent in terms of method and laboratory performance. Therefore, the laboratory would be expected to undertake such PTs specifically covering all the products in their scope on a periodic basis. This would be expected to be clearly detailed in their proficiency testing strategy. EA-4/18 - guidance on the level and frequency of proficiency testing participation Case Study 5 – Matrix Approach (Clinical Chemistry)

22. TESTS PERFORMED ON AUTOMATED ANALYZERS This category includes examinations such as: biochemical, immunochemical, immunological, nephelometric, haematological, etc. Annual participation in appropriate proficiency testing schemes (PTs) is required for all tests included in the laboratory accreditation scope. For clinical tests applied to matrices apart from peripheral blood (e.g. urine or CSF), at least one participation in a suitable interlaboratory proficiency scheme is required for those matrices, within the four-year accreditation cycle. In case of calculated parameters (e.g. LDL) participation in a PT scheme is not required provided that there is participation for the parameters used for the calculation, having set an acceptable z–score, ranging from -2 to 2 and within the range of concentrations that the relative equation applies. Suggested frequencies

23. MICROBIOLOGICAL TESTS Participation should be annually or more frequent, in accordance with the program of the provider of the proficiency testing scheme and for all tests, in an approved interlaboratory program (Proficiency Testing) or in the case of absence of an approved interlaboratory program in an interlaboratory comparison scheme, whose provider complies with specific requirements and is evaluated based on these requirements by participants. Suggested frequencies

24. MOLECULAR TESTS For Molecular Diagnostics laboratories, participation in external quality assessments for each category of tests is done according to the following: - For initial assessment and at any extension of the scope of accreditation, the laboratory must have successful results of interlaboratory comparisons for all laboratory tests for which accreditation is requested. The laboratory could then make groups according to parameter and technique/equipment and rotate on a yearly basis. - Within the four year accreditation cycle, the full scope of accreditation should be covered (matrices/tests/techniques). Suggested frequencies (continued on next page)

25. - In the molecular detection of pathogens or somatic mutations, external quality control shall be more demanding in order to check the sensitivity of the method (to include rare strains or very dilute samples). - In the molecular detection of inherited mutations in genes where multiple mutations exist, external quality control is recommended to include the range of all mutations appearing in the Laboratory Accreditation Scope (and not just specific mutations) or combinations of them. Suggested frequencies (continued on next page)

26. - After the first accreditation cycle (4 years) and on condition that the laboratory has demonstrated excellent results in all the previous External Quality Assessments, the frequency of interlaboratory comparisons may be reduced (once every 2 years) per group of parameters and/or techniques/equipment, after evaluation of laboratory’s performance by AB and the consent of the assessment team. Suggested frequencies

27. FLOW CYTOMETRY Two standardized flow cytometry methods exist, both belonging to the calculation of immunologically identified cell populations by flow cytometry (CLSI Former NCCLS H 42 - A the 2nd (vol 27 No 16) : 1. Calculation of lymphocyte subpopulations. 2. Calculation of CD 34 + hematopoietic progenitor cells. Participation in appropriate Proficiency Testing Schemes, with at least 4 samples per year for both the above mentioned tests, is required. Suggested frequencies

28. MICROSCOPY This category includes microscopical haematological, microbiological, cytological and histopathological tests. The participation of the Laboratory in proficiency testing programs should be at least annual, ensuring the participation of all members of the Laboratory. For the initial evaluation and extension of the Laboratory accreditation Scope, the participation of all the examinations under accreditation is required, while for maintaining the accreditation the Laboratory should cover all the examination under the accreditation scope within the 4-year period. The guidelines of authoritative international scientific societies are also taken into account. Suggested frequencies

29. The interpretation of the PT performance concerns all management levels of the laboratory, from the operator to the top management. The personnel responsible for the measurement will be familiar with the operation of the PT scheme and should normally proceed with the initial evaluation. If any investigations have to be undertaken, they should be treated within the non-conformity procedure of the laboratory’s quality management system. An adequate stepwise investigation procedure should consist of the following steps and involve the personnel that performed the analysis:  analyse the problem based on the raw data, the overall performance of the round, the result of successive interlaboratory studies and internal quality control data;  make a plan for corrective action (s);  execute and record the corrective action (s);  check whether the corrective action (s) was effective. Performance evaluation by the laboratory

30. As the laboratory should be using validated methods along with internal quality controls, any poor performance is to be taken seriously as it indicates that there is a problem with the validation and/or the internal quality controls. Following the careful evaluation of single round results, the monitoring of PT performance over time should be done, in order to identify potential problems related to imprecision, systematic error or human error. Performance evaluation by the laboratory

31. Information on PT providers and/or the availability of PT schemes can be found by various means: a) The EPTIS database (www.eptis.bam.de) lists hundreds of PT schemes operated around the world. The focus is on the field of testing; b) National accreditation bodies can provide details of accredited PT Providers and their associated scope; c) Peer laboratories that already participate in PTs or know about relevant PT schemes; d) The PT providers in the participant's own country, which probably will also have (summarised) information about the PT schemes of other providers; e) A search on the Internet, using relevant keywords, can also provide useful information. Availability of PT schemes

32. Thank you!