WHO Prequalification of Diagnostics – Tips for Success AIDS 2012 Satellite Session 25 July 2012, Washington DC Gaby Vercauteren Diagnostics & Laboratory Technology Department of Essential Medicines & Health Products

2 | AIDS2012 – WHO Satellite Session l 25 July 2012 Approach of PQDx Through a rigorous process identify diagnostics that meet the quality standards Harmonize the WHO prequalification process –More stringent dossier assessment of diagnostics –Inspection of the QMS at production site (ISO 13485/GMP) –Laboratory assessment of performance characteristics Partnering with well known regulatory authorities and with key technical partners

3 | AIDS2012 – WHO Satellite Session l 25 July 2012 Prequalification of Diagnostics Application by Manufacturer Meets Requirements Dossier Assessment Manufacturing Site Inspection Laboratory Evaluation Product Prequalified Post Market Surveillance

4 | AIDS2012 – WHO Satellite Session l 25 July 2012 Prioritization of PQDx applications CommentCurrent prioritization criterion Ensure continuity of supply and quality of products procured Already listed on UN procurement scheme and procured by UN organizations in significant levels Focus on priority disease areas – highest historical procurement Assist diagnosis of infection with HIV-1/ HIV-2 or with malaria Point of Care tests, closer to the patient. Particular relevance for priority diseases Rapid test format Ensure known supply chain; no duplication of effort, best possible prices Original product manufacturers Focus on unmet market / procurement needsFew other prequalified products exist in the product category such as CD4, VL

5 | AIDS2012 – WHO Satellite Session l 25 July 2012 Applications to the PQDx Programme Relevant WHO documents: 1.Prequalification of diagnostics application form 2.Instructions for the completion of the application form Application form + IFU + authorization letter must be submitted in hard copy and electronic copy to WHO/HSS/EHT/DLT Application reviewed through the application checklist

6 | AIDS2012 – WHO Satellite Session l 25 July 2012 What information is included in the application? The application form provides summary information about the diagnostic product and the manufacturer Is it a priority diagnostic product? Does the manufacturer claim to use a QMS approach to manufacturing? Do the instructions for use conform with WHO PQ requirements?

7 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - application form Submit a separate application for each product List all manufacturing sites and key raw material suppliers Ensure no discrepancies between data provided on the form and the IFU: Manufacturer contact details Product name Product code Performance characteristics Poor information and/or poor wording in IFU (GHTF/SG1/N43:2005 Labelling for Medical Devices criteria applied)

8 | AIDS2012 – WHO Satellite Session l 25 July 2012 Prequalification of Diagnostics Application by Manufacturer Meets Requirements Dossier Assessment Manufacturing Site Inspection Laboratory Evaluation Product Prequalified Post Market Surveillance

9 | AIDS2012 – WHO Satellite Session l 25 July 2012 What is a product dossier? Product dossier = the selection of records and documents from the technical documentation that shows how the diagnostic product was developed, designed and manufactured WHO reviews the dossier with the purpose of –Assessing the product and how it performs –Assessing the product manufacture –Determining if the manufacturer's quality management system is of an adequate to ensure consistent quality

10 | AIDS2012 – WHO Satellite Session l 25 July 2012 WHO PQDx dossier requirements  Dossier must demonstrate that the in- vitro diagnostic medical device conforms to the Essential Principles of Safety and Performance of Medical Devices (GHTF/SG1/N41R9:2005) Key Components Product description Design and manufacturing information Product performance specifications & associated validation and verification studies Labelling Commercial history Regulatory history Quality management system

11 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Read thoroughly the "Instructions for compilation of a product dossier" available at:  Submit two hard copies and one electronic copy (CD or DVD only).  Dossier clarity. Product dossier should be clear and well organized. Please be reminded that all the information submitted within the dossier is treated as CONFINDENTIAL.

12 | AIDS2012 – WHO Satellite Session l 25 July 2012 Product dossier checklist Needs to be submitted together with the product dossier Helps the manufacturer in correctly compiling the dossier Helps the dossier assessor to identify dossier sections which correspond to a specific dossier requirement as all dossier sections must be cross-referenced to the cover sheet

13 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier Requirements applicable to the entire dossier Dossier clarity (clear, well-organized, page numbers, bound/ring folder) Layout in order as per instructions (differently formatted dossiers accepted if fully cross-referenced on checklist) Language and units of measure used –English language unless arrangement prior to submission –Certified translations –Metric units of measure except for other internationally accepted units of measurement

14 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Regulatory versions of the product. Clearly identify the regulatory version submitted for prequalification, indicate the full name and product code. All the information submitted in the dossier must relate to the regulatory version undergoing prequalification. The use of the same product code for different regulatory versions is not acceptable under any circumstances!

15 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Essential principles (EP) checklist. The entire checklist should be submitted, including identification of the product name and product code. EP checklist available at: Guidance from TGA to assist in the completion of an EP Checklist. The reference for this is found at the following link: Refer to section “Australian regulatory guidelines for medical devices (ARGMD) Part 1– Introduction”.  Product Design and Manufacturing process. Submit a design flowchart and a manufacturing process flowchart (in one or two separate flowcharts) specific to the product undergoing prequalification.

16 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier Trueness of measurement. Where a reference standard or method is not available, a method comparison study can be submitted. In this case equivalency with an established product should be documented. Measuring range of the assay. In the case that the product undergoing prequalification is a qualitative assay, please provide information on how the assay performs at both spectrums of analyte concentrations, i.e. at high and low levels. Validation of assay cut-off. For qualitative assays, data supporting the distinction between reactive and nonreactive test results should be submitted.

17 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Analytical Studies. Clearly identify the full name and product code of the device used to perform the studies. NOTE: the product used for the studies must be the same undergoing prequalification! Submit the full study report: study protocol, method of data analysis (no raw data) and conclusions.

18 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Clinical evidence (clinical or diagnostic sensitivity and specificity). Clinical evaluation – Manufacturer. Provide evidence for clinical/diagnostic sensitivity, including seroconversion sensitivity; clinical/ diagnostic specificity ; and positive and negative predictive values (PPV and NPVs respectively) for different populations (high and low prevalence). Studies should take into account different clinical stages associated with infection from these organisms; geographical distribution; and organism specific attributes such as genotypic differences, and micro-organism life cycle. Clinical evaluation - Independent study. Provide details of at least one well-designed independent performance evaluation for the product under assessment.

19 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Stability (excluding specimen stability). Clearly identify the full name and product code of the device used to perform the studies. NOTE: the product used for the studies must be the same undergoing prequalification! Claimed shelf life and storage conditions. Clearly define the conditions for individual reagents in the kit and the assembled kit for both, when unopened. In-use stability of buffer and device. Provide information on the stability of the buffer and the device after first opening. Consider external factors such as heat and humidity. Evaluate on products near the end of shelf life, and low positive samples should be used to challenge the assay. Shipping Stability. These studies can be performed under real and/or simulated conditions and should include variable shipping conditions such as extreme temperature (heat and/or cold), humidity, light and/or pressure, etc.

20 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  Labels and instructions for use (IFU). All labels and IFU should indicate the product code(s). Please submit all the foil packs, boxing, cartons, etc. and the actual instructions for use, as provided in the test kit box.  Regulatory history. Provide in a certified copy any regulatory approval obtained in any of the countries of supply of the product undergoing prequalification. Copies may be certified by the manufacturer or by a notary public. Submit translations of the approvals when applicable.

21 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – compilation of the product dossier  ISO 13485:2003 certification. Certified copies of the current ISO 13485:2003 certificate should be submitted. The copies may be certified by the manufacturer or by a notary public.  Fast-track. In order to attempt to fast-track the assessment and inspection processes, please submit the two previous inspection reports issued by the certification body, related to the ISO 13485:2003 certification.

22 | AIDS2012 – WHO Satellite Session l 25 July 2012 Useful ISO references Relevant ISO standards ISO 13485:2003Medical devices - Quality management systems - Requirements for regulatory purposes ISO/TR 10013:2001Guidelines for quality management system documentation ISO 14971:2007Medical devices - Application of risk management to medical devices ISO 17511:In vitro diagnostic medical devices - Measurement of quantities in biological samples - Metrological traceability of values assigned to calibrators and control materials ISO 14155:2003 parts I and IIClinical investigation of medical devices for human subjects ISO 23640:2011In vitro diagnostic medical devices - Evaluation of stability of in vitro diagnostic reagents

23 | AIDS2012 – WHO Satellite Session l 25 July 2012 Useful GHTF references Relevant GHTF guidance GHTF/SG1-N63:2011Summary Technical Documentation (STED) for Demonstrating Conformity to the Essential Principles of Safety and Performance of In Vitro Diagnostic Medical Devices GHTF/SG1/N41R9:2005Essential Principles of Safety and Performance of Medical Devices GHTF/SG1/N70:2011Labelling for Medical Devices GHTF/SG2-N54R8:2006Medical Devices Post Market Surveillance: Global Guidance for Adverse Event Reporting for Medical Devices GHTF/SG2-N57R8:2006Medical Devices Post Market Surveillance: Content of Field Safety Notices

24 | AIDS2012 – WHO Satellite Session l 25 July 2012 Useful CSLI references Relevant CSLI Standards EP07-02Interfering testing in clinical chemistry, 2 nd edition EP25-AEvaluation of Stability of IVDs EP5-A2Evaluation of Precision Performance of Quantitative Measurement Methods, 2 nd edition

25 | AIDS2012 – WHO Satellite Session l 25 July 2012 Prequalification of Diagnostics Application by Manufacturer Meets Requirements Dossier Assessment Manufacturing Site Inspection Laboratory Evaluation Product Prequalified Post Market Surveillance

26 | AIDS2012 – WHO Satellite Session l 25 July 2012 WHO PQDx inspection requirements  The manufacturer must demonstrate that the IVD is produced under a functional quality management system e.g. conforms to ISO 13485:2003 Key Components Quality management system including documentation requirements Management responsibility including customer focus, quality policy Resource management including human resources, work environment Product realization including production and service provision, control of monitoring and measuring devices Measurement, analysis and improvement including control of nonconforming product, improvement

27 | AIDS2012 – WHO Satellite Session l 25 July 2012 WHO PQ Dx inspections requirements AND ALSO Dossier submission data – to confirm is true QC and lot release WHO related/end user issues -IFU -stability (transport, in use, to expiry dates), -training -complaints reporting mechanisms

28 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - be ISO compliant Ensure the basics to avoid common nonconformities –document control; adequately descriptive SOPs –records, raw data poorly kept or non existent –inadequate human resources – lack of qualified technical staff capable of investigating quality problems, with regulatory experience, with quality management system expertise –infrastructure poor - impossible to clean –lack of adequate identification and traceability through production processes –lack of adequate quality control over in-process product – no acceptable quality level (AQL) defined

29 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - be ISO compliant Other common nonconformities to avoid –lack of control of critical key suppliers especially of outsourced parts of manufacturing process –not QA department driven and not independent of production management –risk management – documentation, analysis poorly done –process validation poor / no schedule to repeat validation /poor understanding of validation –version control – unclear identification of product (not using unique codes: is product that was PQed the same one as offered for sale)

30 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - Lot (batch) release data Ensure robust and scientifically sound lot release practices –Adequate sampling procedures of tests for QA/QC –Specimen panels should be challenging enough to detect failure or drift –Independence and adequately staffed QA/QC department Reporting procedures on deviations from expected result –Ensure corrective and preventive action reporting and rectification Monitor on-going stability

31 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success – WHO customer focus Ensure risk management –End user – literacy, particular training needs –Environment – hot, humid, dusty –IFU content accurate with evidence of claimed sensitivity and specificity easy to interpret – pictures, diagrams, simple language, ‘executive summary’ for ready reference controlled document – version clearly identified

32 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - confirmation of dossier submission Ensure access to evidence –Confirmation of studies e.g. stability studies Raw data must be readily available Detailed laboratory reports Submit two lots for WHO laboratory evaluations –Demonstrate through batch manufacturing records that the lots are significantly different –Send test kits from a typical lot, not small lots specifically made for submission for WHO testing.

33 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - avoid critical nonconformities Evidence of falsification of information –relating to batch release QC –submission of information for WHO dossier review –submission to WHO laboratory evaluation –organogram/staff positions. Lack of evidence (raw data) onsite to support significant numbers of claims made in dossier e.g. stability testing, performance claims

34 | AIDS2012 – WHO Satellite Session l 25 July 2012 Prequalification of Diagnostics Application by Manufacturer Meets Requirements Dossier Assessment Manufacturing Site Inspection Laboratory Evaluation Product Prequalified Post Market Surveillance

35 | AIDS2012 – WHO Satellite Session l 25 July 2012 Tips for success - Laboratory evaluation Ensure two representative production lots are submitted –Batch manufacturing records will be verified upon inspection Ensure readiness for laboratory evaluation, if dossier assessment and inspection scheduling are underway Demonstration of test procedure by manufacturer may precede the commencement of the evaluation

36 | AIDS2012 – WHO Satellite Session l 25 July 2012 In conclusion Please read carefully the instructions for compilation of the product dossier and the information for manufacturers regarding inspection –Contact us early, if there is anything that is not clearly understood –Via our address: We are available for one-on-one sessions by teleconference or face-to-face meetings