1. 1 Module PANEL TESTING Intermediate level

2. 2 Content Overview What is panel testing? What is panel testing used for? Organization of a panel testing round Analysis of results; scoring system Forms Feedback

3. 3 What is Panel Testing? One of EQA methods One of EQA methods System of sending stained and/or unstained smears from NRL to peripheral laboratories to check proficiency in performing AFB smear microscopy and reporting AFB results System of sending stained and/or unstained smears from NRL to peripheral laboratories to check proficiency in performing AFB smear microscopy and reporting AFB results Tests individual performance of a laboratory worker, not the laboratory overall Tests individual performance of a laboratory worker, not the laboratory overall

4. 4 Rapid assessment of performance of a laboratory staff to prioritize training and supervisory activities Rapid assessment of performance of a laboratory staff to prioritize training and supervisory activities Quick detection of problems associated with very poor performance Quick detection of problems associated with very poor performance Evaluation of competency of laboratory technicians prior to and following training Evaluation of competency of laboratory technicians prior to and following training Supplement to rechecking programs Supplement to rechecking programs Investigation of excessive errors found in recheckingInvestigation of excessive errors found in rechecking What is Panel Testing Used For?

5. 5 Panels’ Composition The composition of a panel set is determined by NRL The composition of a panel set is determined by NRL 10 slides per panel sets: stained and unstained smears 10 slides per panel sets: stained and unstained smears Unstained smears: Unstained smears: Evaluate staining technique; provide information about stain preparation and qualityEvaluate staining technique; provide information about stain preparation and quality Different degrees of difficulty Different degrees of difficulty degree of difficulty is usually determined by number of scanty and low-positive smearsdegree of difficulty is usually determined by number of scanty and low-positive smears

6. 6 National reference laboratory Intermediate laboratory Peripheral laboratories A POSSIBLE SCHEME OF A PANEL TESTING ROUND Peripheral laboratories

7. 7 Getting Started: Issues to Consider System for sending slides Frequency of testing Forms to record and report results Time allowed for technicians to complete PT Availability of microscopes Scoring system, performance criteria Analysis of PT results, feedback

8. 8 Sending Slides Delivery system based on services, regulations, resources available: mail/post courier supervisory visit Turnaround time Safe package to prevent breakage of slides: strong plastic slide holders

9. 9 Performing a Panel Test Round Frequency: at least one to two times a year Frequency: at least one to two times a year A standardized PT reporting form / an accompanying letter to provide instructions A standardized PT reporting form / an accompanying letter to provide instructions Individual, not group work Individual, not group work No incentives or punitive actions as a result of the PT exercise No incentives or punitive actions as a result of the PT exercise Time allowed to complete the PT exercise, Time allowed to complete the PT exercise,maximum: 2 hours for a stained slide set2 hours for a stained slide set 3 hours for an unstained slide set3 hours for an unstained slide set

10. 10 Individual Results of Panel Testing / Feedback Form

11. 11 Management of PT During a Supervisory Visit Administration of PT during on-site visits: Administration of PT during on-site visits: can be effective in some circumstancescan be effective in some circumstances provides direct observation of work under PT exerciseprovides direct observation of work under PT exercise corrective action may be easily facilitatedcorrective action may be easily facilitated BUT: may be impractical in routine conditionsBUT: may be impractical in routine conditions can be done in a special surveycan be done in a special survey Important: PT must not disrupt routine patients’ examinations, therefore consider: Important: PT must not disrupt routine patients’ examinations, therefore consider: Careful planning of a supervisory visitCareful planning of a supervisory visit Allocating sufficient time for a visitAllocating sufficient time for a visit

12. 12 Analysis of PT Results Scoring system to distinguish major and minor errors, for example: Scoring system to distinguish major and minor errors, for example: Set of 10 slides, each slide is worth 10 points, total possible score = 100 HFP and HFN scores 0HFP and HFN scores 0 LFP, LFN and QE scores 5LFP, LFN and QE scores 5 (QE = 2 grades difference) Passing score = 80 – 90Passing score = 80 – 90

13. 13 Types and Classification of Errors Correct:No errors QEQuantification errorMinor error LFNLow False NegativeMinor error LFPLow False PositiveMinor error HFNHigh False NegativeMajor error HFPHigh False Positive Major error

14. 14 Analysis of PT Results Study the aggregate results from all laboratories Study the aggregate results from all laboratories Post-validate panel slides/batches Post-validate panel slides/batches Assure that poor performance is not due to panel slide problems in NRLAssure that poor performance is not due to panel slide problems in NRL If a majority of technicians fail to report correct results for the same slide/batch it may represent a problem with slide preparation at NRL: → exclude this slide from scoring → check returned discrepant slides → detect problems in preparation of panel smears → undertake measures to improve the quality of panel smears preparation

15. 15 An Example of Post-Validation

16. 16 Interpretation of Results False positive and negative errors should be considered separately False positive and negative errors should be considered separately False positives - lack of proficiency / faulty microscope False positives - lack of proficiency / faulty microscope False negatives - poor stain / inadequate examination time / poor microscope False negatives - poor stain / inadequate examination time / poor microscope

17. 17 Feedback to Laboratories on PT results Timely and confidential Timely and confidential Individual and aggregate test results Individual and aggregate test results Criteria for acceptable performance Criteria for acceptable performance Reports to TB program coordinator should provide appropriate background information and recommendations and not simply scores Reports to TB program coordinator should provide appropriate background information and recommendations and not simply scores Poor performance often requires a visit to laboratory Poor performance often requires a visit to laboratory

18. 18 PT Aggregate Results of Multiple Laboratories

19. 19 PT Aggregate Results Report: Example

20. 20 Key Messages: PT is an effective method when it is necessary to quickly obtain information about capabilities of individual laboratory technicians to read smears and report results according to standards approved by NTP. A well functioning system should be established to distribute panels, collect and analyze data; provide timely feedback to peripheral laboratories. A well functioning system should be established to distribute panels, collect and analyze data; provide timely feedback to peripheral laboratories.