External Quality Assessment (Proficiency testing) in Diagnostic (Renal) Histopathology Professor Peter Furness Leicester UK

Quality – of what? From who’s perspective?  Diagnosis – the ‘bottom line’?  Grading of severity?  Macro/micro description?  Dissection and sampling?  Processing & sectioning?  Patient identification & clerical?  Speed?

The ‘Audit Cycle’ Identify area for audit Define standards Re-define standards?? Test against standards Implement solutions No problems?? Identify solutions Identify problems

Approaches to quality evaluation Routine quality control Random / targeted double reporting Topic-specific audit ‘Benchmarking’ External Quality Assessment (‘Proficiency testing’) Technical Diagnostic Laboratory accreditation

Defining standards for biopsy examination:

Defining standards for biopsy interpretation:

Media response to medical mistakes: Independent on Sunday, May 31st 1998

Bang.

The need for objective assessment? British Medical Journal, 29 th November 1997 Results: Correlations between self assessments and test scores were poor for all three topics studies (R = 0.19, 0.21 and 0.19). Conclusions: as general practitioners cannot accurately assess their own level of knowledge… development programmes that rely on the doctors’ self perceptions to assess their needs are likely to be seriously flawed. BMJ 315: , 1997

Principles of External Quality Assessment  Central laboratory obtains samples representative of routine workload  Samples processed by participating laboratories as if part of the routine workload  Results returned to central laboratory  Analysis against defined standard(s)  Feedback  Corrective action if necessary

History of External Quality Assessment (EQA) in the UK  1960s – Clinical Chemistry laboratories  1970s – Haematology, microbiology  1980s – Immunology  1990s – Interpretative specialties  Developed from informal ‘slide clubs’  1998 – Professional requirement, with definitions of acceptable performance

Problems with histopathology EQA  Representative of routine workload?  Variation in individual practice  Need for further investigations  Is consultation acceptable?  What is the correct diagnosis?  How to handle vague / verbose / differential diagnoses?  How to provide feedback?  Individuals, not laboratories being assessed  What’s acceptable performance?  What to do about unacceptable performance?

UK solutions  ‘Routine workload’  Cases supplied by participants  Selected ‘from cases you reported in last (x) weeks’  Withholding relevant information forbidden  Include (or photograph, or describe) special stains if used  Variation in practice  Allow ‘opt out’ for specific types of case (e.g. transplants)  Is consultation allowed?  No!  What is the correct diagnosis?  What should a competent histopathologist make of the information and material provided?

Bespoke (but now rather old!) software…

Meeting of participants to decide:  Is the case suitable for personal feedback?  Clear consensus diagnosis  No misleading material  Should any of the ‘diagnoses’ be merged?  Does the ‘popularity’ of each diagnosis appropriately reflect its ‘correctness’?  Was the original diagnosis correct?  Are any of the diagnoses so wrong that they justify special consideration? NOTE: Perceived “clinical impact” of an error is NOT a criterion

Definition of ‘persistent sub-standard performance’:  Stage 1  In bottom 2.5% (or centile) of ranked scores in 2 out of 3 consecutive circulations.  ACTION: ‘Dear Colleague’ letter sent Participation now compulsory Anonymity maintained  Stage 2  In bottom 2.5% of ranked scores in 2 out of 3 consecutive circulations.  ACTION: Investigation by Chair of Advisory Panel on Quality Assurance.

Renal EQA: What do we find difficult?

Grading biopsies: Can we do it? SLE case from the UK Renal Pathology EQA Scheme Diagnosis% Lupus nephritis, WHO class II 2 Lupus nephritis, WHO class III 68 Lupus nephritis, WHO class IV 25 Lupus nephritis, WHO class V 1 Other grading system4 Grading ‘activity’. Various systems used. Those who used a scale of 0 to 24 said: 3; 4; 4; 4; 5; 7; 7; 7; 8; 8; 8; 9; 10; 10; 10; 11; 11; 11; 14.

The reaction of pathologists?  Initial hostility from a vocal minority  Almost 100% voluntary participation  No fall in participation when fees introduced  Increasing acceptance as part of professional audit and appraisal  Still (in theory) voluntary but will become a compulsory part of ‘professional recertification’ for histopathologists  Seen as far better than other forms of re-examination!  Facilitates early anonymous self-correction