1. What do screeners refer for checking? Allan Wilson Lead Biomedical Scientist in Cellular Pathology Advanced Practitioner in Cervical Cytology BAC 24th October 2013

2. Plan for today What do screeners refer for checking? Monklands audit of checkers National survey of checker role Interactive session BAC 24th October 2013

3. Screeners referral decisions What do screeners refer to checkers? What influences screeners referral patterns? How do we monitor referrals? Can we influence over-referral and under-referral? BAC 24th October 2013

4. Why do screeners refer slides for checking? Marked inflammation ?dyskaryosis Bland nuclear enlargement Atrophy ?Koilocytes / HPV / dyskeratosis ?GA / ?Endometrium ?Small cells ?degenerate only Local issues BAC 24th October 2013

5. Some ground rules..... Images are representative of slides All slides changed from BNA to negative by checkers are double screened All from Thinprep slides All stained using imager stain Limited biopsy confirmation Our opinions on BNA Negative could be wrong...... BAC 24th October 2013

6. Why do screeners refer slides for checking? Marked inflammation BAC 24th October 2013

12. Why do screeners refer slides for checking? Bland nuclear enlargement –?perimenopausal or post menopausal changes. BAC 24th October 2013

18. Why do screeners refer slides for checking? Atrophy BAC 24th October 2013

22. Why do screeners refer slides for checking? ?HPV / ?Koilocytes BAC 24th October 2013

24. 3a BAC 24th October 2013

28. Why do screeners refer slides for checking? ?glandular abnormality / ?endometrium BAC 24th October 2013

33. Why do screeners refer slides for checking? Immature squamous metaplasia BAC 24th October 2013

36. Why do screeners refer slides for checking? ?degenerate only BAC 24th October 2013

38. 2a BAC 24th October 2013

39. Why do screeners refer slides for checking? Check small cells ?high grade BAC 24th October 2013

44. Why do screeners refer slides for checking? Other oddities........ BAC 24th October 2013

46. What influences screeners referral patterns? Confidence levels Personal issues Fatigue Previous performance Ability Training Post –invasive audit or review BAC 24th October 2013

47. Over referral v Under referral (false +ve v false –ve!) We tend to notice over referral more quickly as it has a “nuisance” value! Under referral is potentially more dangerous and more difficult to detect Usual balance between sensitivity and specificity BAC 24th October 2013

48. How do we monitor referrals? Gut feeling? Worksheets Spreadsheets LIMS Do we have any idea of what a “normal” referral rate is? What should we do if a referral rate is deemed to be too low? BAC 24th October 2013

49. How can we influence screener referrals? Feedback! More feedback! Training Setting targets, self audit Selected slide reviews Don’t just live with over or under referral BAC 24th October 2013

50. Monklands Checker audit BAC 24th October 2013

51. What is a checker? Poorly defined How do we appoint checkers? How do we train? How do we assess competence? Variable role Often overstretched (HPV roles) BAC 24th October 2013

52. Monklands checker survey Four checkers (13-25 years experience) Asked to record categories of slides and numbers called negative or referred over an approximate 3 month period Reflective of rota system in operation BAC 24th October 2013

53. Summary of results 348 slides in audit 56 referred to medic/AP as BNA+ 36 BNA 16 Low grade dyskaryosis 3 High grade (moderate) 1 severe dyskaryosis ?invasive 52 reported as BNA+ by medic/AP 4 reported as negative BAC 24th October 2013

54. Monklands checker survey (2013) CheckerSlides checked Signed outReferred to AP/medic 1174140 (80%)34 (20%) 28270 (87%)12 (13%) 36054 (90%)6 (10%) 43228 (87%)4 (13%) Total348292 (84%)56 (16%) BAC 24th October 2013

55. Monklands checker survey (2006) CheckerSlides checked Signed outReferred to AP/medic 17249 (68%)23 (32%) 213259 (44%)73 (56%) 38660 (69%)26 (31%) 48767 (77%)20 (23%) 510589 (84%)16 (16%) Total482324 (67%)158 (33%)

56. Monklands checker survey Can we suggest referral or signing out targets based on these figures? Are there any outliers? We simply don’t know if checker 2 is cautious (or lazy!) or checker 5 is confident (or dangerous!) We need to do more work over a longer period and involve more labs BAC 24th October 2013

57. What type of slides prompted most referrals? Slides checked Signed outReferred to AP?Medic Inflammation 160134 (84%)26 (16%) Nuc. enlargement 8463 (74%)21 (26%) Atrophy 44 (100%)0 ?HPV 139 (70%)4 (30%) ?GA 2119 (90%)2 (10%) Check small cells 3027 (90%)3 (10%) ?degenerate 3634 (94%)2 (6%) TOTAL 348290 (83%)58 (17%) BAC 24th October 2013

58. Screener analysis ScreenerreferredReported as BNA+ 1 288 (29%) 2 5620 (35%) 3 242 (8%) 4 7612 (16%) 5 60 6 828 (10%) 7 64 (66%) 8 160 9 280 10 260

59. What can we learn from this? Use to focus training/updates/review sessions Reduce inappropriate referrals and encourage appropriate referrals Invasive and HG audit suggests no issue with checkers reporting referred slides Possible link between low referral rates and low high grade sensitivity....... BAC 24th October 2013

60. National survey on checker role BAC 24th October 2013

61. Who replied? Web based survey on BAC website Survey Monkey 98 responses 76 laboratories 85% of UK cytology labs BAC 24th October 2013

62. Health warning on analysis Duplicate entries from same lab How do you select the “correct response” Multiple responses when only one choice requested Accuracy of responses The “doomed” lab BAC 24th October 2013

63. Cervical Cytology labs in the UK England71 Northern Ireland 4 Scotland 9 Wales 4 Total88 BAC 24th October 2013

64. Do you have staff who perform a Checker role? Yes: 75 (99%) No: 1 (1%) BAC 24th October 2013

65. Which statement best describes their role in your department They screen all abnormal slides referred by primary screeners 30 (39%) They only screen difficult slides, ?BNA’s, “please check dots” cases 10 (13%) Perform both of the above roles 43 (57%) BAC 24th October 2013

66. Do your checkers also primary screen? Yes: 69 (91%) No: 7 (9%) BAC 24th October 2013

67. Do you set a minimum number of slides that they must screen? Yes: 38 (50%) No: 31 (41%) No response: 7 (9%) BAC 24th October 2013

68. If you do set a minimum figure, what is that figure? Wide range of responses from 20 to 3750! 34 labs did not respond Top three responses were: 300015 (20%) 750 8 (11%) 1000 6 (8%) BAC 24th October 2013

69. Do your checkers participate in rapid review/preview? Yes: 73 (50%) No: 1 (1%) No response: 2 (3%) BAC 24th October 2013

70. Do you monitor the performance of checkers? Yes: 56 (74%) No: 17 (22%) No response: 3 (4%) BAC 24th October 2013

71. If you do monitor checkers performance what do you measure? Workload 48 (63%) Number of referred abnormal slides from screeners that they agree/disagree with 22 (29%) Number of referred abnormals from checkers to consultant that are agreed/disagreed with 26 (34%) BAC 24th October 2013

72. Does your laboratory require a second opinion on slides where the checker has changed the primary screener report from abnormal to negative? Yes: 59 (74%) No: 17 (22%) BAC 24th October 2013

73. If you do is it for: All such slides 31 (41%) For High Grade changes only 26 (34%) No response 19 (25%) BAC 24th October 2013

74. IQC of checkers If you give a checker a ?small cell slide, how would you know if one checker consistently called them negative and one consistently called them severe dyskaryosis? BAC 24th October 2013

75. Audit trails and CPA……… CPA insist that we record who embeds tissue, who cuts sections and what batch number of Pap stains we use but do we record every opinion given on every slide ? BAC 24th October 2013

76. Over view of survey (1) Most labs use checkers to screen all abnormals and “? BNA’s” Most checkers also primary screen and participate in IQC Less than half of responding labs monitor the rate of slides changed from BNA to negative BAC 24th October 2013

77. Over view of survey (2) A third of labs do not get a second opinion on slides changed from BNA to negative 41% of labs set no minimum workload level Wide variation among those labs that do set a minimum workload level BAC 24th October 2013

78. Conclusions Support and recognition of checker role Ensure opinions are recorded Consider setting targets based on local practice and monitoring A minimum check should be a double screen of all slides reported as BNA and above by screener but changed to negative by checker BAC 24th October 2013

79. Conclusions (2) Review monitoring systems for screener referral and checker reporting Monitor screener referral rates (especially low referral rates) Monitor BNA “turnaround” rates by checkers National guidance on referral rates??? We cannot (and never will) agree on BNA! BAC 24th October 2013

80. Interactive session BAC 24th October 2013

89. And finally........ BAC 24th October 2013

91. Thank you for listening Any Questions? BAC 24th October 2013