1. cap.org v. 1 Gynecologic Consensus Conference Working Group 3, Topic 6: General Quality June 4, 2011

2. Joseph Tworek, MD (Senior Author) Lydia P. Howell, MD (Chair) Ritu Nayar, MD Sana O. Tabbara, MD Barbara Winkler, MD Lynnette Savaloja, SCT Nicole E. Thomas, MPH, CT(ASCP), (CAP Staff) Working Group 6 © 2011 College of American Pathologists. All rights reserved. 2

3. Historical data and national benchmarks o Useful for smaller labs o Historical data will identify trends within lab o Published benchmarks may identify lab drift −National benchmarks not always available Available methods for monitoring quality data © 2011 College of American Pathologists. All rights reserved. 3

4. Justification: Survey and website Monitoring laboratory-wide data against national benchmarks may provide a baseline to identify and stratify lab performance o Not valuable for labs with small numbers of primary screeners o Taken in context with other factors (eg, high-risk population) Comparing individual data to laboratory-wide data may help identify outliers o Retain with other QA documents Statement: Selected metrics should be monitored individually, as well as globally for the laboratory. © 2011 College of American Pathologists. All rights reserved. 4

5. A.Agree with entire statement 95.92% B.Only individual quality data should be monitored; no global monitoring. 0% C.Only global laboratory monitoring; no individual monitoring. 0% D.Disagree with entire statement (ie, quality data should not be monitored at all). 2.04% Vote#56: Selected metrics should be monitored individually, as well as globally for the laboratory. © 2011 College of American Pathologists. All rights reserved. 5

6. A.Agree with entire statement. 92.9% B.Only cytotechnologist quality data should be monitored. 3.57% C.Only pathologist quality data should be monitored. 1.79% D.Disagree with entire statement (ie, individual quality data should not be monitored at all). 1.79% Vote#57: Monitoring of selected metrics for individuals should include both CTs and Pathologists © 2011 College of American Pathologists. All rights reserved. 6

7. Justification: Survey Quality metrics should be shared with each CT and pathologist o From survey, 59% and 81% of labs facilitate comparison of CT to other CTs and to laboratory data respectively o 48% and 60% of labs facilitate comparison of pathologists to other pathologists and to laboratory data respectively o Table 3 page 55 Lab mean data and/or individual data could be shared openly or privately, identified or de- identified at the discretion of the lab Statement: Results of quality metrics should be shared with individual CTs and pathologists. © 2011 College of American Pathologists. All rights reserved. 7

8. A.Agree with entire statement. 98.39% B.Quality metrics should only be shared with CTs. 1.61% C.Quality metrics should only be shared with Paths. 0% D.Disagree with the entire statement (ie, quality metrics should not be shared at all). 0% Vote#58: Results of quality metrics should be shared with individual CTs and pathologists. © 2011 College of American Pathologists. All rights reserved. 8

9. Justification: o Survey −Most common is monthly (62.3% prepare quality report monthly). −Helpful in semi-annual evaluation of CT (eg, determining screening limits) −Labs can decide whether to de-identify results when sharing: – Most labs (70% from survey) do not code CT and pathologists results to maintain confidentiality. Statement: Results of quality metrics should be shared at least twice a year with individuals. © 2011 College of American Pathologists. All rights reserved. 9

10. A.Agree with entire statement. 65.6% B.Time frame is too frequent. 0% C.Time frame is too infrequent. 9.4% D.Time frame should be left to discretion of the lab. 25% E.Disagree with entire statement (ie, quality metrics should not be shared with individuals at all). 0% Vote#59: Results of quality metrics should be shared at least twice a year with individuals. © 2011 College of American Pathologists. All rights reserved. 10

11. Justification: Survey Multi-head review of difficult cases ranked second most useful quality metric 60% of labs conduct in-house review o Share interesting cases o Review of educational program slides o Hone diagnostic criteria o Review cases identified from QA o Review laboratory generated study material Statement: Reviewing selected cases for educational purposes is a useful quality tool. © 2011 College of American Pathologists. All rights reserved. 11

12. A.Strongly Agree 86.4% B.Agree 13.6% C.Disagree 0% D.Strong disagree 0% Vote#60: Reviewing selected cases for educational purposes is a useful quality tool. © 2011 College of American Pathologists. All rights reserved. 12

13. Areas for future development © 2011 College of American Pathologists. All rights reserved. 13

14. Justification: Survey Top 3 methods across (HPV, Cyto-histo cor., 5 yr look backs, and immediate re-screen) o Lab defined action limits/thresholds (46%-66%) o Rate change (23% - 35%) o Lab defined action limits/thresholds from literature (21% - 26%) Insufficient data to determine best methods to identify variance in lab-wide quality metrics. © 2011 College of American Pathologists. All rights reserved. 14

15. Justification: Survey Varies by metric More commonly done for CTs than Paths o 54%, identify outliers o 47%, user defined action limits (arbitrary) o 37%, rate change o 22%, compare S.D. with historical mean o 18%, user defined action limits based on literature Insufficient data to specifically describe how to identify variance in individual quality metrics. © 2011 College of American Pathologists. All rights reserved. 15

16. 85%, identify root cause analysis, address the cause and continue to monitor 42%, conduct in-lab re-education 33%, increase real time re-screen of NILM prior to sign out 32%, decrease slide work load 21%,retrospective re-screen of a defined number of previous NILM cases. Common actions from survey to address variance in lab-wide performance © 2011 College of American Pathologists. All rights reserved. 16

17. 68% identify cause of variance and conduct focused review or education 53% increase real time re-screen of NILM prior to sign out 46% counseling and continued monitoring (warning shot) 45% decrease work load limits 37% conduct in-house tutorial 31% conduct an audit of previous cases Common actions from survey to address variance in individual performance © 2011 College of American Pathologists. All rights reserved. 17

18. Working Group 3 Topic 6 General Quality Additional Voting Questions © 2011 College of American Pathologists. All rights reserved. 18

19. Voting WG3 - General Quality 68.Low-volume methodologies should have a higher-level of quality oversight/control. A.Yes, screened by designated “experts” 11.11% B.Yes, automatically re-screened 6.67% C.Both A&B 20% D.Yes, I agree with statement, but left to discretion of lab 53.33% E.No, I do not agree with statement 8.89% 19 © 2011 College of American Pathologists. All rights reserved.

20. Justification: professional opinion and literature Need more data regarding practice patterns and their validity Consider use of slide set of unknowns to assess initial competency Rescreen percentage of cases for a period of time o The percentage and time length depend upon the laboratory’s resources and patient population (ie, abnormal rate) o For labs with low volume or low abnormal rates, consider additional test of verified unknowns or if possible use “spiked” cases WG3-Statement: Newly hired primary screeners should be monitored, but best method(s) is unclear. © 2011 College of American Pathologists. All rights reserved. 20

21. 69.Newly hired primary screeners should be monitored, but best method(s) is unclear. A.Strongly agree 37% B.Agree 46.3% C.Disagree 14.8% D.Strongly disagree 1.8% Voting WG3 – General Quality © 2011 College of American Pathologists. All rights reserved. 21