cap.org v. 1 Gynecologic Consensus Conference Work Group 3: PAP Proficiency Testing, General Quality Practices, and Workload June 4, 2011

Joseph Tworek, MD (Senior Author) Lydia P. Howell, MD (Chair) Ritu Nayar, MD Sana O. Tabbara, MD Barbara Winkler, MD Lynnette Savaloja, SCT Nicole E. Thomas, MPH, CT(ASCP), (CAP Staff) Working Group 3 © 2011 College of American Pathologists. All rights reserved. 2

First mandated in Clinical Laboratory Improvement Amendments of 1988 Final Rule (42 CFR Part 405). Fed Reg 1992, 57: (CLIA 88); ;Oct. 31, 1988 Not implemented until almost two decades later. PT History © 2011 College of American Pathologists. All rights reserved. 3

Despite initial controversies: o Positive correlation between PT and work performance o Failure is uncommon −99.6% passed after 3 tests (2006 CAP PT) −Initial failure rate: 5% CT, 16% 1 o screening path, 6% 2 o screening path −First time failure rate has plateaued at 3% for past 3 years −Keenlyside et al. Am J Clin Pathol 1999; 112: −Moriarty A et al. Arch Pathol Lab Med 2009; 133: Today © 2011 College of American Pathologists. All rights reserved. 4

PAP PT Consensus Statements

Statement #1 The laboratory should have a written policy that the director (and/or designee) actively monitors results of gynecologic cytology proficiency testing.

Justification Required by Federal Regulation (CLIA ‘88) Survey findings: 84.2% of labs currently follow this practice

The laboratory should have a written policy that the director (and/or designee) actively monitors results of gynecologic cytology proficiency testing. Do you agree with this statement? o A – Yes86.36% o B – No12.12% o C – Other1.52% Vote#31 – Statement 1 © 2011 College of American Pathologists. All rights reserved. 8

Statement #2 Pap PT pass-fail rates should be monitored globally for the laboratory and by individual practitioner (ie, CT and Path).

Survey: o 94.7% of labs monitor pass-fail rate −Individual monitoring: CT= 84.7%, Path = 83.7% −Lab-wide monitoring: 51% o Expert opinion: Both help find outliers −Lab-wide data may be a baseline to find individual outliers −Monitoring individuals may facilitate identifying outliers Justification © 2011 College of American Pathologists. All rights reserved. 10

Pap PT pass-fail rates should be monitored globally for the laboratory and by individual practitioner (ie, CT and Path). Do you agree with this statement? o A – Yes66.67% o B – No31.82% o C – Other1.52% Vote #32 – Statement 2 © 2011 College of American Pathologists. All rights reserved. 11

Pap PT slides should be pre- screened by CTs for pathologists who are secondary screeners. Justification o Survey findings: 93.9% = yes o Regulation and expert opinion: Test environment should reflect normal lab practice. Statement #3 © 2011 College of American Pathologists. All rights reserved. 12

PAP PT slides should be pre- screened by CTs for pathologists who are secondary screeners. Do you agree with this statement? o A – Yes89.06% o B – No10.94% Vote#33 – Statement 3 © 2011 College of American Pathologists. All rights reserved. 13

Statement #4 For a first-time PT failure (CT and Path): o Re-enrollment for re-testing is a CLIA requirement and is sufficient. o No other remedial actions required, unless supported by other performance indicators.

Survey: o Re-enroll: 83% CT, 86% Path o Options for remedial action did not exceed 13% for CTs or 9% for Paths o Labs may recognize that a single failure is not a significant finding Justification © 2011 College of American Pathologists. All rights reserved. 15

Testing alters performance (anxiety?) Almost everyone passes eventually: o 99.6% passed after 3 tests (2006 CAP PT) −Hughes J et al. Arch Pathol Lab Med 2009; 133: −Moriarty A et al. Arch Pathol Lab Med 2009; 133: Justification, con’t © 2011 College of American Pathologists. All rights reserved. 16

For a first-time PT failure (CT and Path): o Re-enrollment for re-testing is sufficient. o No other interventions are required, unless supported by other performance indicators. Do you agree with this statement? o A – Yes94.44% o B – No5.56% Vote#34 – Statement 4 © 2011 College of American Pathologists. All rights reserved. 17

Remedial action policy should not be applied for a passed but non-perfect test (ie, score <100%), even for multiple non-perfect test scores. Justification o Survey: −81.4% report no policy to do so −Even in 2 consecutive non-perfect exams, only 5.2% take remedial action o No literature to show that a non-perfect test is a significant finding. Statement #5 © 2011 College of American Pathologists. All rights reserved. 18

Remedial action policy should not be applied for a passed but non- perfect test (ie, score <100%), even for multiple non-perfect test scores. Do you agree with this statement? o A – Yes93.75% o B – No6.25% Vote#35 – Statement 5 © 2011 College of American Pathologists. All rights reserved. 19

Statement #6 Monitoring of incorrect slide diagnoses on passed PT tests: o Discouraged from inclusion in lab PT policy o No interventions for this test finding are necessary Justification o Survey findings ―This type of monitoring only done by 26% of labs. o Literature ―Recognition that test-taking alters performance (as in statement #3). ―No literature to show that incorrect results are a significant finding.

Monitoring of incorrect slide diagnoses on passed PT tests: o Discouraged from inclusion in lab PT policy o No interventions for this test finding are necessary Do you agree with this statement? o A – Yes76.56% o B – No23.44% Vote#36 – Statement 6 © 2011 College of American Pathologists. All rights reserved. 21

Working Group 3 Additional Voting Questions © 2011 College of American Pathologists. All rights reserved. 22

Voting WG3 - PT 70.For labs performing more than one slide methodology, each methodology should be tested during PT. A.Strongly agree 14.29% B.Agree 21.43% C.Disagree 8.93% D.Strongly disagree 41.07% E.Unsure 10.71% F.Other 3.57% © 2011 College of American Pathologists. All rights reserved. 23

71.There is room for improvement in PT testing methods as a monitor, especially in light of new screening technologies. Do you agree? A.Yes 82.76% B.No 15.52% C.Other 1.72% Voting WG3 - PT © 2011 College of American Pathologists. All rights reserved. 24