Performance indicators in CRC screening program Dr. Kęstutis Adamonis, Dr. Romanas Zykus, 2017 02 15 – 22 2017 03 09 – 15

Evaluation The purpose of program evaluation is to assess the program implementation (processes), outcomes (effectiveness) and costs (efficiency). Success of the programme is measured not only by its impact on public health, but also by its organization, implementation and acceptability.

Types of evaluation Process (evaluation of activities) Impact (evaluation of direct effects of the program) Outcome (evaluation of longer lasting benefits of the program)

Process evaluation How a program works to achieve its goals and objectives How well implementation goes What difficulties exist.

Impact evaluation Determines what direct effects the program actually has on those directly and indirectly experiencing it (also program partners and the community). Provides information whether the program has been able to meet its short-term goals and objectives.

Outcome evaluation Discerns whether a program has been able to meet its long-term goals and objectives. Tracks the maintenance of program effects (i.e., screening over time) Documents longer term effects on morbidity and cancer mortality.

Organisation and management Impact and outcome evaluation Process evaluation Organisation and management Clinical Process Impact and outcome

Organisation and management Clinical Process Impact and outcome Organization indicators Clinical process and quality indicators Early and late impact indicators Organisation and management Clinical Process Impact and outcome

To determine whether a programme has been effective with respect to its impact on mortality and morbidity requires continuous follow-up of the target population over an extended period of time, and ascertainment and recording of the outcomes of the screening process and of the indicators of programme impact. The monitoring and evaluation of the programme therefore require that adequate provision be made in the planning process for the complete and accurate recording of all the relevant data. European Guidelines for Quality Assurance in Colorectal Cancer Screening and Diagnosis - First Edition (2010)

Data A key component in the evaluation of population screening programmes is data collection. The data items and information that must be collected, recorded and stored in order to generate the indicators, analyses and reports required for evaluation. A database consisting of individual records (one record per person for each screening episode) is essential in order to produce results on screening performance.

Recommended minimal data to calculate different indicators (1) Programme type Primary screening test Population base Self registrations Self referrals Targeted Eligible Invited Screened/tested at first screening and at subsequent screening episodes Inadequate tests Positive test or screening

Recommended minimal data to calculate different indicators (2) Follow-up colonoscopy examination attended (diagnostic assessment and/or treatment) Negative follow-up colonoscopy examination (diagnostic assessment and/or treatment) Positive follow-up colonoscopy examination (diagnostic assessment and/or treatment) Lesion detected (at least one) Adenoma detected (at least one) Non-advanced adenoma detected (at least one) Advanced/high-risk adenoma detected (at least one) Cancer detected by stage Severe complications requiring hospitalisation 30 day mortality

European Guidelines for Quality Assurance in Colorectal Cancer Screening and Diagnosis - First Edition (2010) Indicators Description Coverage by invitation Share of people to whom the invitation was delivered by mail according to number of target population eligible for screening Invitation response rate Share of people who responded to the invitation according to the number of target population who received the invitation Unreturned fit rate Share of people who did not returned the stool test kit among people to whom the test kit was sent Time interval between completion of test and issuing of results Share of people who received the test result in 15 days. Sending the results to people with negative test should be included in the program algorithm Referral to follow-up colonoscopy after fit Share of people appointed to colonoscopy among people with positive FOBT result. Different reasons for not being appointed should be monitored Time interval between referral after positive test and performed colonoscopy Share of people with the first colonoscopy performed in 31 days after the day of appointing to the colonoscopy Colonoscopy compliance rate Share of people with at least one colonoscopy in the programme among people who were after positive FOBT appointed for colonoscopy Time interval between laboratory receipt and histological result Share of colonoscopies where result of histopathology diagnostics was available (send in paper form or entered to information system) in 5 or 10 working days Time interval between diagnosis of screen-detected cancer and start of definitive treatment Share of people with colorectal cancer who received definitive treatment in 31 calendar days after histopathology diagnosis of cancer among people who need additional treatment (surgery) Inadequate fit rate Share of people with at least one inadequate fit test, who did not manage to receive test result (positive or negative) among screened (sent stool samples) Positive fit rate Share of people with positive test result among people with test result Colonoscopy after positive fit rate Share of people with at least one colonoscopy in the programme among people with a positive FOBT test result Caecal intubation rate a. Share of total colonoscopies among all performed colonoscopies. b. Share of total colonoscopies according to the number of people with at least one colonoscopy Rate of high-grade neoplasia reported Share of findings which histopathologists classified as high-grade dysplasia among all findings with histopathology diagnose Proportion of cancer cases not requiring surgery Share of people with colorectal cancer detected who did not need surgery, because cancer was removed at endoscopy among people with colorectal cancer as the worst finding detected at colonoscopy

Indicators Description Proportion of adenoma cases referred for surgery Share of people with adenoma as the worst finding who were referred to surgery, among people with adenoma as the worst finding at colonoscopy Surgery compliance rate Share of people referred to surgery with the procedure performed, according to all people referred to surgery Positive predictive value for detection of lesions/adenoma/advanced adenoma/cancer (1.positive predictive value of colonoscopy and 2.positive predictive value of FOBT test) Share of people with at least one lesion /one adenoma/one advanced adenoma/one carcinoma counting the worst finding, among people at least one colonoscopy performed Share of people with at least one lesion /one adenoma/one advanced adenoma/one carcinoma counting the worst finding, among people with positive FOBT test result Endoscopic complications rate Share of screening or therapeutic colonoscopies with complications among all colonoscopies Type of serious complications should be identified and reported from all colonoscopy providers inside the programme 30-day colonoscopy specific mortality Share of people who died in 30 days after colonoscopy because of colonoscopy complications, among all people with at least one colonoscopy Uptake/participation rate Share of people with returned test kit (suitable for testing or not) among all people delivered the invitation minus people excluded because of previous bowel cancer or bowel decease Lesions/adenoma/advanced adenoma/cancer detection rate of FOBT Share of people with at least one lesion /one adenoma/one advanced adenoma/one carcinoma counting the worst finding, among people with FOBT test result (positive and negative) Stage of screen-detected cancers Share of people with different cancer stage among all people with cancer diagnosed inside screening programme, counting the worst finding. TNM (I, II, III, IV) classification is used Interval cancers Share of people with interval cancer (primary colorectal cancer) diagnosed before next planed FOBT or planed colonoscopy control, among people with FOBT test result or people with colonoscopy performed. For this definition strict rules of time period of repeated invitation to next screening round must be followed by screening managing institution

Colonoscopy quality indicators   Colonoscopists with sufficient number of conducted colonoscopies Share of colonoscopists who performed at least 200 colonoscopies per year inside and outside the programme among all colonoscopists in the programme Bowel cleansing, quality of colonoscopy preparation Share of colonoscopies with good cleansing among all performed colonoscopies Colonoscopy withdrawal time Share of colonoscopies without polypectomy or biopsy with at least 6 minutes withdrawal time among all colonoscopies without polypectomy or biopsy Adenoma detection rate (ADR) of first colonoscopy Share of first colonoscopies with detected at least one adenoma among all first colonoscopies Left and right colon adenoma detection proportion Share of adenomas detected in left/ right colon among all adenomas detected in entire colon. Right hemicolon includes flexura lienalis, colon transversum flexura hepatica, colon ascendens, and cecum. Left hemicolon includes anus, rectum, colon sigmoideum and colon descendens Sessile serrated lesion in right colon detection rate –(SSLR) Share of first colonoscopies with at least one detected sessile serrated lesion in right hemicolon among number of all first colonoscopies Mean adenomas per procedure – map), (Mean adenomas per positive procedure – map+) Share of all adenomas detected in first colonoscopies among number of all first colonoscopies (map) or among number of all first colonoscopies with at least one adenoma detected (map+) Referral to surgery or tertiary endoscopy in the same or another center Share of people referred to surgery or additional colonoscopy for polypectomy among number of people with findings at colonoscopy

Summary Table of performance standards in colorectal cancer screening

Suggested quality of colonoscopy indicators and auditable outcomes by EU guidelines of 2010

Quality indicator / auditable outcomes Mandatory Desirable 1. Age and sex of patient QI/AO x 2. Cancer detection rate (all cancers) 3. Cancer detection rate (endoscopically removed cancers) 4. Referral rate into surveillance programmes (total and by risk category) QI 5. Adenoma excision and retrieval rate +/- withdrawal times 6.1 Numbers and detection rates of colorectal lesions, in total and broken down by: polypoid and nonpolypoid (Paris classification: Ip Ls, IIb IIc sessile non-neoplastic) 6.2 Numbers and rates in 6.1 broken down by sector of the colon (caecum; ascending, transverse, descending colon; sigmoid; rectum) AO 7.1 Numbers and detection rates of colorectal lesions, in total, and by predicted histology: 1) nonneoplastic (hyperplastic polyp, sessile serrated lesion, other), 2) neoplastic (low-grade adenoma, high-grade adenoma, submucosal carcinoma) and 3) uncommon lesions 7.2 Numbers and rates in 7.1 broken down by sector of the colon (caecum; ascending, transverse, descending colon; sigmoid; rectum) 8.1 Numbers and detection rates of colorectal lesions, in total, and by confirmed histology: 1) nonneoplastic (hyperplastic polyp, sessile serrated lesion, other), 2) neoplastic (low-grade adenoma, high-grade adenoma, submucosal carcinoma) and 3) uncommon lesions 8.2 Numbers and rates in 8.1 broken down by sector of the colon (caecum; ascending, transverse, descending colon; sigmoid; rectum) 9.1 Numbers and rates of discrepant lesions broken down by categories in 7.1 and 8.1 9.2 Numbers and rates of discrepant lesions broken down by categories in 7.2 and 8.2

Quality indicator / auditable outcomes Mandatory Desirable 10. Withdrawal times from caecum to anus (in patients who have not had biopsy or therapy) QI/AO x 11. Colonoscopy completion rate QI 12. Wait time: FOBT to colonoscopy 13. Wait time: FS to colonoscopy QI 14. Wait time: colonoscopy to pathology results QI 15. Wait time: FS to pathology results QI 16. Wait time: pathology results to definitive treatment QI + 17. Unplanned admission on day of procedure: four options AO 18. Type of insufflation gas (air or C02) 19. Type of sedation used: three options 20. Comfort: only if conscious or no sedation used 21. Adequacy of preparation 22. Delayed adverse outcomes: two optionsKey endoscopic characteristics of polyps written on pathology request form: five key characteristics: number, site, size, completeness of excision, separate pots used for different sites (see also 6–9) 23. Lesions referred elsewhere for excision 24. Patient feedback on information and consent, booking, environment, comfort and aftercare 25. Adverse incidents related to incomplete preassessment 26. Decontamination indicators

Europe against Cancer: Optimisation of the Use of Registries for Scientific Excellence in research (2014) Indicator Numerator Denominator Extension by screening programme N target population within the area with the organized screening programme N of population in corresponding age groups within the whole country Invitational coverage N invited during time frame N eligible in target population Coverage by examination N screened or tested during time frame Compliance to invitation (uptake rate) Screened Invited Rate of inadequate tests Inadequate Rate of test positives Positive test result Referred N with a positive test result Referral rate to colonoscopy after positive test Colonoscopied Compliance to colonoscopy Complete colonoscopies Total colonoscopied Rate of complete colonoscopies

Biopsy rate Biopsy taken Colonoscopied Lesion detection rate N with at least one lesion Screened Adenoma detection rate N with at least one adenoma Advanced adenoma detection rate N with at least one advanced adenoma Cancer detection rate N with at least one cancer PPV for detection of lesions N with colonoscopy PPV for detection of adenoma PPV for detection of advanced PPV for detection of cancer Endoscopic complications N with complications Interval cancer Cancer in screen negatives or episode negatives during the interval  

Quality in screening colonoscopy: position statement of the European Society of Gastrointestinal Endoscopy (ESGE) 2014

Quality assurance item Proposed standard Consent and withdrawal of consent Audit the number of patients who decline colonoscopy on the day of the procedure and the number of intraprocedural withdrawals of consent. Proposed standard: fewer than 5% of cases to withdraw consent on the day of the procedure and fewer than 1% during the procedure. Experience of the screening colonoscopist We recommend that a minimum lifetime colonoscopy experience together with a minimum number of annual screening colonoscopies should be agreed. Proposed standard: to be agreed by screening boards Bowel cleansing The state of bowel cleansing should be audited. Proposed standard: at least 90% of examinations should be rated as “adequate” bowel cleansing or better Sedation, analgesia, and comfort Audit of sedation practices, including average doses used of medication together with comfort scores. Proposed standard: no more than 1% of patients should become hypoxic (saturation below 85% for more than 30 seconds) or for other reasons require administration of a reversal agent Unadjusted cecal intubation rate Audit the completion rate for all colonoscopies. Proposed standard: unadjusted cecal intubation rate of at least 90% Adenoma and cancer detection rates The number of detected adenomas and cancers should be audited. Colonoscope withdrawal time Average withdrawal times should be audited. Proposed standard: a minimum of 6 minutes in at least 90% of purely diagnostic examinations Polyp retrieval rate Screening programs anticipate that all resected polyps are retrieved for histological analysis. Proposed standard: ≥ 90% of resected polyps should be retrieved for histological analysis Significant interval lesions We recommend that screening programs monitor size, appearance, location, and histology of all polyps larger than 1 cm and cancers found between screening examinations as well as after the patient has been discharged from a screening program.

Quality assurance item Proposed standard Specialist referral for removal of larger polyps We anticipate that the removal of larger polyps will be deferred to a dedicated clinical session, perhaps at a separate tertiary referral centre. Screening programs should record how larger polyps detected at screening are managed, together with details of outcomes. Proposed standard: to be agreed by screening boards Cleaning and disinfection Adoption of manufacturers’, national, and European standards for disinfection. Proposed standard: routine microbiological testing at intervals not exceeding 3 months Tattooing sites of larger polyps and cancers We recommend that screening programs set standards regarding which polyp sites should be tattooed. Proposed standard: the placement of tattoos following the removal of all polyps 2 cm or larger outside of fixed colonic landmarks such as the cecum and rectum Unscheduled readmissions We recommend that screening programs record details of all emergency admissions within 30 days of the screening colonoscopy. Perforation rate We recommend that details should be recorded of all perforations complicating diagnostic and therapeutic procedures, that require surgical repair and that occur up to 2 weeks after endoscopy. Proposed standard: fewer than 1:1000 diagnostic or therapeutic examinations should result in a perforation requiring surgical repair Bleeding rate All cases of immediate and late bleeding following polypectomy should be recorded. Proposed standard: fewer than 1:20 cases of bleeding should ultimately require surgical intervention