2. Performance indicators in CRC screening program
Dr. Kęstutis Adamonis, Dr. Romanas Zykus,
– 22
– 15

3. Evaluation
The purpose of program evaluation is to assess the program implementation (processes), outcomes (effectiveness) and costs (efficiency).
Success of the programme is measured not only by its impact on public health, but also by its organization, implementation and acceptability.

4. Types of evaluation Process (evaluation of activities)
Impact (evaluation of direct effects of the program)
Outcome (evaluation of longer lasting benefits of the program)

5. Process evaluation
How a program works to achieve its goals and objectives
How well implementation goes
What difficulties exist.

6. Impact evaluation
Determines what direct effects the program actually has on those directly and indirectly experiencing it (also program partners and the community).
Provides information whether the program has been able to meet its short-term goals and objectives.

7. Outcome evaluation
Discerns whether a program has been able to meet its long-term goals and objectives.
Tracks the maintenance of program effects (i.e., screening over time)
Documents longer term effects on morbidity and cancer mortality.

8. Organisation and management
Impact and outcome
evaluation
Process evaluation
Organisation and management
Clinical
Process
Impact and outcome

9. Organisation and management Clinical Process Impact and outcome
Organization indicators
Clinical process and quality indicators
Early and late impact indicators
Organisation and management
Clinical
Process
Impact and outcome

10. To determine whether a programme has been effective with respect to its impact on mortality and morbidity requires continuous follow-up of the target population over an extended period of time, and ascertainment and recording of the outcomes of the screening process and of the indicators of programme impact.
The monitoring and evaluation of the programme therefore require that adequate provision be made in the planning process for the complete and accurate recording of all the relevant data.
European Guidelines for Quality Assurance in Colorectal Cancer Screening and Diagnosis - First Edition (2010)

12. Data
A key component in the evaluation of population screening programmes is data collection.
The data items and information that must be collected, recorded and stored in order to generate the indicators, analyses and reports required for evaluation.
A database consisting of individual records (one record per person for each screening episode) is essential in order to produce results on screening performance.

13. Recommended minimal data to calculate different indicators (1)
Programme type
Primary screening test
Population base
Self registrations
Self referrals
Targeted
Eligible
Invited
Screened/tested at first screening and at subsequent screening episodes
Inadequate tests
Positive test or screening

14. Recommended minimal data to calculate different indicators (2)
Follow-up colonoscopy examination attended (diagnostic assessment and/or treatment)
Negative follow-up colonoscopy examination (diagnostic assessment and/or treatment)
Positive follow-up colonoscopy examination (diagnostic assessment and/or treatment)
Lesion detected (at least one)
Adenoma detected (at least one)
Non-advanced adenoma detected (at least one)
Advanced/high-risk adenoma detected (at least one)
Cancer detected by stage
Severe complications requiring hospitalisation
30 day mortality

15. European Guidelines for Quality Assurance in Colorectal Cancer Screening and Diagnosis - First Edition (2010)
Indicators
Description
Coverage by invitation
Share of people to whom the invitation was delivered by mail according to number of target population eligible for screening
Invitation response rate
Share of people who responded to the invitation according to the number of target population who received the invitation
Unreturned fit rate
Share of people who did not returned the stool test kit among people to whom the test kit was sent
Time interval between completion of test and issuing of results
Share of people who received the test result in 15 days. Sending the results to people with negative test should be included in the program algorithm
Referral to follow-up colonoscopy after fit
Share of people appointed to colonoscopy among people with positive FOBT result. Different reasons for not being appointed should be monitored
Time interval between referral after positive test and performed colonoscopy
Share of people with the first colonoscopy performed in 31 days after the day of appointing to the colonoscopy
Colonoscopy compliance rate
Share of people with at least one colonoscopy in the programme among people who were after positive FOBT appointed for colonoscopy
Time interval between laboratory receipt and histological result
Share of colonoscopies where result of histopathology diagnostics was available (send in paper form or entered to information system) in 5 or 10 working days
Time interval between diagnosis of screen-detected cancer and start of definitive treatment
Share of people with colorectal cancer who received definitive treatment in 31 calendar days after histopathology diagnosis of cancer among people who need additional treatment (surgery)
Inadequate fit rate
Share of people with at least one inadequate fit test, who did not manage to receive test result (positive or negative) among screened (sent stool samples)
Positive fit rate
Share of people with positive test result among people with test result
Colonoscopy after positive fit rate
Share of people with at least one colonoscopy in the programme among people with a positive FOBT test result
Caecal intubation rate
a. Share of total colonoscopies among all performed colonoscopies.
b. Share of total colonoscopies according to the number of people with at least one colonoscopy
Rate of high-grade neoplasia reported
Share of findings which histopathologists classified as high-grade dysplasia among all findings with histopathology diagnose
Proportion of cancer cases not requiring surgery
Share of people with colorectal cancer detected who did not need surgery, because cancer was removed at endoscopy among people with colorectal cancer as the worst finding detected at colonoscopy

16. Indicators Description
Proportion of adenoma cases referred for surgery
Share of people with adenoma as the worst finding who were referred to surgery, among people with adenoma as the worst finding at colonoscopy
Surgery compliance rate
Share of people referred to surgery with the procedure performed, according to all people referred to surgery
Positive predictive value for detection of lesions/adenoma/advanced adenoma/cancer
(1.positive predictive value of colonoscopy and 2.positive predictive value of FOBT test)
Share of people with at least one lesion /one adenoma/one advanced adenoma/one carcinoma counting the worst finding, among people at least one colonoscopy performed
Share of people with at least one lesion /one adenoma/one advanced adenoma/one carcinoma counting the worst finding, among people with positive FOBT test result
Endoscopic complications rate
Share of screening or therapeutic colonoscopies with complications among all colonoscopies Type of serious complications should be identified and reported from all colonoscopy providers inside the programme
30-day colonoscopy specific mortality
Share of people who died in 30 days after colonoscopy because of colonoscopy complications, among all people with at least one colonoscopy
Uptake/participation rate
Share of people with returned test kit (suitable for testing or not) among all people delivered the invitation minus people excluded because of previous bowel cancer or bowel decease
Lesions/adenoma/advanced adenoma/cancer detection rate of FOBT
Share of people with at least one lesion /one adenoma/one advanced adenoma/one carcinoma counting the worst finding, among people with FOBT test result (positive and negative)
Stage of screen-detected cancers
Share of people with different cancer stage among all people with cancer diagnosed inside screening programme, counting the worst finding. TNM (I, II, III, IV) classification is used
Interval cancers
Share of people with interval cancer (primary colorectal cancer) diagnosed before next planed FOBT or planed colonoscopy control, among people with FOBT test result or people with colonoscopy performed. For this definition strict rules of time period of repeated invitation to next screening round must be followed by screening managing institution

17. Colonoscopy quality indicators
Colonoscopists with sufficient number of conducted colonoscopies
Share of colonoscopists who performed at least 200 colonoscopies per year inside and outside the programme among all colonoscopists in the programme
Bowel cleansing, quality of colonoscopy preparation
Share of colonoscopies with good cleansing among all performed colonoscopies
Colonoscopy withdrawal time
Share of colonoscopies without polypectomy or biopsy with at least 6 minutes withdrawal time among all colonoscopies without polypectomy or biopsy
Adenoma detection rate (ADR) of first colonoscopy
Share of first colonoscopies with detected at least one adenoma among all first colonoscopies
Left and right colon adenoma detection proportion
Share of adenomas detected in left/ right colon among all adenomas detected in entire colon. Right hemicolon includes flexura lienalis, colon transversum flexura hepatica, colon ascendens, and cecum. Left hemicolon includes anus, rectum, colon sigmoideum and colon descendens
Sessile serrated lesion in right colon detection rate –(SSLR)
Share of first colonoscopies with at least one detected sessile serrated lesion in right hemicolon among number of all first colonoscopies
Mean adenomas per procedure – map), (Mean adenomas per positive procedure – map+)
Share of all adenomas detected in first colonoscopies among number of all first colonoscopies (map) or among number of all first colonoscopies with at least one adenoma detected (map+)
Referral to surgery or tertiary endoscopy in the same or another center
Share of people referred to surgery or additional colonoscopy for polypectomy among number of people with findings at colonoscopy

18. Summary Table of performance standards in colorectal cancer screening

19. Suggested quality of colonoscopy indicators and auditable outcomes by EU guidelines of 2010

20. Quality indicator / auditable outcomes
Mandatory
Desirable 1.
Age and sex of patient
QI/AO x 2.
Cancer detection rate (all cancers) 3.
Cancer detection rate (endoscopically removed cancers) 4.
Referral rate into surveillance programmes (total and by risk category) QI 5.
Adenoma excision and retrieval rate +/- withdrawal times
6.1
Numbers and detection rates of colorectal lesions, in total and broken down by: polypoid and nonpolypoid (Paris classification: Ip Ls, IIb IIc sessile non-neoplastic)
6.2
Numbers and rates in 6.1 broken down by sector of the colon (caecum; ascending, transverse, descending colon; sigmoid; rectum) AO
7.1
Numbers and detection rates of colorectal lesions, in total, and by predicted histology: 1) nonneoplastic (hyperplastic polyp, sessile serrated lesion, other), 2) neoplastic (low-grade adenoma, high-grade adenoma, submucosal carcinoma) and 3) uncommon lesions
7.2
Numbers and rates in 7.1 broken down by sector of the colon (caecum; ascending, transverse, descending colon; sigmoid; rectum)
8.1
Numbers and detection rates of colorectal lesions, in total, and by confirmed histology: 1) nonneoplastic (hyperplastic polyp, sessile serrated lesion, other), 2) neoplastic (low-grade adenoma, high-grade adenoma, submucosal carcinoma) and 3) uncommon lesions
8.2
Numbers and rates in 8.1 broken down by sector of the colon (caecum; ascending, transverse, descending colon; sigmoid; rectum)
9.1
Numbers and rates of discrepant lesions broken down by categories in 7.1 and 8.1
9.2
Numbers and rates of discrepant lesions broken down by categories in 7.2 and 8.2

21. Quality indicator / auditable outcomes
Mandatory
Desirable
10.
Withdrawal times from caecum to anus (in patients who have not had biopsy or therapy)
QI/AO x
11.
Colonoscopy completion rate QI
12.
Wait time: FOBT to colonoscopy
13.
Wait time: FS to colonoscopy QI
14.
Wait time: colonoscopy to pathology results QI
15.
Wait time: FS to pathology results QI
16.
Wait time: pathology results to definitive treatment QI +
17.
Unplanned admission on day of procedure: four options AO
18.
Type of insufflation gas (air or C02)
19.
Type of sedation used: three options
20.
Comfort: only if conscious or no sedation used
21.
Adequacy of preparation
22.
Delayed adverse outcomes: two optionsKey endoscopic characteristics of polyps written on pathology request form: five key characteristics: number, site, size, completeness of excision, separate pots used for different sites (see also 6–9)
23.
Lesions referred elsewhere for excision
24.
Patient feedback on information and consent, booking, environment, comfort and aftercare
25.
Adverse incidents related to incomplete preassessment
26.
Decontamination indicators

22. Europe against Cancer: Optimisation of the Use of Registries for Scientific Excellence in research (2014)
Indicator
Numerator
Denominator
Extension by screening programme
N target population within the area with the organized screening programme
N of population in corresponding age groups within the whole country
Invitational coverage
N invited during time frame
N eligible in target population
Coverage by examination
N screened or tested during time frame
Compliance to invitation (uptake rate)
Screened
Invited
Rate of inadequate tests
Inadequate
Rate of test positives
Positive test result
Referred
N with a positive test result
Referral rate to colonoscopy after positive test
Colonoscopied
Compliance to colonoscopy
Complete colonoscopies
Total colonoscopied
Rate of complete colonoscopies

23. Biopsy rate
Biopsy taken
Colonoscopied
Lesion detection rate
N with at least one lesion
Screened
Adenoma detection rate
N with at least one adenoma
Advanced adenoma detection rate
N with at least one advanced adenoma
Cancer detection rate
N with at least one cancer
PPV for detection of lesions
N with colonoscopy
PPV for detection of adenoma
PPV for detection of advanced
PPV for detection of cancer
Endoscopic complications
N with complications
Interval cancer
Cancer in screen negatives or episode negatives during the interval

24. Quality in screening colonoscopy: position statement of the European Society of Gastrointestinal Endoscopy (ESGE) 2014

25. Quality assurance item Proposed standard
Consent and withdrawal of consent
Audit the number of patients who decline colonoscopy on the day of the procedure and the number of intraprocedural withdrawals of consent.
Proposed standard: fewer than 5% of cases to withdraw consent on the day of the procedure and fewer than 1% during the procedure.
Experience of the screening colonoscopist
We recommend that a minimum lifetime colonoscopy experience together with a minimum number of annual screening colonoscopies should be agreed.
Proposed standard: to be agreed by screening boards
Bowel cleansing
The state of bowel cleansing should be audited.
Proposed standard: at least 90% of examinations should be rated as “adequate” bowel cleansing or better
Sedation, analgesia, and comfort
Audit of sedation practices, including average doses used of medication together with comfort scores.
Proposed standard: no more than 1% of patients should become hypoxic (saturation below 85% for more than 30 seconds) or for other reasons require administration of a reversal agent
Unadjusted cecal intubation rate
Audit the completion rate for all colonoscopies.
Proposed standard: unadjusted cecal intubation rate of at least 90%
Adenoma and cancer detection
rates
The number of detected adenomas and cancers should be audited.
Colonoscope withdrawal time
Average withdrawal times should be audited.
Proposed standard: a minimum of 6 minutes in at least 90% of purely diagnostic examinations
Polyp retrieval rate
Screening programs anticipate that all resected polyps are retrieved for histological analysis.
Proposed standard: ≥ 90% of resected polyps should be retrieved for histological analysis
Significant interval lesions
We recommend that screening programs monitor size, appearance, location, and histology of all polyps larger than 1 cm and cancers found between screening examinations as well as after the patient has been discharged from a screening program.

26. Quality assurance item Proposed standard
Specialist referral for removal of larger polyps
We anticipate that the removal of larger polyps will be deferred to a dedicated clinical session, perhaps at a separate tertiary referral centre. Screening programs should record how larger polyps detected at screening are managed, together with details of outcomes.
Proposed standard: to be agreed by screening boards
Cleaning and disinfection
Adoption of manufacturers’, national, and European standards for disinfection.
Proposed standard: routine microbiological testing at intervals not exceeding 3 months
Tattooing sites of larger polyps and
cancers
We recommend that screening programs set standards regarding which polyp sites should be tattooed.
Proposed standard: the placement of tattoos following the removal of all polyps 2 cm or larger outside of fixed colonic landmarks such as the cecum and rectum
Unscheduled readmissions
We recommend that screening programs record details of all emergency admissions within 30 days of the screening colonoscopy.
Perforation rate
We recommend that details should be recorded of all perforations complicating diagnostic and therapeutic
procedures, that require surgical repair and that occur up to 2 weeks after endoscopy.
Proposed standard: fewer than 1:1000 diagnostic or therapeutic examinations should result in a perforation
requiring surgical repair
Bleeding rate
All cases of immediate and late bleeding following polypectomy should be recorded.
Proposed standard: fewer than 1:20 cases of bleeding should ultimately require surgical intervention