CRITICAL READING OF THE LITERATURE RELEVANT POINTS: - End points (including the one used for sample size) - Surrogate end points - Quality of the performed procedures - Differences in the procedure/therapies after randomization - Follow up

SURROGATE END POINTS A laboratory measurement or clinical symptom used as a substitute for a clinically meaningful end point that measures survival directly or other relevant end points Key point: Relationship between the surrogate measure and the clinically relevant end point should be consistent and strong, based on RCT data

SURROGATE END POINTS  Good correlation between disease free survival and overall survival for patients with colon cancer receiving 5-FU after surgical resection (Sargent et al, JCO 2005; 23:8664)  Disease free survival: surrogate for overall survival

CRITICAL READING OF THE LITERATURE RELEVANT POINTS: - End points (including the one used for sample size) - Surrogate end points - Quality of the performed procedures - Differences in the procedure/therapies after randomization - Follow up

Quality of the procedures performed  Quality could change during the trial Gerard JP, JCO 2006; 24:4620.  Should TME have been performed in the standardized way described in the protocol, would the outcomes have been different? Equally distributed in both groups?

CRITICAL READING OF THE LITERATURE RELEVANT POINTS: - End points (including the one used for sample size) - Surrogate end points - Quality of the performed procedures - Differences in the procedure/therapies after randomization - Follow up

Differences in the procedures after randomization Any difference observed in the outcome can be explained by the treatment (if things have been carried out equally for both groups except the treatment under study after randomization) Should postoperative chemotherapy be equally distributed in both groups, would it have modified the outcome? Why Post Op CHT left to the clinician? Buijko, BJS 2006; 93:1215.

RECTAL CANCER SPHINCTER SAVING POST OP CHEMOTHERAPHY Main question Potentially confounding factors CONFOUNDING

CRITICAL READING OF THE LITERATURE RELEVANT POINTS: - End points (including the one used for sample size) - Surrogate end points - Quality of the performed procedures - Differences in the procedure/therapies after randomization - Follow up

Assessment intervals every 8 weeks from start of treatment Source: Panageas KS, Ben-orat L, Dickler MN,et al. When you look matters: The effect of assessment schedule on progression-free survival. JNCI 2007; 99(6): Treatment Start Last Scan with no progression Progression detected Assessed 8 weeks Actual progression time 0 weeks8 weeks16 weeks24 weeks

Role of values in assessing the evidence Critical reading of the literature is not an error finding exercise The question is to understand why a particular decision about design was made and to assess the impact on the outcome Example: QUASAR collaborative group. Adjuvant chemotherapy versus observation in colorectal cancer: a randomised trial. Lancet 2007; 370:

Quasar design and main results QUASAR (Quick And Simple And Reliable) was designed to provide large scale randomised evidence on the value of adjuvant chemotherapy (5- FU+Folinic) in CRC patients and, in particular, stage II. - Pragmatic trial design: local clinical teams categorising patients as having clear or uncertain indication for ADJ CHT. No per protocol definition of the indication of CHT, clinician decision. - Patients with uncertain indication were randomised: CH vs Obs - Yearly follow-up form that requested serious toxicity, recurrence and death. In the UK, also national mortality records. - Between 1994 and 2003, 3239 patients entered by 332 clinicians, in 150 centres in 19 countries. - 91% stage II, 71% colon and 29% rectal cancer. Rdt equally distributed in both groups. - Pathological data only available in 20% of the cases.

MortalityAdj cht 19.2% Obs 22.9% RR: 0.82 ( ) RecurrenceAdj cht 18.1% Obs 22.2% RR: 0.78 ( ) Quality of life: only significant differences during CHT Number needed to treat: - 1/0.037: 1 out 27 Mortality - 1/0.041: 1 out 24 Recurrence Quasar design and main results

Magnitude of the clinical benefit: Is 3.7% decrease in the risk of death enough? And 4.1% for recurrence? Are any other evidence supporting this benefit of borderline statistical significance? Yes. Methodological questions:  Patients included: Too heterogeneous? Could we know risk distribution based on pathology data? Rectal cancer?  What about compliance with CHT? 77% of patients received at least 80% of the full dose and 58% full dose.  Did the follow-up of the patients differ between countries? Could it influence the outcome? Questions for interpretation of the evidence added by quasar trial

Was the trial design too pragmatic? Alternatively, did the authors achieve the aim to replicate the relevant question in the clinical practice? Should we modify our clinical practice/clinical guideline due to the results of this trial? Let’s assume that this trial has proved the efficacy of adjuvant chemotherapy in stage II CRC: Which is the chemotherapy we should recommend? Health policy relevant questions: Questions for interpretation of the evidence added by quasar trial

Role of personal values and experience in the interpretation of the evidence added by a trial The decision about what kind of evidence is the most important in a particular clinical situation is a matter of clinical judgment : Evidence is not a substitute for clinical judgment All these aspects are present in a physician assessment of evidence: Better to make them explicit In a clinical relationship, values of the physician and the patient, previous experiences of the physician in similar cases as well as the meaning of the disease for the patient, how important are side effects, family support …. are important in assessing evidence and applying it to a particular patient

CONSORT statement The Revised CONSORT Statement for Reporting Randomized Trials Douglas G. Altman, DSc; Kenneth F. Schulz, PhD; David Moher, MSc; Matthias Egger, MD; Frank Davidoff, MD; Diana Elbourne, PhD; Peter C. Gøtzsche, MD; and Thomas Lang, MA, for the CONSORT Group Ann Intern Med. 2001;134: