Top Trial Cheats Dr Rod Stables The Cardiothoracic Centre Liverpool UK No Conflicts of Interest to Declare

Presentation Outline Examples without accusation Three topic areas Establishing the denominators Lost or not evaluated at follow-up Odds ratio v Risk ratios Composite outcome measures Mixing safety and efficacy measures

Identify the Denominator: Tracking Trial Subjects Need to follow the fate of ALL patients From inclusion to outcomes Warning signs ! Results expressed as simple %age Smokescreen phrases : ‘In patients with outcome data …’ ‘In evaluable (surviving) patients …’ ‘When assessed at follow-up …’

Identify the Denominator: Examples Rickards and Stables CABRI long term follow-up Widely presented (but never published) Every result presented as a % value Masked incomplete follow-up No data on >120 Dutch patients More sinister impact - reported outcome measures

Not Assessed for the Outcome Measure ? Trial of drug A - exercise capacity in heart failure 100 patients randomised 1 : 1 drug v placebo Outcome measure Mean exercise duration at FUP Baseline exercise duration - equal Follow up Drug 3.2 mins Placebo 2.4 mins Clinical and statistical significance Mechanism ?

Not Assessed for the Outcome Measure ? Widespread implications Incomplete QCA follow up in angiographic trials Numerical handling of extreme examples QCA - MLD of occluded vessel ? = 0 QoL - score of dead patient = 0 (?excluded) Competing outcomes ↑ mortality = ↓ repeat revascularisation Event curve analysis - No of patients at risk

SoS Trial: Repeat Revasc Event Curve

BASIL Trial : Amputation - free Survival

Solutions ? Clear exposition of patient flows

Trial Profile and Patient Flow Diagram

Solutions ? Clear exposition of patient flows All event rates expressed: n / d (p%) (CIs) Describe status of patients not assessed for outcome measures Express rates with correct denominators eg Repeat revasc rate per 100 patient years of Fup Full description of complex data handling methods QCA QoL etc

Odds and Risk

Throw one of a pair of dice - Aim - To get a 6 Odds against = 5:1 (Odds) Event occurs 1:6 of all throws (Risk) Scope for confusion and manipulation of data ……. and audience Most observers interpret odds ratio as relative risk

Example: Relative Risk in a Classic RCT Randomised trial Tx A v Tx B Each group comprises 100 patients Death rate on Tx A = 50/100 (= 0.5) Death rate on Tx B = 25/100 (= 0.25) Relative risk = 0.5 / 0.25 = 2 (p values and confidence intervals)

Odds Ratio Calculation Tx A: No dead = 50 (a) No alive = 50 (c) Tx B: No dead = 25 (b) No alive = 75 (d) Odds ratio = ad/bc = (50 x 75)/(25 x 50) = 3 Sounds more !!

From the Thesis of Dr R H Stables DM (Oxon) …..

Relative Risk and Odds Ratio If event rate is low OR is approx = RR If OR is interpreted as RR it will overstate the risk (benefit) Reported Ratio Event Rate %

Some Basic Rules Odds ratio correct in 2 x settings Backward investigation - Case control

Case Control Studies Breast cancer patients - series of 200 History of OCP = 50 (a) No OCP = 150 (c) Odds = a/c = 0.33 Age sex matched controls - series of 200 History of OCP = 25 (b) No OCP = 175 (d) Odds = b/d = 0.14 Odds Ratio = (a/c) / (b/d) = a/c x d/b Odds ratio = ad/bc = (50 x 175)/(150 x 25) = 2.3

Some Basic Rules Odds ratio correct in 2 x settings Backward investigation - Case control Multiple regression analyses Report independent association strength factor and outcome In all others - RCT and cohort studies Risk ratio applicable and best

Mixing Efficacy and Safety Outcomes

Developing area - Subject of continued debate CVA addition to MACE in revascularisation trials PCI trials - MACE outcomes ? Add in bleeding complications Usually PCI related (puncture ± adjunct drugs) Seems reasonable ? Value in trials of antithrombotic therapy ?

Mixing Efficacy and Safety Outcomes Cloud and confuse : Data analysis and reporting Procedural development BENESTENT 1 Stent puncture site complications 7% !! Unrelated to coronary efficacy of stent Reduced to current levels - changes in Drugs devices techniques - experience

Mixing Efficacy and Safety Outcomes Efficacy and safety issues Parallel tracks - causation and solution Further complicate analysis and reporting Minefield of composite outcomes Perhaps best avoided

Questions and Discussion