The golden standard or fools gold Selective data, impact on safety Clinical trials on trial HAI Europe Open Seminar 2008 Neues Stadthaus, Berlin, 21 November 2008 Joan-Ramon Laporte

RCTs – The golden standard or fools gold? Internal validity External validity - Efficacy vs Effectiveness Publication bias Fraud

Internal validity Did the control group receive optimal treatment?

Internal validity Did the control group receive optimal treatment? Was the dose of the control group adequate?

Favours atypical Favours haloperidol Drop out rates by dose of comparator drug in trials of patients with schizophrenia or related disorders (risk difference and 95 % confidence intervals) Geddes et al., mg haloperidol > 12 mg haloperidol

Internal validity Did the control group receive optimal treatment? Was the sample size adequate to identify any relevant difference? Was the dose of the control group adequate?

Internal validity Did the control group receive optimal treatment? Did the published results refer to the primary variable? Was the sample size adequate to identify any relevant difference? Was the dose of the control group adequate?

Internal validity Did the control group receive optimal treatment? Have all the trial results been published? Did the published results refer to the primary variable? Was the sample size adequate to identify any relevant difference? Was the dose of the control group adequate?

Internal validity Did the control group receive optimal treatment? Were the results presented as a relative risk reduction, or as an absolute risk reduction? Have all the trial results been published? Did the published results refer to the primary variable? Was the sample size adequate to identify any relevant difference? Was the dose of the control group adequate?

RCTs – The golden standard or fools gold? Internal validity External validity - efficacy vs effectiveness Publication bias Fraud

External validity of clinical trials Context Reference population Selection criteria Diagnostic criteria Follow up Treatments (doses, compliance) Duration Primary and other variables Adverse effects Lancet 2005; 365: 82-93

(Un)transferability to clinical practice Patients in RCTs differ from those in real practice: –Age –Comorbidity –Other treatments –Doses taken, compliance –Diagnostic criteria

Efficacy vs effectiveness RCT UCP Nº of patients Duration Short Longer Populations High risk groups Potentially the excluded whole population ComorbidityGenerally excludedOften present Conditions Well definedIll-defined Nº of drugs One or limited Undetermined Dose/dosage Generally constantOften variable Pattern of use ContinuousIntermittent Follow up CarefulLess careful

RCTs – The golden standard or fools gold? Internal validity External validity - efficacy vs effectiveness Publication bias Fraud

RCTs – The golden standard or fools gold? Internal validity External validity - efficacy vs effectiveness Publication bias Fraud

Third Report of the Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (Adult Treatment Panel III) (ATP III) Financial Disclosure: Dr Grundy has received honoraria from Merck, Pfizer, Sankyo, Bayer, and Bristol-Myers Squibb. Dr Hunninghake has current grants from Merck, Pfizer, Kos Pharmaceuticals, Schering Plough, Wyeth Ayerst, Sankyo, Bayer, AstraZeneca, Bristol-Myers Squibb, and G. D. Searle; he has also received consulting honoraria from Merck, Pfizer, Kos Pharmaceuticals, Sankyo, AstraZeneca, and Bayer. Dr McBride has received grants and/or research support from Pfizer, Merck, Parke-Davis, and AstraZeneca; has served as a consultant for Kos Pharmaceuticals, Abbott, and Merck; and has received honoraria from Abbott, Bristol-Myers Squibb, Novartis, Merck, Kos Pharmaceuticals, Parke-Davis, Pfizer, and DuPont. Dr Pasternak has served as a consultant for and received honoraria from Merck, Pfizer, and Kos Pharmaceuticals, and has received grants from Merck and Pfizer. Dr Stone has served as a consultant and/or received honoraria for lectures from Abbott, Bayer, Bristol-Myers Squibb, Kos Pharmaceuticals, Merck, Novartis, Parke-Davis/Pfizer, and Sankyo. Dr Schwartz has served as a consultant for and/or conducted research funded by Bristol-Myers Squibb, AstraZeneca, Merck, Johnson & Johnson-Merck, and Pfizer.

Conclusions The RCT is the best epidemiological method for causal inference However, it is often performed in a way which favours the sponsors treatment: –In its design –In data analysis and interpretation –In the publication of results At best, RCTs are one of many pieces of evidence about therapeutic interventions

Conclusions The appraisal of innovation should not only take into account the so-called EBM, but also other evidence: –Pharmacodynamics –Pharmacokinetics –Availability of therapeutic alternatives The medical literature is no longer reliable for valid information Research should be performed, analyzed and published in a way which should be independent from commercially interested parties