Q&A – studying medicine or health-related topics at university Q&A, applying to university – 15 mins
People should not know which treatment they get People in a treatment group may experience improvements (for example, less pain) because they believe they are receiving a better treatment, even if the treatment is not actually better (this is called a placebo effect), or because they behave differently (due to knowing which treatment they received, compared to how they otherwise would have behaved). If individuals know that they are receiving (they are not “blinded” to) a treatment that they believe is better, some or all of the apparent effects of the treatment may be due either to a placebo effect or because the recipients behaved differently. Implication: Be cautious about relying on the results of treatment comparisons if the participants knew which treatment they were receiving, this may have affected their expectations or behaviour. The results of such comparisons could be misleading.
Why double blind, controlled randomized trials?
Symplicity HTN-1 study
Symplicity HTN-1 study
Symplicity HTN-1 study At 36 months: BP down 32mm Hg systolic, 14mm Hg diastolic Drop of 10mm Hg or more seen in 93% of patients
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10 Shun-Shin M J et al. BMJ 2014;348:bmj.g1937 ©2014 by British Medical Journal Publishing Group
Symplicity HTN-3 study
Symplicity HTN-3 study Blinded, randomized, sham-control group Primary end point: change in BP and incidence of major adverse effects
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10 Shun-Shin M J et al. BMJ 2014;348:bmj.g1937 ©2014 by British Medical Journal Publishing Group
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10 Shun-Shin M J et al. BMJ 2014;348:bmj.g1937 ©2014 by British Medical Journal Publishing Group
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10 Shun-Shin M J et al. BMJ 2014;348:bmj.g1937 ©2014 by British Medical Journal Publishing Group
`
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10 Shun-Shin M J et al. BMJ 2014;348:bmj.g1937 ©2014 by British Medical Journal Publishing Group
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10 Shun-Shin M J et al. BMJ 2014;348:bmj.g1937 ©2014 by British Medical Journal Publishing Group
http://www.evidentlycochrane.net/invisible-unicorn-grazing-office/ An invisible unicorn has been grazing in my office for a month… Prove me wrong.
“no evidence of a difference” “evidence of no difference” Don’t confuse “no evidence of a difference” with “evidence of no difference” Systematic reviews sometimes conclude that there is “no evidence” of effect when there is uncertainty about the difference between two treatments. This is often misinterpreted as meaning that there is no difference between the treatments compared. However, studies can never show that there is “no effect” or “no difference”. They can only rule out important effects or differences. Implication: Don’t be misled by statements of “no effect” or ”no difference” between treatments. Consider instead the degree to which it is possible to confidently rule out an important difference.
“statistical significance” Don’t confuse “statistical significance” with “importance” Statistical significance is often confused with importance. The cut-off for considering a result as statistically significant is arbitrary, and statistically non-significant results can be either informative (showing that it is very unlikely that a treatment has an important effect) or inconclusive (showing that the relative effects of the treatments compared are uncertain). Implication: Claims that results were significant or non-significant usually mean that they were not statistically significant or non-significant. This is not the same as important or not important. Do not be misled by such claims.
Key Concepts: What do we need to understand to be able to evaluate claims that are made about the effects of treatments?
Claims: are they justified? Treatments can harm Anecdotes are unreliable evidence Association is not the same as causation Common practice is not always evidence-based Newer is not necessarily better Expert opinion is not always right Beware of conflicting interests More is not necessarily better Earlier is not necessarily better Hope may lead to unrealistic expectations Explanations about how treatments work can be wrong Dramatic treatment effects are rare
Comparisons: are they fair and reliable? Comparisons are needed to identify treatment effects Comparison groups should be similar Peoples’ outcomes should be analyzed in their original groups Comparison groups should be treated equally People should not know which treatment they get Peoples’ outcomes should be assessed similarly All should be followed up Consider all of the relevant fair comparisons Reviews of fair comparisons should be systematic
Comparisons: are they fair and reliable? Peer-review and publication does not guarantee reliable information All fair comparisons and outcomes should be reported Subgroup analyses may be misleading Relative measures of effects can be misleading Average measures of effects can be misleading Fair comparisons with few people or outcome events can be misleading Confidence intervals should be reported Don’t confuse “statistical significance” with “importance” Don’t confuse “no evidence of a difference” with “evidence of no difference”
Choices: making informed choices Do the outcomes measured matter to you? Are you very different from the people studied? Are the treatments practical in your setting? Do treatment comparisons reflect your circumstances? How certain is the evidence? Do the advantages outweigh the disadvantages?
Any questions?
Thank you, please give us your feedback bit.ly/CochraneUK SRV Cochrane is a not-for-profit organization with contributors from more than 130 countries working together to produce credible, accessible health information that is free from commercial sponsorship and other conflicts of interest. Cochrane exists so healthcare decisions get better. Cochrane aims to achieve this by producing reviews that summarize the best available evidence generated through research to inform decisions about health. Lynda and I are from Cochrane UK – one of the many Cochrane centres around the world. We don’t produce Cochrane Reviews – rather, lots of our work is to disseminate and share evidence in ways which can be quickly and easily understood. Thank you, please give us your feedback bit.ly/CochraneUK