1. Q&A – studying medicine or health-related topics at university
Q&A, applying to university – 15 mins

2. People should not know which treatment they get
People in a treatment group may experience improvements (for example, less pain) because they believe they are receiving a better treatment, even if the treatment is not actually better (this is called a placebo effect), or because they behave differently (due to knowing which treatment they received, compared to how they otherwise would have behaved). If individuals know that they are receiving (they are not “blinded” to) a treatment that they believe is better, some or all of the apparent effects of the treatment may be due either to a placebo effect or because the recipients behaved differently.
Implication:  Be cautious about relying on the results of treatment comparisons if the participants knew which treatment they were receiving, this may have affected their expectations or behaviour. The results of such comparisons could be misleading.

3. Why double blind, controlled randomized trials?

5. Symplicity HTN-1
study

6. Symplicity HTN-1
study

7. Symplicity HTN-1
study
At 36 months:
BP down 32mm Hg systolic, 14mm Hg diastolic
Drop of 10mm Hg or more seen in 93% of patients

8. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report.
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10
Shun-Shin M J et al. BMJ 2014;348:bmj.g1937
©2014 by British Medical Journal Publishing Group

9. Symplicity HTN-3
study

10. Symplicity HTN-3
study
Blinded, randomized, sham-control group
Primary end point: change in BP and incidence of major adverse effects

11. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report.
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10
Shun-Shin M J et al. BMJ 2014;348:bmj.g1937
©2014 by British Medical Journal Publishing Group

12. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report.
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10
Shun-Shin M J et al. BMJ 2014;348:bmj.g1937
©2014 by British Medical Journal Publishing Group

13. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report.
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10
Shun-Shin M J et al. BMJ 2014;348:bmj.g1937
©2014 by British Medical Journal Publishing Group

15. `

17. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report.
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10
Shun-Shin M J et al. BMJ 2014;348:bmj.g1937
©2014 by British Medical Journal Publishing Group

18. Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding. Each point represents the point estimate of reduction in systolic blood pressure from one trial report.
Reported reductions in systolic blood pressure according to whether there was randomisation, whether blood pressure was documented automatically or by a doctor, and whether there was blinding.9 Each point represents the point estimate of reduction in systolic blood pressure from one trial report. As the quality of the trial design increased, the reported effect size decreased. The Symplicity HTN-3 trial is unique in being randomised, blood pressure being documented by a blinded member of staff, and the patient being blinded using a placebo procedure. This trial failed to meet its primary endpoint. Our mathematical prediction is that its effect size will be in the dotted area10
Shun-Shin M J et al. BMJ 2014;348:bmj.g1937
©2014 by British Medical Journal Publishing Group

19.
An invisible unicorn has been grazing in my office for a month… Prove me wrong.

20. “no evidence of a difference” “evidence of no difference”
Don’t confuse
“no evidence of a difference”
with
“evidence of no difference”
Systematic reviews sometimes conclude that there is “no evidence” of effect when there is uncertainty about the difference between two treatments. This is often misinterpreted as meaning that there is no difference between the treatments compared. However, studies can never show that there is “no effect” or “no difference”. They can only rule out important effects or differences.
Implication:  Don’t be misled by statements of “no effect” or ”no difference” between treatments. Consider instead the degree to which it is possible to confidently rule out an important difference.

21. “statistical significance”
Don’t confuse
“statistical significance”
with “importance”
Statistical significance is often confused with importance. The cut-off for considering a result as statistically significant is arbitrary, and statistically non-significant results can be either informative (showing that it is very unlikely that a treatment has an important effect) or inconclusive (showing that the relative effects of the treatments compared are uncertain).
Implication:  Claims that results were significant or non-significant usually mean that they were not statistically significant or non-significant. This is not the same as important or not important. Do not be misled by such claims.

22. Key Concepts: What do we need to understand to be able to evaluate claims that are made about the effects of treatments?

23. Claims: are they justified?
Treatments can harm
Anecdotes are unreliable evidence
Association is not the same as causation
Common practice is not always evidence-based
Newer is not necessarily better
Expert opinion is not always right
Beware of conflicting interests
More is not necessarily better
Earlier is not necessarily better
Hope may lead to unrealistic expectations
Explanations about how treatments work can be wrong
Dramatic treatment effects are rare

24. Comparisons: are they fair and reliable?
Comparisons are needed to identify treatment effects
Comparison groups should be similar
Peoples’ outcomes should be analyzed in their original groups
Comparison groups should be treated equally
People should not know which treatment they get
Peoples’ outcomes should be assessed similarly
All should be followed up
Consider all of the relevant fair comparisons
Reviews of fair comparisons should be systematic

25. Comparisons: are they fair and reliable?
Peer-review and publication does not guarantee reliable information
All fair comparisons and outcomes should be reported
Subgroup analyses may be misleading
Relative measures of effects can be misleading
Average measures of effects can be misleading
Fair comparisons with few people or outcome events can be misleading
Confidence intervals should be reported
Don’t confuse “statistical significance” with “importance”
Don’t confuse “no evidence of a difference” with “evidence of no difference”

26. Choices: making informed choices
Do the outcomes measured matter to you?
Are you very different from the people studied?
Are the treatments practical in your setting?
Do treatment comparisons reflect your circumstances?
How certain is the evidence?
Do the advantages outweigh the disadvantages?

27. Any questions?

28. Thank you, please give us your feedback bit.ly/CochraneUK
SRV
Cochrane is a not-for-profit organization with contributors from more than 130 countries working together to produce credible, accessible health information that is free from commercial sponsorship and other conflicts of interest.  
Cochrane exists so healthcare decisions get better.
Cochrane aims to achieve this by producing reviews that summarize the best available evidence generated through research to inform decisions about health. 
Lynda and I are from Cochrane UK – one of the many Cochrane centres around the world.
We don’t produce Cochrane Reviews – rather, lots of our work is to disseminate and share evidence in ways which can be quickly and easily understood.
Thank you, please give us your feedback bit.ly/CochraneUK