1. Course: Research in Biomedicine and Health III Seminar 4: Critical assessment of evidence

2.  EBM steps ◦ Step 1: Formulating questions that can be answered ◦ Step 2: Finding best evidence ◦ Step 3: Quick critical assessment of the evidence ◦ Step 4: Applying evidence ◦ Step 5: Assessing effectiveness and efficiency of the process

3. 3. Critical assessment of evidence 1.Are study results valid? 2.What are the results? 3.Can the results be applied to a concrete patient?

4. Example: RCT Validity of research:  Is the clinical question clearly formulated?  Was the allocation of participants into groups random?  Was the allocation sequence know to the physician treating patients?  Were all participants analyzed and was it intention to treat analysis?  Were the participants in experimental and control group similar in known prognostic factors?  Were the participants/physicians/assessors blinded to the study groups?  Was the follow-up of participants complete?  Were the data adequately statistically analyzed?

5. Example: Your neighbor asks if zinc pastils would help her 10 year-old daughter in recovering from a common cold  The researchers state that the study was randomized and describe the randomization method.  Out of 124 children in the experimental group, 7 (6%) terminated the study, and 3 (2%) out of 125 children in the control group  During analysis, all participants were included, regardless whether they quit the study;  Authors provide detailed data on the characteristics of two groups and show that they are similar.  Authors also state that all involved in the study were blinded to the received treatment.  In the end, you are convinced that the strength of evidence in the study is high enough and that the risk of bias is small.

6. Example: RCT Study result: Outcome measures  ARR = absolute risk reduction = EER – CER  RRR = relative risk reduction = (CER – EER) / CER = ARR/CER  NNT = number needed to treat = 1/ARR  95% confidence interval (95%CI)

7. Example: Your neighbor asks if zinc pastilles would help her 10 year-old daughter in recovering from a common cold School missing (No. children missing) RRRARRNNT Placebo CER Zinc pastilles EER (CER-EER)/CERCER-EER1/ARR 20.8%18.5%(20.8%- 18.5%)/20.8% = 11.1% 20.8% - 18.5% = 2.3% 1/2.3% = 44 patients Adverse effects (e.g., poor taste in the mouth; dizinnes; irritation of the oral cavity, tongue and throat, diarrhoea) RRI (increase in RR) ARI (increase in AR) Number needed to harm (NNH) Placebo CER Zinc pastils EER (CER-EER)/CERCER-EER1/ARI 79.8%88.6%(79.8%- 88.6%)/79.8% = 11.0% 79.8% - 88.6% = 8.8% 1/8.8% = 11 patients

8. What needs to be reported in an article on RCT CONSORT statement http://www.consort-statement.org/ Checklist Flow diagram

9. http://www.consort-statement.org Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Ann Int Med 2010;152 (11):726-32 Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG, for the CONSORT Group. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. BMJ 2010;340:c869

10. checklist of essential items that should be included in reports of RCTs diagram for documenting the flow of participants through a trial

13. 13 Title and abstract 1aIdentification as a randomised trial in the title 1bStructured summary of trial design, methods, results, and conclusions (for specific guidance see CONSORT for abstracts)

14. 14 Background and objectives 2aScientific background and explanation of rationale 2bSpecific objectives or hypotheses

15. 15

16. 16 Trial design 3aDescription of trial design (such as parallel, factorial) including allocation ratio 3bImportant changes to methods after trial commencement (such as eligibility criteria), with reasons Participants 4aEligibility criteria for participants 4bSettings and locations where the data were collected

17. 17

18. 18

19. 19 Interventions 5The interventions for each group with sufficient details to allow replication, including how and when they were actually administered Outcomes 6aCompletely defined pre-specified primary and secondary outcome measures, including how and when they were assessed 6bAny changes to trial outcomes after the trial commenced, with reasons

20. 20

21. 21

22. 22 Sample size 7aHow sample size was determined 7bWhen applicable, explanation of any interim analyses and stopping guidelines

23. 23

24. 24 Randomisation  Sequence generation 8aMethod used to generate the random allocation sequence 8bType of randomisation; details of any restriction (such as blocking and block size)  Allocation concealment mechanism 9Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned

25. 25

26. 26

27. 27 Randomisation  Implementation 10 Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Blinding 11aIf done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how 11bIf relevant, description of the similarity of interventions

28. 28

29. 29

30. 30 Statistical methods 12aStatistical methods used to compare groups for primary and secondary outcomes 12bMethods for additional analyses, such as subgroup analyses and adjusted analyses

31. 31

32. 32

33. 33 Participant flow (a diagram is strongly recommended) 13aFor each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome 13bFor each group, losses and exclusions after randomisation, together with reasons Recruitment 14aDates defining the periods of recruitment and follow-up 14bWhy the trial ended or was stopped

34. 34

35. 35

36. 36 Baseline data 15A table showing baseline demographic and clinical characteristics for each group Numbers analysed 16For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups

37. 37

38. 38

39. 39 Outcomes and estimation 17aFor each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) 17bFor binary outcomes, presentation of both absolute and relative effect sizes is recommended CONSORT items and examples Results

40. 40

41. 41

42. 42 Ancillary analyses 18Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory Harms 19All important harms or unintended effects in each group (for specific guidance see CONSORT for harms

43. 43

44. 44

45. 45 Limitations 20Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses Generalisability 21Generalisability (external validity, applicability) of the trial findings Interpretation 22Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence

46. 46

47. 47

48. 48

49. 49 Registration 23Registration number and name of trial registry Protocol 24Where the full trial protocol can be accessed, if available Funding 25Sources of funding and other support (such as supply of drugs), role of funders

50. 50

51. RCT Applicability of results to individual patients  Are the participants in the study similar to patients who you want to treat with the intervention in question?  Were all clinically relevant outcomes studied?  Can the intervention be administered in the patient’s setting?  Are there other types of treatments?  Is the benefit of the intervention greater than possible risk or price?

52. 4. Clinical application of evidence Guidance for clinical practice Decision analysis: probability x Subjective assessment of the condition (utility) 5. Evaluation