Quantitative Synthesis II: Interactive Quiz Thomas Trikalinos, MD, PhD Joseph Lau, MD Tufts EPC

Interactive Quiz: Instructions Open this presentation as a slideshow. This will activate the hyperlinks. When you come to a question, choose the red box corresponding to your choice. If you are correct, you will be directed forward. If you are incorrect, you will be directed back to the question to choose again. – – Click on forward hyperlinks (Red Boxes) to follow through based on your responses. – – Click on home hyperlink (Blue House Icon) to go back to the last correct step in the series.

Participation Quiz: Question 1 Assume that you are performing a meta- analysis of RCTs on vitamin D supplementation and mortality in the institutionalized elderly. You want to explore whether the effect differs across trials according to their maximum followup. Maximum length of followup is A patient-level covariate A study-level covariate

Q1: A Study-level Covariate Correct! Study-level covariates pertain to the whole study, and do not vary across patients in the same study. Maximum followup is the same for all patients in a study. [SELECT RED BOX]

Q1: A Patient-level Covariate Not exactly… – – Patient level covariates differ across patients in the same study or in the same study arm. – – Maximum followup length is the same for all patients in a study. [Click on Blue Box to Go Back]

Participation Quiz: Question 2 What if you wanted to explore whether the effect differs across trials by protocol- specified vitamin D dose? The protocol specified dose is A patient-level covariate A study-level covariate

Q2: A Study-level Covariate Correct! All characteristics of the protocol or the study design are study- level covariates. [SELECT RED BOX]

Q2: A Patient-level Covariate Not exactly… – – Characteristics of the protocol or the study design apply to the whole study (the protocol does not differ across participants) [Click on Blue Box to Go Back]

Participation Quiz: Question 3 What if you wanted to explore whether the effect differs across trials by the received vitamin D dose (assume that you actually know this information)? The mean dose of vitamin D that was actually received is A patient-level covariate A study-level covariate

Q3: A Study-level Covariate Not exactly… – – Strictly speaking, the received dose is a patient-level covariate, as it may be different than the protocol-specified dose. – – Whenever the covariate is a mean level of a quantity or a proportion of patients with some characteristic, it is something that varies at the patient level. [Click on Blue Box to Go Back]

Q3: A Patient-level Covariate Correct! Whenever the covariate is a mean level of a quantity or a proportion of patients with some characteristic, it is something that varies at the patient level [SELECT RED BOX]

Participation Quiz: Question 4 Because of the risk of ecological fallacy, meta-regressions on patient-level covariates should never be performed. I agree I disagree

Q4: I agree Never say never. – – Meta-regressions of patient-level covariates are not by definition false. For example, the average received vitamin D dose in a well conducted RCT may be very close to the protocol dose and may be practically representative of the dose received by each patient. – – Further, meta-regression is about exploration and forming of hypotheses. If an association from a meta-regression on the mean of a patient-level covariate is biologically plausible, it merits further study. [Click on Blue Box to Go Back]

Q4: I disagree Correct! Meta-regression is about exploration and forming of hypotheses. If an association from a meta-regression on the mean of a patient-level covariate is biologically plausible, it merits further study. [SELECT RED BOX]

Participation Quiz: Question 5 Communication or Misinterpretation [Go to the next slide]

Question 5: Vitamin D and Effects on Health Ann Epidemiol 2009 Which is more likely to hold? These conclusions are based on a well conducted meta- regression (see on the right) and are definitely valid The analyses and conclusions are suspect Mean 25(OH)D concentration

Q5: Analyses and Results Are Valid As a general rule, one should be suspicious of conclusions that are too strong and results that seem too good to be true… Take another guess… [Click on Blue Box to Go Back]

Q5: Analyses and Results Are Suspect Correct! This is a problematic analysis. – – Ecological fallacy may operate here. (Why?) – – The meta-regression is not correctly specified. You have to read the study and the papers cited therein to know this for sure, but there is at least one telltale sign of a mis-specified meta-regression: the scale of the Y-axis should be logarithmic for the OR and the RR (it should be linear for RD). Here, it is linear for the OR, a bizarre choice. (Can you find more?) [SELECT RED BOX]

Congratulations! You have successfully completed the quiz!