1. Copyright H Photography 2006 iStockphoto.com Pragmatic cohort studies and comparative effectiveness Eric S Johnson, PhD Kaiser Permanente’s Center for Effectiveness and Safety Research AHRQ’s Effective Healthcare Program: Scientific Resource Center June 18, 2010 UC Davis School of Medicine http://www.itv.com/Drama/family/JeevesandWooster/picturegallery.html

2. Copyright Matt Knannlein 2005 iStockPhoto.com The risk of death associated with the use of conventional versus atypical antipsychotics among elderly patients Sebastian Schneeweiss, Soko Setoguchi, Alan Brookhart, Colin Dormuth, Philip S Wang Canadian Medical Association Journal 2007;176:627-32

3. Copyright Matt Knannlein 2005 iStockPhoto.com BMJ’s PICO format: Study question Do conventional antipsychotic drugs pose risks equal to or greater than those associated with newer, atypical drugs in an elderly population?

4. Copyright Matt Knannlein 2005 iStockPhoto.com BMJ’s PICO format: Design, intervention and control We conducted a retrospective cohort study of elderly patients with a diagnosis of dementia who started a conventional or an atypical antipsychotic drug.

5. Copyright Matt Knannlein 2005 iStockPhoto.com BMJ’s PICO format: Participants and setting We followed 37,241 patients 65 years or older who started an antipsychotic drug between 1996 and 2004 and were residents of British Columbia.

6. Copyright Matt Knannlein 2005 iStockPhoto.com BMJ’s PICO format: Outcomes and their timing We compared the 180-day all-cause mortality rate for patients who started conventional versus atypical antipsychotic drugs.

7. Copyright Matt Knannlein 2005 iStockPhoto.com BMJ’s PICO format: Results and role of chance Of the 24,359 (control cohort) patients who started an atypical antipsychotic drug, 9.6% died by 180 days. In adjusted analyses, patients who started a conventional drug were 32% more likely to die (hazard ratio=1.32; 95% CI, 1.23 to 1.42)— an excess of 3.5 deaths per 100 patients.

8. Copyright Matt Knannlein 2005 iStockPhoto.com A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers Kevin Thorpe, Merrick Zwarenstein, Andrew Oxman, and others J Clin Epidemiology 2009;62:464-75. “Pragmatic trials seek to answer the question: ‘Does the intervention work under usual conditions.’ ”

9. Copyright 2007 Robert Churchill iStockPhoto.com The pragmatic to explanatory trials continuum* Flexibility of comparison intervention Practitioner Expertise (experimental) Flexibility of the experimental intervention Eligibility criteria Primary analysis Practitioner adherence Participant compliance Outcomes Follow-up intensity Practitioner expertise (comparison) E * J Clin Epidemiology 2009;62:464-75

10. Copyright 2007 Robert Churchill iStockPhoto.com New atypical antipsychotic drug New conventional antipsychotic drug No antipsychotic drugs filled: year-long baseline Time Zero Intention-to- treat analyses to 180 days A pragmatic cohort study: new users

11. Copyright Matt Knannlein 2005 iStockPhoto.com Evaluating medication effects outside of clinical trials: New-user designs Wayne Ray Am J Epidemiol 2003;158:915-20 “[P]revalent users can introduce two types of biases: (1)Underascertainment of events that occur early in therapy (2)Inability to control for predictors that may have been altered by the study drugs.” Prevalent users also preclude meaningful propensity scores.

12. Copyright 2007 Robert Churchill iStockPhoto.com One class of intervention Usual care: New or current users? Consistency of alternatives? Any criteria for medications filled in year-long baseline? Time Zero Valid comparisons with usual care?

13. Copyright 2006 Ethan Myerson iStockPhoto.com How might we design a score to predict the usefulness of pragmatic cohort studies? How well does the cohort answer decision-makers’ questions? Inferiority or non-inferiority? How closely does the cohort resemble a pragmatic trial? Design? Analysis? How easy is the cohort for systematic reviewers to GRADE and synthesize? An unintended outcome, a harm?.

14. Copyright 2006 Ethan Myerson iStockPhoto.com “[A] national CER program should promote the professional ethos that places the interests of patients and the larger community above all other considerations.” Harold Sox and Sheldon Greenfield Annals of Intern Med 2009;151:203-5 Comparative effectiveness research: a report from the Institute of Medicine

15. Copyright Juan Monino 2006 iStockPhoto.com “He wants to look at his plate, see a steak, and say, ‘I like steak!’ ” “Give people what they want, then later you can give them what you want.” Pascal in Big Night (1996)

16. Copyright Matt Knannlein 2005 iStockPhoto.com Variation in the risk of suicide attempts and completed suicides by antidepressant agent in adults Sebastian Schneeweiss, Amanda Patrick, Daniel Solomon, Jyotsna Mehta, Colin Dormuth, Matthew Miller, Jennifer Lee, Philip S Wang Archives of General Psychiatry 2010;67:497-506

17. Copyright Matt Knannlein 2005 iStockPhoto.com Design, intervention and control: Advanced strategies for propensity scores “After plotting and comparing the distribution of propensity scores…we truncated our study population to the area of overlap. [S]ubjects with non-overlapping propensity scores may have absolute clinical indications or contraindications for a particular treatment.”

18. Copyright Matt Knannlein 2005 iStockPhoto.com Design, intervention and control: Advanced strategies for propensity scores “We constructed a high-dimensional propensity score based on the top 500 identified variables in addition to the variables identified by us…We consider results from this analysis to be the most completely adjusted… ”

19. Copyright Matt Knannlein 2005 iStockPhoto.com Design, intervention and control: Advanced strategies for propensity scores “We created adjusted Kaplan-Meier plots by weighting each subject by the inverse of his or her probability for treatment as estimated in the propensity score analysis.”

20. Probability of remaining free of suicide or suicide attempt since treatment with sertraline (36,135) or fluoxetine (22,207), weighted by the inverse probability of treatment from the propensity score. From Schneeweiss et al. Arch Gen Psychiatry 2010;67:497-506