1. Improving PCOR Methods: Causal Inference
Jason Gerson, PhD
Associate Director
CIMPOD 2016
February 25, 2016

2. Improving PCOR Methods: Why Methods Matter
PCORI’s Mission
PCORI was created to support research that provides high-integrity, evidence-based information to patients, clinicians, and the broader healthcare community.
Why Methods Matter
Rigorous research methods are needed to produce relevant, trustworthy findings that can improve patients’ healthcare outcomes.
Methods for PCOR
Methods include systematic processes, designs, tools, and techniques used to generate the evidence needed to answer questions about which healthcare options work best for particular patients.

3. Improving PCOR Methods: Program Goals
Identify Methods
Gaps
Identify methodological gaps relevant to the conduct of PCOR
Fund Research
Fund high impact studies which address gaps in methodological research
Disseminate Promising/Best Practices
Disseminate and facilitate the adoption of new methods to improve the conduct of PCOR

4. Methods Program Overview
73 funded projects, $76 million awarded
25 states, DC, and Quebec
Methods for Patient and Stakeholder Engagement
Methods for Patient-Centered Outcomes and Patient-Reported Outcomes
Research Related to the Ethical Conduct of PCOR/CER
Methods to Improve Study Design
Methods to Improve Validity and Efficiency of Analyses
Methods to Support Data Research Networks

5. CER Methods Program Portfolio (73 projects as of 2/25/2016)
Causal inference
Heterogeneity of treatment effect
Missing data
Treatment adherence
Some projects are classified in ≥1 category

6. Importance of CI Methods for PCOR/CER
Many important clinical CER questions cannot be answered with RCTs (impossible/undesirable/unethical) – need for observational studies and to support improvements in their analytic methods.
Comparison of complex treatment strategies sustained over long periods of time using data collected from routine clinical practice (or EHRs).
Need to “transport” findings from our funded methodological projects to other parts of the PCORI portfolio, esp. large pragmatic studies and PCORnet.
Intent of our investments in this area is to provide guidance/ recommendations to research community about what CI approaches are more or less optimal for a particular problem– toolkits, decision trees, standards, best practices.

7. Causal Inference for Effectiveness Research in Using Secondary Data
Potential Impact
Could change PCOR by improving methods for adjusting for confounding in observational studies, thereby improving the ability of those studies to generate robust findings
Engagement
A stakeholder advisory panel of 10 patients and will meet 5 times per year to provide feedback
Methods
Expansive simulation studies and clinically relevant sample studies to test and improve approaches for adjusting for confounding in database analyses
Provides guidance on the optimal use of two innovative and highly promising approaches (high-dimensional propensity scoring and targeted maximum likelihood estimation) for robust confounding adjustment in database analyses.
Sebastian Schneeweiss, MH, ScD MD, Brigham and Women’s Hospital
Boston, MA
CER Methods and Infrastructure, awarded September 2013

8. Causal analyses of electronic health record data for assessing the comparative effectiveness of treatment regimens
Potential Impact
Could change PCOR by improving existing causal inference methods for comparing complex treatment strategies and informing decision making by clinicians and patients for the management of chronic conditions
Engagement
A multi-stakeholder committee will inform and prioritize the development and dissemination of the proposed statistical methods
Methods
Simulation studies to evaluate the statistical methods; application of the methods to analyze EHR data from a recent observational study of type II diabetes
Advances and adapts existing causal inference methods to improve evaluation of realistic adaptive treatment strategies that reflect real-world adherence to treatment decisions and recommended monitoring schedules.
Romain Neugebauer, PhD
Kaiser Foundation Research Institute
Oakland, CA
CER Methods and Infrastructure, awarded September 2014

9. Patient Centered Adaptive Treatment Strategies (PCATS) using Bayesian Causal Inference
Potential Impact
Could change PCOR by improving methods for analyzing data in cases where the treatment/exposures varies over time and improve clinical practice by developing evidence-based shared decision making tools for identifying optimal PCATS at the point-of- care.
Engagement
A stakeholder advisory panel including two parent representatives, and a partnership with PR-COIN (part of PCORnet) will ensure patient and stakeholder engagement throughout the research process, including in the design, conduct, analysis, and dissemination.
Methods
Bayesian modeling strategies, simulation studies, and empirical analyses of PCATS for polyarticular Juvenile Idiopathic Arthritis (pJIA).
Develops Bayesian double robust causal inference methods for use in analyzing large registry and electronic health records to evaluate the clinical effectiveness of treatments and inform patient centered adaptive treatment strategies (PCATS).
Bin Huang, PhD
Cincinnati Children’s Hospital Medical Center
Cincinnati, OH
CER Methods and Infrastructure, awarded May 2015

10. Modeling Strategies for Observational CER: What Works Best When?
Potential Impact
Could change PCOR by helping researchers select the best method(s) for a particular observational dataset and CER question
Methods
Systematic review of existing literature on relevant analytical methods; simulations to assess performance of methods in cases where the literature is insufficient
Develops a decision tool for use in selecting the most appropriate analytical methods to control bias from treatment self-selection (“confounding by indication”) in analyses of observational data for CER.
Douglas Landsittel, PhD
University of Pittsburgh
Pittsburgh, PA
CER Methods and Infrastructure, awarded December 2013

11. Development of a Causal Inference Toolkit for Patient- Centered Outcomes Research
Potential Impact
Could change PCOR by: 1) improving the validity of analytic methods for common applications; 2) enhancing the reproducibility, transparency, and replication of PCOR; and 3) improving the utility of “real-world” data in conducting PCOR studies
Engagement
A technical advisory panel will participate in approximately three conference calls per year to provide advice on this project
Methods
Develop guidelines for the use of inverse probability weighting and g-formula analytical techniques and software to facilitate the use of these techniques
Develops a causal inference toolkit to provide a comprehensive, practical, and accessible guide to implementing inverse-probability weighting and the g-formula analytical techniques for PCOR studies using large observational data.
Yi Zhang, MS, PhD
Medical Technology & Practice Patterns Institute
Bethesda, MD
CER Methods and Infrastructure, awarded September 2013

12. Causal Inference Guidelines for Pragmatic Clinical Trials
Potential Impact
Provides investigators with better tools to estimate patient-centered effects in pragmatic clinical trials.
Engagement
Patients and stakeholders will be engaged in identifying the effects of interest in pragmatic trials and developing standards for estimating these effects.
Methods
Systematic review, focus groups, and case studies with supporting software will be used to create and implement a revised framework for the design and conduct of randomized pragmatic trials.
Develops a set of causal inference standards specifically designed for the analysis of pragmatic clinical trials.
Miguel Hernan, MPH, MD, DPH
Harvard University School of Public Health
Boston, MA
CER Methods and Infrastructure, awarded September 2015

13. Thank you