Borrowing Strength across Outcomes to Strengthen Impact Estimates: A Hierarchical Bayesian Approach Presentation at the International Conference on Health Policy Statistics Charleston, SC January 12, 2018 Lauren Vollmer • Mariel Finucane • Randy Brown The contents of this presentation are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Department of Health and Human Services or any of its agencies.
Game Plan Problem: Impact estimates are often based on samples that are too small to produce reliable estimates Solution: Hierarchical Bayesian models borrow strength across related impact estimates Today: The Comprehensive Primary Care Initiative Borrowing strength across: Regions Outcomes
Case Study: Comprehensive Primary Care (CPC) Initiative CPC seeks to improve delivery of primary care by enhancing: Access and continuity of care Planned chronic and preventive care Risk-stratified care management Patient and caregiver engagement Coordination of care across the medical neighborhood CPC provides: Money to support the changes (~$20/patient/month) Data feedback on practices’ and regions’ performance relative to peers A learning environment so that practices can learn from experts and each other CMS/MPR is evaluating the effects of CPC on Medicare expenditures, quality of care, and patient outcomes: From ~500 CPC practices In 7 regions (states/metro areas) of the U.S. Serving 2.5 million patients (~500,000 Medicare) Compared to patients in propensity-score matched comparison practices, using traditional difference-in-differences We’ll showcase the advantages of this approach using a case study from Mathematica’s evaluation of CPC.
Borrowing Strength Across Outcomes Frequentist Bayesian
Borrowing Strength Across Outcomes Frequentist Bayesian
Borrowing Strength Across Outcomes Frequentist Bayesian
Borrowing Strength Across Regions
Borrowing Strength Across Regions
Borrowing Strength Across Regions
Borrowing Strength Across Outcomes Domain Medicare expenditures Hospitalizations Outpatient ED visits Compliance with four diabetes measures Quality of care process Percentage of visits attributed to PCP 14-day follow-up to hospitalization ACSC admissions Quality of care outcome 30-day hospital readmission
Borrowing Strength Across Outcomes Domain Medicare expenditures Hospitalizations Outpatient ED visits Compliance with four diabetes measures Quality of care process Percentage of visits attributed to PCP 14-day follow-up to hospitalization ACSC admissions Quality of care outcome 30-day hospital readmission
Bayesian Hierarchical Model 𝑦 𝑖𝑗𝑡 =𝛼+ 𝑎 𝑖 + 𝑏 𝑗 + 𝑐 𝑡 + 𝛽 𝑖×𝑗 + 𝛾 𝑖×𝑡 + 𝛿 𝑗×𝑡 + 𝜀 𝑖𝑗𝑡 𝑖 indexes outcomes 𝑗 indexes geographic regions 𝑡 indexes years
Bayesian Hierarchical Model 𝑦 𝑖𝑗𝑡 =𝛼+ 𝑎 𝑖 + 𝑏 𝑗 + 𝑐 𝑡 + 𝛽 𝑖×𝑗 + 𝛾 𝑖×𝑡 + 𝛿 𝑗×𝑡 + 𝜀 𝑖𝑗𝑡 Each outcome-specific term is… centered on its domain-specific counterpart. 𝑎 𝑖 ∼𝑁 𝑑 𝑔 𝑖 , 𝜎 𝑎 2 𝑑 𝑔 𝑖 ∼𝑁 0, 𝜎 𝑑 2 𝛽 𝑖×𝑗 ∼𝑁( 𝜁 𝑔 𝑖 ×𝑗 , 𝜎 𝛽 2 ) 𝜁 𝑔 𝑖 ×𝑗 ∼𝑁(0, 𝜎 𝜁 2 ) 𝛾 𝑖×𝑡 ∼𝑁( 𝜉 𝑔 𝑖 ×𝑡 , 𝜎 𝛾 2 ) 𝜉 𝑔 𝑖 ×𝑡 ∼𝑁(0, 𝜎 𝜉 2 ) 𝜀 𝑖𝑗𝑡 ∼𝑁( 𝜆 𝑔 𝑖 ×𝑗×𝑡 , 𝑠 𝑖𝑗𝑡 2 + 𝜏 2 ) 𝜆 𝑔 𝑖 ×𝑗×𝑡 ∼𝑁(0, 𝜎 𝜆 2 )
Borrowing Strength Across Outcomes Frequentist Bayesian
Borrowing Strength Across Outcomes Frequentist Bayesian
Borrowing Strength Across Outcomes Frequentist Bayesian
Borrowing Strength Across Outcomes Frequentist Bayesian
Recall the Game Plan Problem: Impact estimates are often based on samples that are too small to produce reliable estimates Solution: Hierarchical Bayesian models borrow strength across related impact estimates Today: The Comprehensive Primary Care Initiative Borrowing strength across regions and outcomes Why – correct for multiple comparisons, decrease false and What – more precise, plausible, intuitive estimates
For More Information Lauren Vollmer Mariel Finucane lvollmer@mathematica-mpr.com Mariel Finucane mfinucane@mathematica-mpr.com