1. 1 Community HealthChoices Overview of Evaluation Design MLTSS Sub-MAAC February 3, 2016 Howard B. Degenholtz, Ph.D. Principal Investigator Medicaid Research Center

2. Outline 2  Purpose of the Evaluation  Overview of the Evaluation Design

3. Overview 3  The University of Pittsburgh will conduct a state-wide, 7 year evaluation of the implementation, process and outcomes of CHC  Provide independent, scientifically rigorous evidence of program impact with respect to:  Opportunities for community-based living  Service coordination  Quality and accountability  Program innovation  Efficiency and effectiveness  Multiple Data Sources and Methods  Key Informant Interviews  Focus Groups  Participant and Caregiver Experience Interviews  Administrative Data  Role of Department of Human Services  Collaborated on design  Provides funding and oversight through Evaluation Work Group  Role of MLTSS Sub-MAAC  We will provide regular updates to Sub-MAAC  Provide feedback & suggest course changes  Sub-MAAC representation on Work Group (Oversight)

4. Research Questions 4  Goal 1: Enhance Opportunities for Community Living  Does CHC increase use of HCBS?  Does CHC prevent or delay institutionalization?  Does CHC facilitate return to the community (among long-stay and short stay nursing home residents)?  Goal 2: Improve Service Coordination  Does CHC improve coordination of medical care for people with complex medical needs and disabilities?  Does CHC improve coordination between the medical care system and LTSS providers?  Does CHC improve coordination between Medicaid and Medicare?  Goal 3: Enhance Quality and Accountability  Does CHC improve the quality of life and well-being of participants and family caregivers?  Does CHC improve quality of care of acute and ambulatory care?  Does CHC improve quality of care across the spectrum of LTSS?  Goal 4: Advance Program Innovation  Does CHC lead to new models of care delivery, new approaches to care coordination, innovative use of technology, or innovations in providing housing or access to employment?  Goal 5: Increase Efficiency and Effectiveness  Does CHC control the cost of care for participatns? (both physical health and LTSS)  Does CHC improve access to preventive care and reduce unnecessary medical care?  Does CHC increase the use of LTSS?

5. Multiple Data Sources Provide Multiple Perspectives on Program Performance 5 Key Informant Interviews and Focus Groups (Purposive Samples) Participant and Caregiver Interviews (Representative Sample) Analysis of Administrative Data (Entire Population)

6. Summary of Data Sources for Each Population 6 AgeLTSS Use Key Informant Interviews Focus Groups Administrative Data Participant Interviews Caregiver Interviews 21-59Community ✔✔✔✔✔ Facility ✔✔✔ * ✔ 60+Community ✔✔✔✔✔ Facility ✔✔✔ * ✔ None (Duals) ✔✔✔✔✔ *Evaluation plan is being revised to add participant interviews in facilities (2-26-2016)

7. Key Informant Interviews 7  Goal  Monitor implementation from multiple perspectives  Provide early, independent, ongoing insight (e.g., spring 2017)  Methods  Semi-structured, open-ended interviews  Qualitative analysis  Conducted on a rolling basis  before, during and after implementation in each region  Informants:  Advocacy Groups  Participants  Age 21-59 HCBS User  Age 60 + HCBS User  Dual Eligible, no-HCBS  Caregivers  Age 21-59 HCBS and Facility  Age 60+ HCBS and Facility  Providers:  Personal Care/AL  Nursing Home  Centers for Independent Living  Home Health  Personal Assistance  Adult Daily Living  Hospice  Meals  Transportation  Home Modification  Habilitation  Respite  Service Coordinator  Primary Care Physician  Hospital  LIFE  Government  State Officials  County Officials  Area Agency on Aging  Ombudsperson

8. Participant and Caregiver Focus Groups 8  Goal  Gather early impressions and feedback from participants and caregivers during rollout (in each Phase)  Group settings elicit different responses than individual interviews  Methods  Professional focus group moderator will lead structured conversation  Thematic analysis  Conducted early in the implementation year in each phase  Sample  Represent major categories:  Urban  Rural/Adjacent  Participants  Caregivers

9. Participant and Caregiver Experience Interviews 9  Goal  Measure quality of life and satisfaction  Methods  Structured, closed and open-ended interviews  Prior to enrollment, 1 st and 2 nd year of enrollment  In-person with participant, phone with proxy and caregivers  Sample  Age 21-59 Community LTSS users  Age 60+ Community LTSS users  Age 60+ non-LTSS users (duals)  Caregivers (unpaid) for each subgroup

10. Study Design – PCE Interviews 10 Month Type of AnalysisRegion122436 Before and After w/ Comparison Groups Phase I Phase II ObservationalPhase I Phase II Phase III  In 2017 (12m), we will compare participants living in the Phase I region to people in Phase II & III  During 2018 (24m), we will compare Phase I & II to Phase III  In 2019 (36m), the program will be statewide, so we will measure outcomes, but there is no comparison group.

11. Administrative Data Analysis 11  Goal  Effect of CHC on use of HCBS, institutionalization, acute care, and cost  Methods  Medicaid & Medicare Claims  Nursing Home Minimum Data Set (MDS) 3.0  Level of Care & Service Plan  Managed Care Organization Performance Metrics  Analysis  Difference-in-difference models compare trend in Phase I to trend in Phase II and Phase III groups  Propensity score models adjust for unobserved differences between participants in each region  Data lag by 6-8 months  E.g., data for Year 1, Phase I, will be available in late 2018  Analysis of two year’s of data for all 3 phases in 2022 201720182019202020212022 Phase IStartYear 1Year 2Year 3 Phase IIStartYear 1Year 2Year 3 Phase IIIStartYear 1Year 2(Year 3)

12. Study Design – Administrative Data 12 Phase I Phase II Phase III Comparison Groups Program Groups Notes: 2015-2016 data will be used for pre-post comparisons Phase II, III data from 2017 will be used as contemporaneous comparison for Phase I Phase III data from 2018 will be used as contemporaneous comparison for Phase I, II 20162017 20182019 202020152021 Baseline Data

13. Challenges 13  Key Informant Interviews  Participation and cooperation of stakeholders  Participant and Caregiver Interviews  Recruitment and retention of sample  Are Phase II and Phase III regions good comparison groups?  Administrative Data  Data are complex  There are changes to data systems taking place during the study  New LCD tool  Change to ICD10  Concerns over the quality of the data  Are Phase II and Phase III regions good comparison groups?

14. Summary 14  Evaluation is designed to provide rigorous, independent analysis of the effects of CHC on multiple outcomes for multiple populations  Rigorous:  Study design takes advantage of phased implementation to construct comparison groups and estimate causal effects, for example “Did the CHC program cause an increase in the number of people receiving LTSS in a community setting?”  Multiple Perspectives:  Wide range of providers types and advocacy groups  Participants in different living arrangements, health conditions, urban/rural settings  Multiple Methods:  Participants and providers interviews, focus groups and administrative data provide multiple perspectives on the big picture  Short and Longer-Term:  Early insights are important for planning: “What’s happening?”  Inform course correction for 2018, 2019 Phases  Longer-term outcomes important to answer the question: “Does it work?”

15. Study Team  Department of Health Policy and Management  Howard B. Degenholtz, PhD, Principal Investigator  Marian Jarlenski, PhD  Damian DaCosta  Lexi Drozd  Ray VanCleve  Meredith Hughes  Health Policy Institute - Medicaid Research Center  Evan Cole, PhD  Phil Rocco, PhD  Aiju Men  Qualitative, Evaluation and Stakeholder Engagement Center  Susan Zickmund, PhD  Megan Hamm  Office of Health Survey Research  Todd Bear  Health Services Research Data Center  Jeremy Kahn, MD  Dan Ricketts  Consultants  Richard Morycz, MD (Abuse and Safety Concerns)  Julie Donohue, PhD (Pharmacy and Mental Health)  Walid Gellad, MD (Pharmacy)  Richard Schulz, PhD (Caregiving) 15

16. Contact Information 16 Howard B. Degenholtz, PhD, Principal Investigator Department of Health Policy and Management Graduate School of Public Health Center for Bioethics and Health Law Health Policy Institute Medicaid Research Center University of Pittsburgh 130 DeSoto St., A748 Pittsburgh, PA 15261 (412) 624-6870 degen@pitt.edu

17. Additional Slides 17

18. Goal 1: Enhance Opportunities for Community Living 18 Study AimsPrimary Research Questions (Directional Hypotheses or Descriptive) To study the effect of CHC on the use of HCBS. HCBS use will increase among CHC participants, relative to comparable individuals in non-participating areas. To study the effect of CHC preventing or delaying institutionalization. CHC participants will have lower rates of institutionalization, relative to comparable individuals in non- participating areas. To study the effect of CHC on facilitating return to the community. CHC participants will be more likely to return to the community after a hospitalization or facility based post- acute care, relative to comparable individuals in non-participating areas. CHC participants who are long-stay residents will be more likely to return to the community, relative to comparable individuals in non- participating areas.

19. Goal 2: Improve Service Coordination 19 Study AimsPrimary Research Questions (Directional Hypotheses or Descriptive) To describe coordination among different types of care. To what extent does CHC facilitate improved care coordination between acute, ambulatory, behavioral and LTSS providers? To describe integration of care between Medicare and Medicaid. To what extent does CHC lead to improved care coordination for dual eligibles without LTSS needs?

20. Goal 3: Enhance Quality and Accountability 20 Study AimsPrimary Research Questions (Directional Hypotheses or Descriptive) To study the effect of CHC on quality of life and well-being for participants and family caregivers. CHC participants will have higher quality of life and well-being, relative to comparable individuals in non- participating areas. Informal caregivers of CHC participants will have higher quality of life and well- being, relative to comparable individuals in non-participating areas. To describe quality of care across the spectrum of acute and LTSS providers. What is the association between CHC and quality of care across the spectrum of acute and LTSS providers?

21. Goal 4: Advance Program Innovation 21 Study AimsPrimary Research Questions (Directional Hypotheses or Descriptive) To describe the model of care used by physical health providers. To what extent does CHC lead to incorporation of innovations such as person-centered care goals into primary care? What proportion of participants receive physical health care from a multidisciplinary team? To describe models for care coordination. Is CHC leading to new models of care coordination? (e.g., that span chronic and LTSS needs) To describe changes in LTSS providers and service provision. Is CHC leading to new types of LTSS providers or new combinations of housing and LTSS services? To describe changes in use of technology. Is CHC leading to increase use of technology among LTSS providers? (e.g., telehealth, electronic medical records, visit verification) To describe the impact of CHC on employment opportunities. Is CHC leading to new forms of employment for participants? Are there new types of community supports for employment? To describe the impact of CHC on the type of housing. Is CHC leading to new combinations of housing and services? Is CHC expanding the opportunities for participants to remain in the community?

22. Goal 5: Increase Efficiency and Effectiveness 22 Study AimsPrimary Research Questions (Directional Hypotheses or Descriptive) To study the effect of CHC on cost of care. Monthly and annual cost of care for CHC participants will be the same or lower than comparable individuals in non-participating areas. To study the effect of CHC on utilization patterns. Aggregate care utilization measures for CHC participants will be the same or lower than comparable individuals in non-participating areas. HCBS use will be higher, and hospitalizations lower, among CHC participants relative to comparable individuals in non-participating areas.

23. Study Design – PCE Interviews 23 Phase I Phase II Phase III Comparison Groups Program Period Notes: Baseline interviews conducted in late 2016 Follow-up interviews in spring and fall New samples for Phase II and Phase III will be recruited in 2017 and 2018 Individuals will be interviewed for 3 years 20162017 20182019 20202021 Baseline Data