1. Community HealthChoices Overview of Evaluation Design MLTSS Sub-MAAC February 3, 2016 Howard B. Degenholtz, Ph.D. Principal Investigator Medicaid Research Center

2. Outline
Purpose of the Evaluation
Overview of the Evaluation Design

3. Overview
The University of Pittsburgh will conduct a state-wide, 7 year evaluation of the implementation, process and outcomes of CHC
Provide independent, scientifically rigorous evidence of program impact with respect to:
Opportunities for community-based living
Service coordination
Quality and accountability
Program innovation
Efficiency and effectiveness
Multiple Data Sources and Methods
Key Informant Interviews
Focus Groups
Participant and Caregiver Experience Interviews
Administrative Data
Role of Department of Human Services
Collaborated on design
Provides funding and oversight through Evaluation Work Group
Role of MLTSS Sub-MAAC
We will provide regular updates to Sub-MAAC
Provide feedback & suggest course changes
Sub-MAAC representation on Work Group (Oversight)

4. Research Questions Goal 1: Enhance Opportunities for Community Living
Increase HCBS, delay or prevent institutionalization
Goal 2: Improve Service Coordination
Improve coordination of medical care and HCBS, and between Medicaid and Medicare
Goal 3: Enhance Quality and Accountability
Impact on quality of life and well-being for participants and caregivers
Impact on quality of care across spectrum of acute and LTSS
Goal 4: Advance Program Innovation
Models of care, care coordination, technology, housing and employment
Goal 5: Increase Efficiency and Effectiveness
Effect on cost and utilization of acute and LTSS

5. Multiple Data Sources Provide Multiple Perspectives on Program Performance
Key Informant Interviews and Focus Groups
(Purposive Samples)
Participant and Caregiver Interviews (Representative Sample)
Analysis of Administrative Data
(Entire Population)

6. Summary of Data Sources for Each Population
Age
LTSS Use
Key Informant Interviews
Focus Groups
Administrative
Data
Participant Interviews
Caregiver Interviews
21-59
Community ✔︎ ✔
Facility
60+
None (Duals)

7. Key Informant Interviews
Goal
Monitor implementation from multiple perspectives
Provide early, independent, ongoing insight (e.g., spring 2017)
Methods
Semi-structured, open-ended interviews
Qualitative analysis
Conducted on a rolling basis
before, during and after implementation in each region
Informants:
Advocacy Groups
Participants
Age HCBS User
Age 60 + HCBS User
Dual Eligible, no-HCBS
Caregivers
Age HCBS and Facility
Age 60+ HCBS and Facility
Providers:
Personal Care/AL
Nursing Home
Centers for Independent Living
Home Health
Personal Assistance
Adult Daily Living
Hospice
Meals
Transportation
Home Modification
Habilitation
Respite
Service Coordinator
Primary Care Physician
Hospital
Government
State Officials
County Officials
Area Agency on Aging
Ombudsperson

8. Participant and Caregiver Focus Groups
Goal
Gather early impressions and feedback from participants and caregivers during rollout (in each Phase)
Group settings elicit different responses than individual interviews
Methods
Professional focus group moderator will lead structured conversation
Thematic analysis
Conducted early in the implementation year in each phase
Sample
Represent major categories:
Urban
Rural/Adjacent
Participants
Caregivers

9. Participant and Caregiver Experience Interviews
Goal
Measure quality of life and satisfaction
Methods
Structured, closed and open-ended interviews
Prior to enrollment, 1st and 2nd year of enrollment
In-person with participant, phone with proxy and caregivers
Sample
Age Community LTSS users
Age 60+ Community LTSS users
Age 60+ non-LTSS users (duals)
Caregivers (unpaid) for each subgroup

10. Study Design – PCE Interviews
In 2017 (12m), we will compare participants living in the Phase I region to people in Phase II & III
During 2018 (24m), we will compare Phase I & II to Phase III
In 2019 (36m), the program will be statewide, so we will measure outcomes, but there is no comparison group.
Month
Type of Analysis
Region 12 24 36
Before and After w/ Comparison Groups
Phase I 
Phase II
Observational
Phase III

11. Administrative Data Analysis
Goal
Effect of CHC on use of HCBS, institutionalization, acute care, and cost
Methods
Medicaid & Medicare Claims
Nursing Home Minimum Data Set (MDS) 3.0
Level of Care & Service Plan
Managed Care Organization Performance Metrics
Analysis
Difference-in-difference models compare trend in Phase I to trend in Phase II and Phase III groups
Propensity score models adjust for unobserved differences between participants in each region
Data lag by 6-8 months
E.g., data for Year 1, Phase I, will be available in late 2018
Analysis of two year’s of data for all 3 phases in 2022
2017
2018
2019
2020
2021
2022
Phase I
Start
Year 1
Year 2
Year 3
Phase II
Phase III
(Year 3)

12. Study Design – Administrative Data
2016
2017
2018
2019
2020
2015
2021
Phase I
Phase II
Phase III
Baseline
Data
Comparison
Groups
Notes:
data will be used for pre-post comparisons
Phase II, III data from 2017 will be used as contemporaneous comparison for Phase I
Phase III data from 2018 will be used as contemporaneous comparison for Phase I, II
Program
Groups

13. Challenges Key Informant Interviews
Participation and cooperation of stakeholders
Participant and Caregiver Interviews
Recruitment and retention of sample
Are Phase II and Phase III regions good comparison groups?
Administrative Data
Complex data
Changes to data systems
Quality concerns

14. Summary
Evaluation is designed to provide rigorous, independent analysis of the effects of CHC on multiple outcomes for multiple populations
Rigorous:
Study design takes advantage of phased implementation to construct comparison groups and estimate causal effects
Multiple Perspectives:
Wide range of providers types and advocacy groups
Participants in different living arrangements, health conditions, urban/rural settings
Multiple Methods:
Participants and providers interviews, focus groups and administrative data provide multiple perspectives on the big picture
Short and Longer-Term:
Early insights are important for planning: “What’s happening?”
Inform course correction for 2018, 2019 Phases
Longer-term outcomes important to answer the question: “Does it work?”

15. Study Team Department of Health Policy and Management
Howard B. Degenholtz, PhD, Principal Investigator
Marian Jarlenski, PhD
Damian DaCosta
Lexi Drozd
Ray VanCleve
Meredith Hughes
Health Policy Institute - Medicaid Research Center
Evan Cole, PhD
Phil Rocco, PhD
Aiju Men
Qualitative, Evaluation and Stakeholder Engagement Center
Susan Zickmund, PhD
Megan Hamm
Office of Health Survey Research
Todd Bear
Health Services Research Data Center
Jeremy Kahn, MD
Dan Ricketts
Consultants
Richard Morycz, MD (Abuse and Safety Concerns)
Julie Donohue, PhD (Pharmacy and Mental Health)
Walid Gellad, MD (Pharmacy)
Richard Schulz, PhD (Caregiving)

16. Contact Information
Howard B. Degenholtz, PhD, Principal Investigator Department of Health Policy and Management Graduate School of Public Health Center for Bioethics and Health Law Health Policy Institute Medicaid Research Center University of Pittsburgh 130 DeSoto St., A748 Pittsburgh, PA (412)

17. Additional Slides

18. Goal 1: Enhance Opportunities for Community Living
Study Aims
Primary Research Questions (Directional Hypotheses or Descriptive)
To study the effect of CHC on the use of HCBS.
HCBS use will increase among CHC participants, relative to comparable individuals in non-participating areas.
To study the effect of CHC preventing or delaying institutionalization.
CHC participants will have lower rates of institutionalization, relative to comparable individuals in non-participating areas.
To study the effect of CHC on facilitating return to the community.
CHC participants will be more likely to return to the community after a hospitalization or facility based post-acute care, relative to comparable individuals in non-participating areas.
CHC participants who are long-stay residents will be more likely to return to the community, relative to comparable individuals in non-participating areas.

19. Goal 2: Improve Service Coordination
Study Aims
Primary Research Questions (Directional Hypotheses or Descriptive)
To describe coordination among different types of care.
To what extent does CHC facilitate improved care coordination between acute, ambulatory, behavioral and LTSS providers?
To describe integration of care between Medicare and Medicaid.
To what extent does CHC lead to improved care coordination for dual eligibles without LTSS needs?

20. Goal 3: Enhance Quality and Accountability
Study Aims
Primary Research Questions (Directional Hypotheses or Descriptive)
To study the effect of CHC on quality of life and well-being for participants and family caregivers.
CHC participants will have higher quality of life and well-being, relative to comparable individuals in non-participating areas.
Informal caregivers of CHC participants will have higher quality of life and well-being, relative to comparable individuals in non-participating areas.
To describe quality of care across the spectrum of acute and LTSS providers.
What is the association between CHC and quality of care across the spectrum of acute and LTSS providers?

21. Goal 4: Advance Program Innovation
Study Aims
Primary Research Questions (Directional Hypotheses or Descriptive)
To describe the model of care used by physical health providers.
To what extent does CHC lead to incorporation of innovations such as person-centered care goals into primary care?
What proportion of participants receive physical health care from a multidisciplinary team?
To describe models for care coordination.
Is CHC leading to new models of care coordination? (e.g., that span chronic and LTSS needs)
To describe changes in LTSS providers and service provision.
Is CHC leading to new types of LTSS providers or new combinations of housing and LTSS services?
To describe changes in use of technology.
Is CHC leading to increase use of technology among LTSS providers? (e.g., telehealth, electronic medical records, visit verification)
To describe the impact of CHC on employment opportunities.
Is CHC leading to new forms of employment for participants? Are there new types of community supports for employment?
To describe the impact of CHC on the type of housing.
Is CHC leading to new combinations of housing and services? Is CHC expanding the opportunities for participants to remain in the community?

22. Goal 5: Increase Efficiency and Effectiveness
Study Aims
Primary Research Questions (Directional Hypotheses or Descriptive)
To study the effect of CHC on cost of care.
Monthly and annual cost of care for CHC participants will be the same or lower than comparable individuals in non-participating areas.
To study the effect of CHC on utilization patterns.
Aggregate care utilization measures for CHC participants will be the same or lower than comparable individuals in non-participating areas.
HCBS use will be higher, and hospitalizations lower, among CHC participants relative to comparable individuals in non-participating areas.

23. Study Design – PCE Interviews
2016
2017
2018
2019
2020
2021
Phase I
Phase II
Phase III
Baseline
Data
Notes:
Baseline interviews conducted in late 2016
Follow-up interviews in spring and fall
New samples for Phase II and Phase III will be recruited in 2017 and 2018
Individuals will be interviewed for 3 years
Comparison
Groups
Program
Period