1. Evaluation
Goal: Ensure learnings from the program are identified and recorded, in particular:
What roles can CHCs best play in addressing SDOH?
What types of projects were easiest to implement and why?
What challenges did you face in implementing your projects and how did you address them?
What ROOTS program activities were most helpful?
Did participation in this program increase your clinic’s capacity to address patients’ SDOH?
Hi all,
I thought it might be helpful to provide a recap of our evaluation goals and design. As a reminder, the overall goal of the evaluation is really to ensure that what you learn during participation in this program gets recorded and informs future efforts in this area.

2. Evaluation Baseline project lead interviews: Nov-Dec 2017 
Baseline staff (and patient) surveys: Dec 2017 – Jan 2018 – in process
Mid course in-person interviews: Apr-May 2018
Follow-up staff (and patient) surveys: Sep-Oct 2018
Follow-up project lead interviews: Sep-Oct 2018
Collection of site-specific process and outcome metrics: Sept-Oct 2018
Throughout: Surveys to gauge satisfaction with program sessions, and one-on-one evaluation TA.
We are going to answer these questions through several activities laid out on this slide:
First baseline project lead interviews, in Oct and November, to better understand your project goals, the challenges you anticipate having to overcome, the TA you think you will need and to discuss how you plan to evaluate your project.
Second a survey of providers and if possibly patients at each of your sites to gauge current provider capacity to address SDH and patients attitudes about integration of social and medical services.
Then we do in person on site interviews with project teams, patients and partners in Feb-Apr to better understand how the projects are going.
Then we’ll do follow up interviews and surveys after the program ends.
And we’ll collect site-specific process and outcome metrics to better understand the impact of the projects.
Throughout we’ll be doing evaluation of program sessions and also provide one on one TA about evaluation of your projects to

3. Process Outcomes Impact Outcomes
(Measuring activities)
(Measuring impacts of activities)
Patient
Screening and referral
Number screened
Number with needs
Number referred
Number who followed through on referrals
Numbers receiving services
Etc.
Social needs
Global (risk scores)
Need specific, eg. food insecurity, housing insecurity, etc.
Health Behaviors and Health Status
Global measures, eg. self-rated health, health-related quality of life (eg. VR-12, SF-12), etc.
Specific measures, eg. HbA1c and BP control, BMI, mental health, healthy eating, drug use, etc.
Health Care Utilization
Primary care visits
Apt. adherence
Specialist visits
ED visits
Hospitalizations
Health Care Costs
Total cost of care
Costs associated with specific services, eg. primary care, or ED use
Experience of Care
Satisfaction with care
Trust in provider
Loyalty to clinic
Health Care Quality
Provider
Staff/
Knowledge, capacity and confidence to address patients’ SDH
Job satisfaction
Burnout
Turnover
Leadership support
Financial sustainability and ROI
System efficiency
Clinic
Workflow systems capacity
EHR systems capacity
Strength of community partnerships
Data integration with community partners
Staff training for SDH work; Recruitment of specialized staff
Process Outcomes Impact Outcomes
(Measuring activities) (Measuring impacts of activities)
This table is provides a framework for thinking about outcomes of social needs interventions in clinical settings. There are two types of outcomes: process outcomes and impact outcomes. Process outcomes measure the activities carried out, and impact outcomes measure the impacts of those activities. Process outcomes will really depend on the program activities. Here are examples for a screening and referral program. You would want to tailor this to the activities you are doing. Impact outcomes are further categorized in three buckets: patient-level outcomes, staff and provider level outcomes, and clinic level outcomes. For the impact outcomes at the patient level, 5 types of outcomes are highlighted: social needs outcomes, health behaviors and health status outcomes, health care utilization, health care costs, and experience of care. For each of these the table provides a few examples. The hardest outcomes often are the health behaviors and health status outcomes. For these it can be useful to think about global measures, such as self-rated health or quality of life, and specific measures that relate to a specific aspect of health. Often overlooked impact outcomes include patient experience of care and provider outcomes, including their experience of care, job satisfaction, burnout and turnover. In general when thinking about evaluation, you want to make sure you are capturing both process outcomes so you know what you did, and impact outcomes, so you can guage the impact. Which impact outcomes you chose should depend on the goals of the program and what matters most to you and others involved in the project. The other key thing to think about is timing. Experience of care is often one of the easiest thing to measure and one of the quickest things to change. Social needs impacts will likely be the next things to change, then the other impacts. It may take a while and also a large change in social needs to see health or health utilization impacts. One more note, this is not meant to be completely comprehensive, there are likely to be other clinic level process outcomes that may apply for your project as well as clinic impact outcomes. Same with the staff and provider level. But this hopefully helps you think about the categories of outcomes that you can consider for your projects.

4. Outcomes measurement considerations
Tailor!
Timing and dose
Some measures of utilization and costs may increase, eg. primary care vs. ED.
Tailor – You will want to tailor all of your measures to your intervention, Eg. For West County, no. screened doesn’t make sense, would look on the process side more at system level measures.
Timing and dose – you obviously want to think about what impacts you think you can achieve given the dose of intervention and the follow up time you have. Social needs data and patient experience of care could be short term impact measures, while health outcomes and health care data might be longer term data.
Finally, when considering health care measures, and ROI it’s important to realize that some utilization measures may increase.