Evaluation Overview: Rochelle Schultz Spinarski, Rural Health Solutions Community Care Learning Collaborative October 29, 2014

Definition: Program Evaluation A systematic method for collecting, analyzing, and using information to answer questions about projects, policies and programs, particularly about their effectiveness and efficiency. Engage Stakeholders Describe the Program Focus the Evaluation Gather Credible Evidence Justify Conclusions Ensure Use and Share

Why Program Evaluations?  Demonstrate and/or improve program Effectiveness  Better manage limited resources  Document and share program accomplishments  Make program improvements  Increase stakeholder engagement and understanding  Increase or maintain a sense of satisfaction – both internal and external  Program sustainability and/or development/expansion  Fulfill mission and work towards vision  Document and share to support or foster program implementation

Outcomes: Changes in someone or something (other than the program or its staff) Hierarchy of Effects (Bennett and Rockwell, 1995) Health Outcomes Changes in social condition Changes in Environmental Condition Changes in economic conditions Changes in health indicators System & Environment Change Learning KnowledgeOpinionsSkills obtainedAspirations Actions Changes in pattern of behavior Reactions Degree of interest Feelings towards the program Acceptance of methods Participation # of people reached Characteristics of people Intensity of contact

Community Care Coordination  Continued development and implementation of a statewide regionally based network to provide an integrated approach to health care delivery through care coordination that supports primary care providers and links patients with community resources necessary to empower patients in addressing biomedical and social determinants of health to improve health outcomes. The Safety Net Network has historically used the following definition, which was developed by the National Academy of State Health Policy: “Community health teams, also known as community networks, pods, or hubs, are locally based care coordination teams comprising multidisciplinary staff from varied disciplines, such as nursing, behavioral health, pharmacy, and social work. In partnership with primary care practices, teams connect patients, caregivers, providers, and systems through care coordination, collaborative work, and direct patient engagement.” IA Code , HF 2463, FY15 HHS Appropriations Act, Div. III, Sect. 4.

ActivityStaffGranteesNotesSeptOctNovDecJanFebMarchAprilMayJuneJuly Monthly Conference Call IPCA RHS Webster Scheduled in advance for a date after the 15 th NA Cerro GordoNA SeasonsNA Jennie E.NA Monthly Reporting  Narrative  Data IPCA Webster Cerro Gordo Seasons Jennie E. NA 15 th Final ReportIPCANA 30th Patient SurveysUof IA Contact information due to UofIA 3/15 Maintain patient names, contact information, and patient’s perceived health status.15th NA Community Partner SurveyRHS Updated contact info due by all grantees 3/15 NA 15 th NA Health care provider Survey RHSNA 15 th NA Steering Committee Interviews RHSSeasonsNA 15 th NA Jennie E.NA 15 th NA On-site Stakeholder Interviews RHS Jennie Edmonson Seasons 3 – Jennie Edmonson 3 - Seasons NA Quarterly ReportingRHSDallas County Allen Memorial 15 th

Lessons Learned  Community partners  Engagement in project planning, development, implementation  Engagement telling people  Takes time, lots of it  Build trust  One of the biggest factors affecting sustainability  Initiative updates  Healthcare providers  Initiative updates  Patient follow-up information  Some want to be engaged, most not  Patients  Follow-through and timeliness are critical  They want to be engaged and heard  Communications  One time vs 13 times  One method vs 5 methods  F2F is important  Document decisions/changes made and share with all Use the evaluation findings to support your program’s development. Use the data you are collecting for your own program development purposes Consider building program evaluation activities into your program planning and development process : Brief service reports Case studies 1. Steering committee meetings 2. Interactions with project partners 3. Interactions with healthcare providers

Reminders for the Year We will be looking for many things, including: Health outcomes data – changes in HEALTH STATUS – CRITICAL Care coordination that supports healthcare providers in empowering patients Monthly reports due on the 15 th of each month* Patient contact information will be due on April 15* Patient perceived health status data will be due on March 15* * Indicates for implementation grantees only.

Thank You! Questions? Rochelle Schulz Spinarski Rural Health Solutions