1. Susan M. Wolfe, Ph.D.

2. The Programs, Logic Model and Performance Measures

3.  Funded by the Health Resources and Services Administration (HRSA)  Program purpose  Local Evaluator Role

4.  Form 7 – Number of clients by type, race and ethnicity (includes everyone)  Form 9 – Performance Measures (to be discussed)  Number of pregnant women by age / race / ethnicity / income / entry to prenatal care / adequacy prenatal care  Live singleton births by race / ethnicity / weight  Maternal deaths  Interconceptional by age / race / ethnicity / number of infants /  Male participants  Risk reduction services by prenatal, ICC, infant for each topic including risk reduction counseling and referral for assessment or treatment

5.  Progress in meeting objectives  Example: #21 – The percentage of women participating in the program who have a completed referral, among those who received a referral.  Example: #22 – The degree to which the program facilitates health providers’ screening of women participants for risk factors.

6. How I Used Them

7. MorningAfternoon  9:00 a.m.  Ice Breaker  9:30 a.m.  Logic Model – The Services and Systems  10:00 a.m.  BREAK  10:15 a.m.  Red light performance indicators  11:30 a.m.  Lunch  1:00 p.m.  Yellow light performance indicators  2:00 p.m.  Green light performance indicators  2:45 p.m.  BREAK  3:00 p.m.  Goals, Timelines, Activities, and Responsibilities  4:00 p.m.  Adjourn

8.  Two Truths and a Lie  Improve This  Tell participants: You have exactly 60 seconds to improve your seating arrangement when I tell you to begin.

9.  Logic Models  Shows them the larger picture  Where they fit into the larger picture  Performance Measures  Introduce them to program accountability  Let them see how they play a role in helping the program meet its goals and objectives

10.  Red light – the program missed the target objective  Yellow light – the program met or narrowly missed the objective and is at risk of not meeting it next year, or has struggled to meet the objective  Green light – the program met the objective and is on target to meet it in the next years as well

11.  #21 – The percentage of women participating in the program who have a completed referral, among those who received a referral.  Objective: 80.0%  188 women received a referral  131 of them completed the referral  69.7% actual

12.  The percentage of completed referrals among program participants.  Objective: 80.0%  Indicator 48.7%  Problem: how referrals were being entered into the system.

13.  The percentage of children 0 to 18 participating in the program who receive coordinated, ongoing, comprehensive care within a medical home  Objective 92% last year; 94% next year  273 children participating  254 with medical home  93% of children

14.  #20 – The percentage of women participating in the program who have an ongoing source of primary and preventive care services for women (medical home)  Objectives: 93% this year, 94% next year, 95% following year  303 women in program  290 have medical home  95.7%

15.  Red and yellow light – revisit struggles and strengths  Staff develop strategies  Set timelines for implementing each strategy  Staff decide who will take responsibility for each – accountability

16. Prepare Performance Improvement Plan

17. ActivityDeadlineResponsible #21: The percentage of women participating in the program who have a completed referral, among those who received a referral. Ensure the Database audit system for tracking referral completion follow-up is working properly 11/13/2012 CM supervisor Database developer Practice running referral tracking audit reports 11/29/2012 Case managers Case workers Staff will receive training on how to use audit reports to track referral completion at the Case Manager’s meeting from 9 a.m. to 1 p.m. 11/30/2012 CM supervisor Data entry clerk Program manager Case management staff will all run the referral audit report at least twice monthly and present results at a supervision meeting each month. Implement 12/01/2012 Case managers Case workers CM supervisor Weekly quality assurance reports will be run to ensure accuracy and completeness of data entry Implement 12/01/2012 Data entry clerk Program manager CM supervisor

19.  Made some improvements  In 2013 decided not to seek funding during the next cycle  Spent the remaining time transitioning program participants to other services

20.  Continued to work on database development and staff competency  Made additional improvements  Program manager had intermittent and frequent absences until resignation in 2013  Program applied for funding for the new cycle and received a Level 2 award

21.  Followed through and completed all tasks on the performance improvement plan  Applied for funding for the next cycle and received it at Level III as a mentoring site  Scheduling a similar retreat for December this year to kick off the new grant

22. Susan M. Wolfe, Ph.D. CEO, Susan Wolfe and Associates, LLC Susan.Wolfe@susanwolfeandassociates.net