Texas Homeless Education Office The University of Texas at Austin Charles A. Dana Center 2901 N IH 35, Room Austin, Texas 78722

Performance Evaluation Training AGENDA Review performance evaluation purposes and processes Create sample outcomes and activities with criteria and measures Assess YOUR project’s outcome measures for SMART processes

Performance Evaluation Performance measurement: a process that systematically evaluates whether your efforts are making an impact/change on/to the clients you are serving or the problem you are targeting.

Why should programs be interested in performance measurement?

We are all in the business of helping people, which means we need to… understand whether current activities are working to achieve intended results.

Why should programs be interested in performance measurement? We are all in the business of helping people, which means we need to… understand whether current activities are working to achieve intended results, drive program improvement and share information on effective practices with others.

Why should programs be interested in performance measurement? We are all in the business of helping people, which means we need to… understand whether current activities are working to achieve intended results, drive program improvement and share information on effective practices with others, and acknowledge that high-performing programs are more likely to receive funding through competitive funding processes.

Creating Objectives with Measurable Outcomes Specific……….

Creating Objectives with Measurable Outcomes Specific………. Measurable….

Creating Objectives with Measurable Outcomes Specific………. Measurable…. Activity-oriented…..

Creating Objectives with Measurable Outcomes Specific………. Measurable…. Activity-oriented…. Realistic/feasible….

Creating Objectives with Measurable Outcomes Specific………. Measurable…. Activity-oriented…. Realistic/feasible… Time-oriented..

Building Blocks of Performance Measurement

 Inputs include resources dedicated to, or consumed by, the program—e.g., money, staff and staff time, volunteers and volunteer time, facilities, equipment and supplies.

Building Blocks of Performance Measurement  Inputs include resources dedicated to, or consumed by, the program—e.g., money, staff and staff time, volunteers and volunteer time, facilities, equipment and supplies.  Activities are what the program does with the inputs to fulfill its mission, such as providing school supplies, conducting a tutoring or summer program, providing staff training.

Building Blocks of Performance Measurement  Inputs include resources dedicated to, or consumed by, the program—e.g., money, staff and staff time, volunteers and volunteer time, facilities, equipment and supplies.  Activities are what the program does with the inputs to fulfill its mission, such as providing school supplies, conducting a tutoring or summer program, providing staff training.  Outputs are the direct products of program activities. They usually are presented in terms of the volume of work accomplished—e.g., number of students served and the number of staff trained.

Building Blocks of Performance Measurement  Inputs include resources dedicated to, or consumed by, the program—e.g., money, staff and staff time, volunteers and volunteer time, facilities, equipment and supplies.  Activities are what the program does with the inputs to fulfill its mission, such as providing school supplies, conducting a tutoring or summer program, providing staff training.  Outputs are the direct products of program activities. They usually are presented in terms of the volume of work accomplished—e.g., number of students served and the number of staff trained.  Outcomes are benefits or changes among clients during or after participating in program activities. Outcomes relate to measurable change in student knowledge, behavior, skills, conditions, or other attributes.

Performance Measurement Process Activities Training Tutoring Summer program Field trips School supplies Outputs # staff trained on MV # of students in tutoring # of students in summer program # of students receiving school supplies Outcomes 30% more identified 40% increase TAKS 25% incrs reading retention 50%increased attendance How do we document our efforts? What did our efforts achieve? Inputs Programs Infrastructure Staff Partners Supplies Should we adjust how we spend our resources? Should we add or change use of resources to expand our impact?

Outputs vs. Outcomes  An outcome is:  An output is:  Focused on what the student will gain/how will change  Focused on what the program will do (activity) to achieve the outcome.  A way to measure the student-level impact with clear targets and methods for measuring change.  A way to quantify the frequency and intensity of the activity.  Reasonably attributable (a result of) to an output or outputs  Specific to the activity described  Meaningful and attainable.  Feasible and attainable. Be mindful to distinguish between outputs and outcomes. If outcomes show the program works, then outputs are needed to understand how to replicate results

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students Examples Answers

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students Examples Answers Output

Group Exercise: Outcome, Output, or Neither? School supplies given to 150 students 90% of students pass the TAKS Examples Answers Output

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students  90% of students pass the TAKS Examples Answers Output Outcome

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students  90% of students pass the TAKS  75% of program staff will be MV trained Examples Answers Output Outcome

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students  90% of students pass the TAKS  75% of program staff will be MV trained Examples Answers Output Outcome Output

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students  90% of students pass the TAKS  75% of program staff will be MV trained  40% of students improve reading level one grade level Examples Answers Output Outcome Output

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students  90% of students pass the TAKS  75% of program staff will be MV trained  40% of students improve reading level one grade level Examples Answers Output Outcome Output Outcome

Group Exercise: Outcome, Output, or Neither? School supplies given to 150 students 90% of students pass the TAKS 75% of program staff will be MV trained 40% of students improve reading level one grade level Student attendance improves by 50% Examples Answers Output Outcome Output Outcome

Group Exercise: Outcome, Output, or Neither?  School supplies given to 150 students  90% of students pass the TAKS  75% of program staff will be MV trained  40% of students improve reading level one grade level  Student attendance improves by 50% Examples Answers Output Outcome Output Outcome

Example: Output or Outcome? OutputsOutcomes 100 % of students in homeless situations are enrolled immediately All campus staff are MV trained

Example: Output or Outcome? OutputsOutcomes Students in homeless situations attendance increased by 50% 100 % of students in homeless situations are enrolled immediately 100 percent of students receive school supplies and uniforms All campus staff are MV trained

Example: Output or Outcome? OutputsOutcomes 75% of students who attend after school tutoring will increase reading level by one grade level Students in homeless situations attendance increased by 50% 100 % of students in homeless situations are enrolled immediately 95 students attend after school tutoring 100 percent of students receive school supplies and uniforms All campus staff are MV trained

Can you collect the data you need in order to prove your outcome? What data will you need to collect? How will you collect it? What system will you put in place to collect the data? What criteria will tell you that you are on the right track? When do you need to begin collecting data?

Sample MV Outcomes Objective: What do you hope to accomplish? Outcome: How will you know (measure) you’ve accomplished it? What will tell you that you have been successful? Activities: What activities will you put in place to accomplish your outcome? Criteria: What criteria will you look at to see if the activities are being completed? Measure: What will be the measure (output) of each criteria/activity to tell you that you that you are on track to meet your outcome?

Sample MV Outcomes Objective: Enroll students in homeless situations immediately. Outcome: 95% of Students in homeless situations will be enrolled within 2 days on each campus. Activities: Train campus enrollment personnel. Criteria: number of trainings held, number of staff attending trainings, number of campuses sending staff to training Measure: 3 trainings held, Elem, Middle and HS 200 staff attend trainings 100% of campuses send at least 1 staff member

Sample MV Outcomes Objective: Community service providers, families and students in homeless situations in the community will know about their right to enroll immediately, even without records. Outcome: Referrals to the district from community service providers will increase by 25% Activities: Criteria: Measure:

Sample MV Outcomes Objective: Attendance for students in homeless situations will improve district wide. Outcome: Students in homeless situations will increase attendance by 50% Activities: Criteria: Measure:

Sample MV Outcomes Objective: Students in homeless situations will increase academic success. Outcome: Students in homeless situations will increase TAKS scores by 50% on both reading and Math. Other outcomes? Activities: Criteria: Measure:

More Sample MV Outcomes Objective: Outcome: Other outcomes? Activities: Criteria: Measure:

Your MV Outcomes Program Design, Evaluation & Budget Objective: Outcome: Other outcomes: Activities: Criteria: Measure:

Thanks to: Brooke Spellman & Michelle Abbenante For sharing their knowledge and expertise regarding program evaluation, and for granting their permission to use much of their presentation delivered at the NAEH Conference on Ending Family Homelessness - February 8, 2008

THEO Hotline: Barbara James: Jeanne Stamp: Tim Stahlke: Patrick Lopez: Janie Phillips: Contact Us: