1. PROGRAM EVALUATION Mark Spolidoro, University of Massachusetts Boston

2. What is Evaluation?  Program evaluation is the systematic assessment of the processes and outcomes of a program with the intent of furthering its development and improvement. *  It involves collecting and analyzing information about a program’s:  Features  Activities  Outcomes * (Office of Educational Assessment, University of Washington)

3. Importance of Evaluation  Improve your program’s:  Design  Implementation  Demonstrate your program’s impact  Leverage findings:  To garner community support  To attract potential funders  To attract new families to your program

4. Sources of Information  Everyone involved in the program including:  Families  Staff Coaches Counselors  Children

5. Sources of Information Family StaffChild Child Impact

6. Tools of Evaluation  Program surveys  Paper and pencil  Online  Observations  Interviews  In-person  Phone  Logs

7. Tools of Evaluation Tool:Administered to: Program SurveysFamilies Staff Children In-person InterviewsStaff Children Phone InterviewsFamilies ObservationsChildren LogsStaff

8. Examples of Evaluation Objective: What are parents’ reasons for sending their child to camp? Camp Shriver: Below is a list of goals that family members have for their child at Camp Shriver. Please rank the three things that are most important to you. improve physical fitness have opportunities to be proud of himself/herself make new friends improve sports skills / athletic ability improve social skills improve self-confidence have fun feel like he/she belongs Other: _____________________________

9. Examples of Evaluation Objective: Did the program work? Camp Shelanu How did we do? Please rate the following aspects of camp PoorFairGoodGreat! Don't know or N/A Registration process Schedule/structure of camp Communication with families Organization Customer Service Counselors Camp Director Inclusion Staff Safety Facility Activites Field trips Your camper's enjoyment

10. Examples of Evaluation Objective: Is inclusion taking place? Camp Crossroads: 1. How do you feel that the overall spirit of inclusion is working this summer? o I don’t know what inclusion is o It’s going okay o It’s going well but it doesn’t really apply to me o It’s going great! 2. If you were a counselor last summer, have you noticed any changes in inclusion? ________________________ 3. What are the biggest difficulties or challenges that you face with inclusion? ________________________

11. Examples of Evaluation Objective: Did the program improve the child’s social skills? Camp Shriver  Use the number scale below to show how often each child engaged in each of the following statements during the past week. Just do your best to answer each item, based on how well you know each camper. 0 = never1= rarely 2 = sometimes 3 = often DK= don’t know Camper’s Name Interacts well with other campers Can calm down when excited Makes new friends easily Easily changes from one activity to another Disobeys rules or requests Offers help to other campers Has difficulty when waiting for turn (ex. In the lunch line) Encourages and cheers on peers Invites other campers to play with them

12. Examples of Evaluation Objective: Did the program improve the child’s swimming skills? Camp Shriver Camper’s Swimming Level Not getting in Getting in and holding on without life jacket Getting in and holding on with life jacket Getting in and letting go without life jacket Getting in and letting go with life jacketWalking without life jacket Walking with life jacketTaking feet off the ground without life jacket Taking feet off the ground with life jacket Learning to stroke, move forward without life jacket Learning to stroke, move forward with life jacket Traveling, swimming without life jacket Traveling, swimming with life jacketPlaying without life jacket Playing with life jacketTaking the swim test Taking off life jacketJumping off the diving board

13. Examples of Evaluation Objective: What did the child like about camp? Camp Shriver I didn’t like it at all I liked it a little I liked it a lot It was the best thing ever! a. Gym b. Pool c. Field d. Creative room e. The food at camp f. Free time after lunch g. The kids on your team h. The kids at camp i. The counselors on your team

14. What makes a good evaluation?  It is tailored to your program  It takes into account many different perspectives  The methods are as rigorous as possible

15. Strategies in Developing an Evaluation  Think about the most essential questions: What do you want to know about your program?  Use research and program evaluation resources of the community  Determine what is feasible  Network with other programs to find out what works for them  Get buy-in from stakeholders