1. NICK BAYER LYZA INGRAHAM BRANDON W. YOUKER GRAND VALLEY STATE UNIVERSITY SCHOOL OF SOCIAL WORK Using Service Learning In the Teaching of and Research on Program Evaluation

2. Introduction Graduate social work course: Program Planning, Monitoring, and Evaluation Students evaluated Indian Trails Camp using Goal-Free Evaluation or Success Case Method Programs Evaluated:  Respite Care  Skill Building  Community Living Supports

3. Evaluation Models: GFE & SCM Goal-Free Evaluation  Evaluator is screened from goal- & objective-related information  Evaluator observes & measures all actual outcomes in judging program Success Case Method  Evaluator works with stakeholders to identify “success cases”  Evaluator collects factual in depth data from multiple sources on the conditions “causing” success

4. The Course Requirements In groups, student-evaluators…  Design, conduct, & report on an evaluation  Produce a/an:  Literature review  Evaluation proposal  Evaluation report  Oral presentation

5. Research Methodology Students completed a 20-item semantic differential regarding their attitudes toward the service learning project Program staff completed 25-item semantic differential regarding their attitudes toward the utility of the evaluation reports The content of the evaluation reports were analyzed & compared

7. Findings: Student Responses “I felt that I was helping the directors and program planners of the camp to improve their services” “It definitely helped the learning process to be performing an actual evaluation instead of a hypothetical one” “Learning evaluation from a textbook is almost nothing like actually doing” “Overall it was a great experience. I was happy to have my work count for something more than a grade.”

9. Findings: Camp Staff Responses Respite SCM: staff were highly critical; called it “biased,” “unclear,” & “not balanced” Respite GFE: staff were mixed; some called it “well written” & “complete” & wrote “I agreed with most statements made” while others criticized the small sample size of interviewees Skill Bldg SCM: staff mostly approved; called it “useful,” “helpful,” & they liked the quotes and recommendations Skill Bldg GFE: staff were mixed; called it “enlightening” while others criticized it for being “unrealistic,” not taking “into account how disabilities affect clients’ ability to perform,” & criticized the small sample size

10. Findings: Content Analysis of Reports ProgramRespite Skill Building CLS Evaluation Model GFESCMGFESCMGFE Methods Semi- structured interviews; direct observation Semi-structured questionnaire; group interview; face-to-face interviews Direct observation; phone & face-to-face interviews; review of consumer records; self-administered surveys Semi-structured interviews Direct observation; semi-structured interviews; document review; self-esteem instrument Sample 4 staff; 4 caregivers 3 staff; 2 consumers 8 consumers; 1 caregiver; 1 community member 4 staff; 3 caregivers 3 staff; program director Criteria Safe/secure; effective; ethical; accessible Personal care & supervision to campers; recreational; camper- staff relationships Safe; accessible; educational; guardian satisfaction; responsive; compassionate/ empathetic; social; fun; ethical; community- integrated; professional; legal; efficient; scientific Relieving; safe; ethical; engaging; resource efficient; meets consumers’ needs; evidenced- based Meets consumers’ needs; safe/secure; ethical; resource efficient; accessible

11. Lessons Learned Lesson 1: Reduce anxiety Lesson 2: Manage expectations Lesson 3: Fact check reports Lesson 4: Create system for contacting program clients & stakeholders Lesson 5: Motivate program staff to complete surveys

12. LYZA INGRAHAM NICK BAYER BRANDON W. YOUKER For more information contact: youkerb@gvsu.edu