1. Assessing the Engaged Student: Using Evaluation Tools to Reexamine and Strengthen Civic Engagement Programs Ethan A. Kolek, Associate Director of Institutional Research Sarah C. Barr, Public Service Internship Coordinator Amherst College March 30, 2009

2. Evaluation Research Concern with Policy Space Stakeholders Program Effectiveness Summative Evaluation Formative Evaluation

3. Center for Community Engagement Data Collection Methods Quantitative –Tracking/Participation –Institutional Database –Surveys –Pre/Post Tests Qualitative –Focus Groups –Interviews –Analyses of student blogs –Student Self Assessment and Staff Observation

4. Surveys + Generalizable Results + Efficient collection/analysis + Confidentiality/anonymity - Low response rate - Questionnaire design - Best with objective/attitudinal items - Lack nuance

5. Focus Groups and other Qualitative Methods + Complex data + Context & Depth + Interpretation + Exploration + Participant voice - Not generalizable - Time-consuming analysis

6. Center for Community Engagement Created in 2007 Major programs –Community Based Learning Courses and Research –Community Engagement Leaders –Internships Fellowships for Action Amherst Select Internship Program –Pioneer Valley –Beyond

7. Pioneer Valley Summer Internship Program Created in 2008 38 students Interned at 22 community based organizations Worked four days a week for 8 weeks Participated in enrichment activities every Friday from 9 am to 4 pm

8. What Data Did We Need? Multiple perspectives on program Generalizable data Formative Assessment of Program Learning/Development Student Voice

9. What We Did End of Experience Web Survey (Interns) Evaluations of Interns (Supervisors) Evaluation of Program (Supervisors) Blogs (written by Interns)

10. Student Survey Satisfaction Preparation Internship Work Impact on Organization Workplace Climate Reasons for participation Learning and Skill Development Demographics

11. Supervisor Evaluation How much did the intern work? What did they do? Were they paid/compensated? How did interns perform? Were interns prepared? What skills did they development? Overall Evaluation

12. Program Evaluation Intern selection process Impact of Friday programming Impact of interns on org. capacity Working with CCE

13. Blogs Reflection Goal setting Analysis of significance of work Connections between activities and goals

14. Results

16. Please indicate the extent to which you felt prepared in each of the following areas. If an area does not apply to you, please mark "Not Applicable."

17. Results Please estimate the proportion of time you spent working in each of the following areas during your internship (your response total must equal 100).

18. Results Please indicate whether each of the following were a major reason, a minor reason or not a reason why you participated in the Citizen Summer Internship Program.