1. Assessment of Service-Learning: Principles and Techniques Barbara A. Holland, Ph.D. Senior Scholar, Indiana University-Purdue University Indianapolis Director, National Service-Learning Clearinghouse

2. Activity What is motivating you to think about assessment?

3. Effective Assessment Plans Focus on core issues Reinforce common definitions/terms Are grounded in data Reflect all participant perspectives Document strengths and areas for improvement Anticipates the audiences for and applications of results What you measure is what you get!

4. Assessment as Improvement Mechanism to tell your story What have you learned that is useful –For your own work –For you to share with others Focus on documenting impact and outcomes; barriers and facilitators Considers all perspectives; SL is collaborative

5. Outcomes of Assessment Descriptive information Analytic information and comparisons Case studies Evidence of impacts Principles of good practice Ideas for program improvement

6. Good and Bad Assessments Good assessments will: –define strengths –validate knowledge –provide evidence for resource decisions –identify opportunities for improvement Bad assessments will: –consume energy and resources –Undermine program activities

7. Service-Learning is: Integrated into courses or other learning activities to meet specific learning objectives in ways that also… Enhance community Transform participants Are intentional and rigorous Experiential and Reflective

8. Planning for Service-Learning What are the expected learning goals? How does SL help students reach those goals? What will be the teaching role of community? What will be evidence of mastery of content? How will we measure impact on student learning and development? How will we measure impact on community capacity?

9. Learning Objectives Community Learning about: –A particular community or population –A particular issue, challenge, opportunity –The provision of services to community –A particular organization or grass-root effort –Relevant public policies; historic perspectives –The role of stakeholders

10. Learning Objectives (Continued) Inter and Intra-personal Learning –Working collaboratively with others –About other groups and cultures (diversity) –Practicing effective communications –Developing self-efficacy –Developing empathy –Learning to appreciate different views

11. Learning Objectives (Continued) Learning to be a Learner –Active -Independent –Extract meaning from experience –Apply knowledge to real world –Use evidence to articulate ideas –Learn across subject areas –Find and assess the quality of information resources

12. Activity Using a few key words, describe one or two goals for service-learning….what impacts do you expect service-learning will have on: Students, or teachers/leaders/faculty, or community partners?

13. Planning for Assessment Before you Begin: What is the aim of your assessment? Who wants/needs the assessment? What resources are available? Who will conduct the assessment? How can you ensure results are used?

14. The Matrix/Multiple Method Approach Using your service-learning goals as a framework: Build an Assessment Matrix –Core concepts –Key indicators –Multiple methods –Sources of information Use the matrix for implementation, analysis and reporting

15. Assesment Design Project goals – What do we want to know? Core Concepts – What will we look for? Indicators – What will be measured? Methods – How will we measure? Then: –Analysis –Improvement actions –Dissemination

16. Key Concepts: Students Awareness of community Commitment to service Career exploration Self-awareness Understanding course content Communications skill development Cross-cultural skills

17. Student Example Goal: Service-learning helps students discover their potential role in community life. Concept: Career exploration Indicators:Expressed career interest; Demonstrated career skills and attributes; Knowledge of career requirements; Understanding of career responsibility to public Methods: Survey, interviews, journals

18. Another Student Example Goal: Prepare students to be effective and active citizens in their communities Concept: Awareness of community Indicators: Knowledge of issues, ability to identify assets/needs, understanding of problems and policies Methods: survey, interviews, observation

19. Key Concepts: Faculty/Teacher Motivation and attraction to engagement Professional development Impact/influence on teaching strategies Impact/influence on scholarship (higher ed) Other personal/professional impact Identification of barriers and facilitators Satisfaction with experience

20. Faculty/Teacher Example Goal: Service-learning will improve teaching Concepts: philosophy of teaching, teaching and learning strategies Indicators: teaching roles, class format, organization, environment, values Methods: observations, interviews, lesson plans

21. Key Concepts: Community Capacity to fulfill organizational mission Economic effects Social benefits Perception of mutuality Satisfaction Sustainability of the partnership

22. Community Example Goal: SL activities are collaboratively designed to meet organizational needs of partner organizations Concept: Capacity to fulfill mission Indicators: Insights about organizational directions and operations; staff impacts; Number of clients served; Changes in activities offered; Insights into assets and needs; Leveraged resources/funding Methods: Interview, observation, reports/documents

23. The Role of Partnership Assessment Strengthen the partnership Build a foundation of mutual understanding, based on a clear philosophy and common goals Reinforce mutual learning and decision-making Focus on feedback and improvement Remember that attitudes and perceptions matter as measures of benefit and satisfaction with the partnership

24. Activity Think of your own SL goals: –Identify one key concept you want to assess (what will you look for?) –For that concept, propose two measurable indicators (what will you measure?) Focus on concepts and indicators; DO NOT think about data collection methods!

25. Selecting Assessment Methods What instrument(s) to use Why and when to use it/them Consideration of characteristics –Types of questions –Format/design Process of data collection What to do with the data

26. Effective Methods Focus groups – Efficient, interactive Interviews – time intensive; deeper views of individual experiences Observation- time intensive; multi-purpose Surveys – time efficient, objective, anonymous, can be superficial Journals, syllabi, documents, site reports- useful for validation and cross-checking

27. Resources Gelmon, Holland et al., Assessment Handbook, Campus Compact, 2001(www.compact.org) Furco Institutionalization Assessment Rubric, Forthcoming from Anker Pub, 2004 afurco@uclink4.berkeley.edu CART (Compendium of Assessment and Research Tools). RMC Research http://cart.rmcdenver.com/ National Service-Learning Clearinghouse www.servicelearning.org

28. Contact Barbara A. Holland, Ph.D. Director, National Service-Learning Clearinghouse www.servicelearning.org Barbarah@etr.org Toll-free 866-245-7378 ext. 273