1. Assessment of Learning Objectives in Study Abroad An Integrative Model Diane Jonte-Pace, Santa Clara University Barbara Molony, Santa Clara University Joan Gillespie, IES IES April 8, 2010

2. Assessment and accreditation  New expectations for accountability/ transparency /assessment –National regulations –Regional accrediting bodies  Change is hard –Challenges on university campuses –Further challenges face Study Abroad  SCU offers meaningful & manageable assessment model –Designed during re-accreditation process –Continues in systematic multi-year cycle

3. A proposal  Step 1: Fall 2010 Pilot –Integrate IES Rome into SCU Core assessment plan –One Religious Studies course  Step 2: Future years –Integrate SCU model into IES practices more broadly

4. Accreditation and Assessment Accreditation Agencies request assessment of student learning  Middle States (MSCHE)  New England (NEASC-CIHE)  North Central (NCA-HLS)  Northwest commission (NWCCU)  Southern Association (SACS)  Western Association (WASC Examples  The institution’s student learning outcomes and expectations for student attainment are clearly stated at the course, program and, as appropriate, institutional level. WASC Standard 2.3  The institution’s faculty takes collective responsibility for establishing, reviewing, fostering, and demonstrating the attainment of expectations for student learning and attainment. WASC Standard 2.4

5. Key Concepts for SCU  Goals and Objectives –Broad Goals –Goals –Measurable Learning Objectives  Course level & program level  Explicit assignment mapping  Direct vs. Indirect assessment  Program Level Assessment  Rubric Scoring vs Grading  Systematic multi-year assessment plan

6. Similar to IES Key Concepts  Learning Outcomes –Each syllabus should list the expected outcomes for the course, i.e. what the student is expected to know and be able to do upon completion of the course.  Required Work & Assessment –The required work and content should be clearly linked to learning outcomes.  Assessment –Grading, percentages  Other IES Concepts and Practices

7. Knowledge  Global Cultures  Arts & Humanities  Scientific Inquiry  Science & Technology  Diversity  Civic life Habits of Mind and Heart  Critical Thinking  Mathematical Reasoning  Complexity  Ethical Reasoning  Religious Reflection  Communication Engagement with the World  Perspective  Collaboration  Social Justice  Civic Engagement Broad Learning Goals SCU

8. SCU goals distributed across multiple Core areas Selected Core Areas  Goals  Religion, Theology, Culture Civic Engagement Cultures & Ideas 3 KnowledgeCivic LifeGlobal Cultures Diversity Habits of Mind & Heart Crit Thinking Complexity Religious Reflection Communicat.Critical Thinking Engagement with the World Collaboration Civic Engagement Perspective

9. From Goals to Objectives at SCU  Faculty Core Committees translate goals into measurable program learning objectives for Core Area  Bloom’s taxonomy of cognitive development provides framework –Identify, Describe, Recognize –Compare, Apply, Analyze –Evaluate, Synthesize  All syllabi in Core Area include Area (Program) Objectives

10. Goals transformed into objectives Religion, Theology & Culture 2  Goals: Critical thinking, complexity, religious reflection Objectives: Students will:  2.1 Analyze complex and diverse religious phenomena, such as architecture and art, music, ritual, scriptures, theological systems, and other cultural expressions of religious belief.  2.2 Integrate and compare several different disciplinary approaches to a coherent set of religious phenomena.  2.3 Clarify and express beliefs in light of their critical inquiry into the religious dimensions of human existence.

11. Assignment Mapping: An Example from SCU  The first paper will ask students to focus on a single issue in Hinduism (e.g. spiritual practice, religious iconography, nature of self & divine) from a single disciplinary perspective (Core LO 1)  The second paper will ask students to analyze a single religious phenomenon such as a devotional song or icon from multiple disciplinary perspectives (e.g., theological, historical, ethnographic, art historical) (Core LO 1 & 2)  The third paper will focus on the experience of contemporary Hindus, with a focus on belief and practice in communities centering around female gurus in India and abroad (Core LO 1 & 3)

12. SCU multi-year assessment plan (partial list)  Fall 2010 Religion Theology Culture 2  Fall 2011 Natural Science  Fall 2012 Civic Engagement  Fall 2013 Experiential Learning Soc Justice  Fall 2014 Cultures & Ideas 3  Fall 2015 Pathways

13. Proposal, Step 1 Include IES Rome Religion Course in Multi-year Assessment Plan  IES RL 435 Monotheisms: The Children of Abraham –Term paper –Midterm –Final  Fall 2010, ~20 SCU students enrolled –All student work for targeted learning objective included in SCU assessment

14. How would the pilot assessment project work?  One learning objective selected for assessment  At SCU, work gathered from randomly selected students in each RTC2 class for selected learning objective  In Rome (with fewer students), all student work gathered for selected learning objective  Rubric scoring party January 2011  Invite IES to SCU rubric scoring party via tele-conference  Close feedback loop: Reflect on assessment reports  Goal: Enhance teaching & learning

15. Rubric Scoring Parties  6-10 volunteers  2-3 hours  Rubrics for one (or two) learning objectives  Readers calibrated with common readings  Two readers per paper; third reader if needed  Food

16. Proposal Step 2 Integrate SCU model into IES more broadly Learning objectives at the program level Explicit assignment mapping at the course level Direct assessment of selected student work Rubric Scoring Systematic multi-year assessment plan

17. Challenges  Aren’t SCU’s processes and assumptions different from IES’s?  Logistical obstacles: distance, language, distinct pedagogical culture in each IES site Challenges not insurmountable  IES shares practices with SCU  SCU and IES Learning Goals are aligned with national vision of best practices  Logistics: Exciting opportunity for global collaboration through systematic plan for rubric development

18. SCU & IES: Shared Vision & Practice  IES 3-D paradigm similar to SCU’s Goals –Cognitive Goals/Ends –Interpersonal Goals/Ends –Intrapersonal Goals/Ends  IES Syllabus Guidelines already require assignment alignment  IES is committed to ongoing assessment & improvement of student learning

19. IES 3-D Program Model & SCU’s Core Means  Ends  CurriculumCocurriculumCommunity CognitiveKnowledge InterpersonalHabits of Mind & Heart IntrapersonalEngagement with the World

20. Knowledge/ Cognitive Ends  Global Cultures  Arts & Humanities  Scientific Inquiry  Science & Technology  Diversity  Civic life Habits of Mind and Heart/ Interpersonal Ends  Critical Thinking  Mathematical Reasoning  Complexity  Ethical Reasoning  Religious Reflection  Communication Engagement with the World/ Intrapersonal Ends  Perspective  Collaboration  Social Justice  Civic Engagement SCU Learning Goals with IES “Ends”

21. Alignment with National Vision AAC&UIESSCU Knowledge of Human Cultures Intellectual and Practical Skills Inquiry and analysis Critical and creative thinking Written and Oral Communication Information Literacy Teamwork Problem Solving Personal and Social Responsibility Integrative and Applied Learning  Cognitive Interpersonal Intrapersonal Knowledge Habits of Mind & Heart Engagement with the World

22. IES & SCU: Similar Practices IES  Require clear learning objectives on syllabi at course level  Request alignment of assignments  Require assessment of student learning at course level SCU  Require learning objectives on syllabi at course & program level  Require explicit assignment mapping  Assess student learning at program level as well as course level

23. Major Difference: Program Level Assessment How does Program Level Assessment work?  Determine Program Level Objectives  Select program level learning objective for assessment  Develop rubric for assessment of study work  Gather randomly selected student work  Host 2-3 hour rubric scoring party, assess student work for selected learning objective  Provide feedback to participating departments

24. IES incorporates SCU practices 1. Develop program level learning objectives 2. Include program level learning objectives on syllabi 3. Map assignments explicitly to learning objectives 4. Develop multi-year assessment plan Logistics of Proposal

25. Summary  Fall 2010 SCU includes IES Rome course in Core Assessment plan for Religion, Theology, Culture 2  IES incorporates key aspects of SCU assessment model – Measurable Learning Objectives at program level –Explicit assignment mapping –Multi-year assessment plan Direct assessment of student work in rubric scoring parties Feedback; reflection; improvement of teaching & learning  SCU includes SCU student work from other IES sites in multi year assessment plan

26. Contact us  Office of Undergraduate Studies  Santa Clara University  www.scu.edu/assessment www.scu.edu/assessment  www.scu.edu/core www.scu.edu/core  Diane Jonte-Pace, Vice Provost, Undergrad Studies  Carol Ann Gittens, Director of Assessment  Barbara Molony, Chair, History Dept; SCU Core Coordinator; IES Curriculum Committee member  Joan Gillespie, Associate Vice President of Academic Affairs, IES