0. Mission and Educational Effectiveness at
Middlebury College

1. Agenda
About Middlebury
Middlebury’s Mission
Assessment Challenges

2. About Middlebury
Middlebury College 2,475 undergraduates Language Schools 1,500 students, summer immersive instruction Non-degree, MA, and DML programs Bread Loaf School of English 400 graduate-level students Non-degree, MA, MLitt programs

3. About Middlebury
Middlebury C.V. Starr Schools Abroad 37 sites, 17 countries Middlebury Institute of International Studies at Monterey 700 master’s degree students School of the Environment 20 summer students

4. Mission
At Middlebury College we challenge students to participate fully in a vibrant and diverse academic community. The College's Vermont location offers an inspirational setting for learning and reflection, reinforcing our commitment to integrating environmental stewardship into both our curriculum and our practices on campus. Yet the College also reaches far beyond the Green Mountains, offering a rich array of undergraduate and graduate programs that connect our community to other places, countries, and cultures. We strive to engage students' capacity for rigorous analysis and independent thought within a wide range of disciplines and endeavors, and to cultivate the intellectual, creative, physical, ethical, and social qualities essential for leadership in a rapidly changing global community. Through the pursuit of knowledge unconstrained by national or disciplinary boundaries, students who come to Middlebury learn to engage the world. - Adopted by the Board of Trustees, March 2006, reaffirmed in February 2011

5. Mission: Key Components
Setting (sense of place)
Environmental stewardship
Connect our community to other places, countries, and cultures
Rigorous analysis and independent thought
Intellectual, creative, physical, ethical, and social qualities essential for leadership in a rapidly changing global community
Pursuit of knowledge unconstrained by national or disciplinary boundaries to engage the world

6. Assessment Challenges
No formal or documented systems or mechanisms
Learning goals
Assessment measures
Direct evidence
Lack of awareness
Varying levels of buy-in
We did have:
Good amount of indirect evidence
Faculty who were doing assessment
Faculty who were revising their curricula based on assessment
Pockets of faculty who were eager to formalize assessment

7. Comprehensive Evaluation Result
The not-unexpected news:
April 4, 2012: “…the College give emphasis to its success in… implementing a comprehensive approach to the assessment of student learning at the undergraduate level…”
The good news:
Groundwork had been laid with learning goals for majors and administrative structures in place

8. Mission and Educational Effectiveness at
Middlebury College

9. Agenda
Structures
Approach
Examples

10. Structures Institutional Support Assessment functions
Educational Affairs Committee
Oversees College's curriculum
Led development of college-wide learning goals
Six faculty members
Assessment Committee
Oversees curriculum-wide assessment efforts

11. Structures Office of Assessment and Institutional Research
Assists departments and programs with direct and indirect data collection
Provides departments/programs with data the institution already has (historic enrollments, senior and other student surveys, study abroad rates, post-graduation, etc.)
Monitors progress on department/program assessment

12. Approach Institutional support at the highest level
Collaborative and guiding, not demanding
Develop learning goals for each major
Learning goals already existed but were not documented
Provided examples from other schools and professional associations
Small group and 1-on-1 consultations

13. Approach Develop assessment methods
Help faculty realize they are already “doing assessment”
Provide examples and rubrics from other schools and external groups
Use existing data and indirect evidence to identify areas of concern
Keep the assessment efforts manageable
Develop college-wide learning goals
Used existing faculty governance structures
Codified what we were already doing
Overwhelmingly endorsed

14. Neuroscience Program
Survey data indicate variability with student satisfaction with the curriculum
Program faculty undertook a curriculum mapping exercise
Identified that they had to teach the same specific foundational material in a number of courses due to varying points of entry into the major
Conducted focus groups to gather additional data about student experience
Students reported the curriculum was sometimes repetitive and limiting
External review of the program provided feedback on the structure of the curriculum
Result: restructured major that included a common entry point into the major and other curricular changes

15. Chemistry Department
In 2010, department instituted a new methods course to improve writing of a primary research article based on independent work
Methods course provided practice in writing in the discipline and exposure to lab techniques students could use in projects
In , assessment showed a higher quality of senior work submitted to the department and that students had valuable skills for post-graduate labs

16. Philosophy Department
Assessing learning goal of majors’ “ability to recognize, analyze, and evaluate structures of arguments”
Administer a “logic quiz” to assess one of their learning goals
Given during the introductory course and to seniors
Marked improvement over time (44% to 75%)
After a couple of years, debate around whether the logic quiz is an effective assessment model
Also have conducted focus groups with assessment office
Highlighted repetition in the curriculum and prevalence of non-majors in 300-level courses
Recent external review will help guide curricular changes

17. Oral Communication Skills
2012 Alumni Survey
Identified a gap between the value alumni placed on oral communication skills and what they learned about those skills at Middlebury
Widest gap among the 14 skills and capacities
Value Today
Learned at Middlebury
4.8
4.0

18. Oral Communication Skills
Assessment Committee surveyed faculty asking about oral communications instruction prior to and following a presentation

19. Oral Communication Skills

20. Oral Communication Skills
Based on the faculty survey data, assessment committee sought to directly assess oral presentation skills
Adapted AAC&U Oral Communications VALUE rubric
Organization
Language
Supporting materials/using supporting materials effectively
Central message
Delivery
Integration of multiple formats of information simultaneously

21. Oral Communication Skills
Created repository of recorded student presentations
Committee members scored presentations and then discussed assessments
Scores averaged around 3 on a 4-point scale
Use of multiple formats scored lowest
Beyond this assessment, committee was pleased with the huge benefit of faculty having conversations about teaching and learning
Expand this to others and think of it as faculty development

22. Lessons Learned Institutional support was essential
Collaborative and supportive approach works
Acknowledge that learning goals and assessment already do exist to help bring faculty along
Use indirect evidence to find areas of concern and further exploration
Find faculty ambassadors of assessment

23. Mission and Educational Effectiveness at
Middlebury College