A Case Study in Evaluation & Assessment for a Virtual Learning Commons Mellon Library/Faculty Fellowship for Undergraduate Research Pat Davitt Maughan | Project Manager Mellon Library/Faculty Fellowship for Undergraduate Research University of California, Berkeley

“assessment alone changes little” “Nine principles of Good Practice for Assessing Student Learning” American Association for Higher Education (AAHE)

VISION & VALUES GOALS & OBJECTIVES QUESTIONS & METHODS DATA COLLECTION & ANALYSIS REPORTING audiences & stakeholders...… audiences & stakeholders …… audiences & stakeholders project reports ……… theme reports student feedback

The assessment plan Captures agreement about what matters Gives directions for actions Provides a way of knowing if progress is being made Provides feedback to students & stakeholders Feeds back into program/course development to improve performance & learning

Assessment plan | success indicators Reflects an understanding that learning is …  multidimensional | integrated | revealed in performance over time Compares educational performance against student learning outcomes

UC Berkeley Mellon Library/Faculty Fellowship Program for Undergraduate Research

Mellon Program Elements Collaborations Institute I-teams Assessment

Assessment | Programmatic To create a scaleable & sustainable model for promoting changes in courses & curricula To create a community of fellows who demonstrate their commitment to student learning through inquiry based strategies & serve as change agents To compile student performance data on the library research components of course assignments as one indicator of student learning To expand the community of faculty committed to providing undergraduate research that requires the use of library collections To strengthen collaboration among Academic Partners in support of instructors and instruction

Assessment | Thematic Show themes and Questions

Student Learning Assessment | Chemistry 1A 1,200 students | highest impact lower division course | important gateway course BEFORE MELLON Involve students in a structured process of scientific inquiry Paired students to apply basic chemistry concepts to real life problems Students presented findings in poster session

Case study | Chemistry 1A AFTER MELLON Assignment staged over longer period Developed a schedule that detailed all assignment related activities GSI training | introducing the assignment | locating & evaluating sources| hypothesis development | development/use of rubric Students explicitly led through processes of topic selection, hypothesis formation, research design, information gathering, communicating results Assignment & performance expectations formally introduced

Case study | Chemistry 1A AFTER MELLON i-Team & Fellow developed 2 student worksheets worth points 1 – hypothesis development 2 – evaluation of sources students documented research methods GSIs provided feedback i-Team & Fellow developed course web site - posted Grading rubrics for disciplinary content, information literacy & presentation skills | Models | Information sources | short tutorials Rubric allowed uniform grading across sections & used to collect performance data on discrete learning outcomes

Case study | Chemistry 1A Assessed from three data streams  Students Projects | self assessment  GSI’s assessment of their learning | student learning  Faculty’s assessment of what students learned

Summary Project Assessments Collaboration  mid-year | post-Institute survey of Academic Partners Institute  pre-, mid- and post-Institute Surveys Faculty Fellows  reflections on their own learning & their students’ learning | changes in practice | video interviews Student learning  performance & rubrics  post-course self-assessments  UCUEs (UC NSSE correlate) surveys  video interviews

Project outcomes Past 3 years, 35 faculty  redesigned 33 lower & upper division courses to include research components influencing 8,000 students  many extended research-based & IL into other courses Partner units developed a richer appreciation for the multiple skills & perspectives offered by their peers Good practices are shared  among faculty, librarians, and partner contributors

Project outcomes Closer relationships between the library & campus teaching & learning units  ETS | GSI | AC | Student Advising | OED | DUE New focus within the library on library’s educational role  New AUL for Educational Initiatives  Strengthening connections between the library & other campus units – teaching & learning focus Library involvement in other campus wide initiatives  e-Berkeley symposium “Teaching & Critical Thinking in the ‘Point & Click’ Age” – spotlighted Mellon models  B-space | Sakai

Acknowledgments The thoughts and writings of the following individuals informed the development of parts of this presentation: Elizabeth Dupuis, Head of Instructional Services University of California, Berkeley Cynthia Schrager, Special Assistant to the Vice Chancellor for Undergraduate Education University of California, Berkeley Mary Sue Ammon, Assessment Consultant Mellon Library/Faculty Fellowship for Undergraduate Research