Collaborative Development of a Library Instruction Assessment Plan for Continuous Improvement Sarah Lucchesi Learning Services Librarian University of Southern Maine

University of Southern Maine Four year public university 7,800+ students 3 campuses Portland Gorham Lewiston/Auburn - Before we jump in, wanted to share brief background info on USM to help you contextualize our instruction and assessment process. - We work with many courses on campus across a wide range of majors and colleges.

Assessment Background No formalized university learning goal for info lit Little coordination among librarians who teach Developed internal self-assessments to as a stepping-stone to larger assessment effort Rubric assessments only allow us to evaluate our work in course-integrated sessions where students produce easily assessable work products such as written papers or presentations. This talk will focus on the development of our continuous improvement processes and benefits we have seen so far.

Assessment Goals Self-Assessment Program Assessment Improve teaching practice Determine value of IL instruction to students Develop new pedagogical skills Determine benefits to the university Looks at individual classes and librarians Looks at entire instruction program - higher level

Program Development April 2018 Listed data that could be collected from four groups Students Faculty Each other Ourselves Examples of data included session worksheets, surveys, final papers or projects, informal conversations, lesson plan feedback from colleagues, workshop attendance, and resource usage. (c) someecards.com

Program Development Settled on four methods: Peer observation Student work analysis Self-assessment Student and/or faculty feedback Retreat – focused on developing outcomes, some academic year planning, and developing continuous improvement processes. Got off campus! Very important.

Peer Observation Each librarian observed once/semester Pre-meeting and post-meeting Open-ended discussion questions vs. rubrics Both parties benefit Pre-meeting no more than a week in advance. Discuss any specific areas of focus, as well as any off-limits areas. Post-observation meeting no more than a week after. Elected to use an open-ended form rather than a rubric because we wanted to encourage reflection and discussion rather than a summative evaluation. The score is not the point. Librarians give input to coordinator on the type of sessions or skills they are interested in observing. For example, if one librarian is very good at providing context and another librarian is interested in developing her skills in that area, that would be a good match for observation. Librarian being observed feels that they are modeling a good practice as well as being evaluated.

Student Work Analysis Focus on in-session work Formative or summative One course per semester Determine whether in-session activities are useful Example: collect student worksheets Same class over multiple semesters How well do students understand concepts based on worksheet answers? While rubric assessments focus on end-of-semester student work products, this method focuses on in-session student work such as worksheets or answers to online forms, etc. Are students struggling with in-session work? Conversely, is it too easy? Goal is to modify future sessions to better fit student needs

Self Assessment Based on ACRL Proficiencies for Instruction Librarians Each librarian modifies for own responsibilities Once/year Most personal and flexible of new methods Uses include: Setting annual goals Identifying professional development Basis for journaling Identifying skills to observe in peer observation. ACRL Proficiencies include instructional design, administrative, communication, teaching, and others. Not every teaching librarian has administrative duties, and some do very little instructional design. Each librarian therefore modified the list of proficiencies to reflect their own job duties. Each year, in the spring. Then summer is spent on an improvement plan. Since this could touch on sensitive areas, librarians can use this tool however they like. No requirement to share with the group or discuss with supervisor.

Student and Faculty Feedback Students: If you previously had a library session, how much of today’s session was still new to you? The librarian was approachable and I left comfortable asking questions (if I had them) (agree/disagree scale) What was the most/least useful thing about today’s session? (free response) Name two differences between a scholarly/academic journal and a magazine. (free response) Faculty: The library session met the information needs of my students (agree/disagree scale) Student participation was encouraged (agree/disagree scale) The pedagogical techniques used during this session were effective and appropriate to the information covered. (free response) How do you anticipate that the information covered in the library session will help your students achieve the course goals? (free response)

Challenges Team building needed to develop trust Long history of working independently Fear of how assessment data will be used Slower start than I had hoped for

Next Steps Survey faculty again at end of semester Bring more formalization to processes Close the loop on self-assessment Program-level assessment Continue pedagogical sessions for teaching librarians Surveying faculty at the end of the semester to see if library instruction helped with final products - don’t want to be too annoying sending them multiple surveys. Also not sure if they will respond after end of semester, but we’re going to try it. More formalization – tried to do too much all at once with the implementation of these different methods. Improve processes for collecting and assessing in-session student work, as well as some type of close the loop activity on self-assessment. Hope to do another retreat this August and talk about improvements. Currently we don’t survey students who have been in an instruction session – many studies have shown that students have difficulty determining whether a particular class or lesson was useful to their learning so this has not been a priority for us, but we do want to try it. All of the librarians agreed that once per semester pedagogical sessions focused on a teaching technique would be very helpful. Last summer we talked about developing session outcomes, and over the winter we had a team lunch where we discussed ways to encourage student participation in class. Ideas for these pedagogical sessions can now come from the continuous improvement processes.

Sarah Lucchesi sarah.lucchesi@maine.edu Thank you! Sarah Lucchesi sarah.lucchesi@maine.edu