LOEX 2007 Cathy Palmer Head, Education and Outreach UC Irvine Libraries Applying Student Learning Data to Enhance Instruction An Anteater Initiative LOEX 2007

Outline  Background  Program Overview  Incentives/Challenges  Methodology  Results  Lessons Learned  What next?

Writing Program/Library Collaboration  Lower-division writing requirement  Fulfilled in part by successful completion of Writing 39C: Argument and Research  Course integrated library research assignment since 1986  Library research assignment evolved as course curriculum evolved

Library Incentives for Evaluation  Major investment of library resources Personnel Space  Availability of numeric data Number of students reached Number of sessions taught Number of librarian hours spent  Were our resources being put to the best use?

Yearly Statistics 1996/ Students 76 Sessions 82 Hours 2005/ Students 126 Sessions 150 Hours

Session Outline  Libraries Homepage (Lecture/Demo) Ask a Librarian Connect from off-campus  ANTPAC catalog (Lecture/Demo)  Expanded Academic (Lecture/Demo)  Hands-on exploration of assigned databases Sociological Abstracts L/N Congressional Universe United States Newspapers  Review of each database (Discussion)  Evaluations

Evaluation 1.0 Overall Value of Session to Student Very Useful59144% Useful62146% Somewhat Useful1159% Not Useful101% No Response00% Total

Student rating of Librarian Presentation Understand Clearly61650% Understand52543% Understand Somewhat 20% Don't Understand00% No Response847

Understand Clearly52940% Understand70252% Understand Somewhat695% Don't Understand30% No Response353% Choose Databases—student self rating

Select Search Terms-student self rating Understand Clearly52339% Understand61846% Understand Somewhat695% Don't Understand20% No Response1269%

Evaluate Results of Search-self rating Understand Clearly59344% Understand53040% Understand Somewhat 796% Don't Understand60% No Response13010%

Interpret Citations-Find Material-self-rating Understand Clearly45734% Understand59144% Understand Somewhat 14511% Don't Understand111% No Response13410%

But... What were the students REALLY learning in the session?

The "Eureka!" moment Esther Grassian workshop Ask the students questions about the content of the session as part of the evaluation.

The BIG IDEA Explicit Objective: Students will understand that they have a variety of information resources available to them and that each resource has strengths and weaknesses. They will be challenged to answer the question "Why use this resource rather than that resource?"

The Invisible BIG IDEAs Implicit Objectives: Students will leave knowing that the librarians are there to help them. Involve students in their own learning by making the session as hands-on as possible.

Learning Outcomes At the end of the 50 minute session, students will:  Recognize the characteristics of generalized and specialized information resources and be able to differentiate between them.  Learn and apply basic search strategies that will help them locate information  Select, explore and use 3-5 library databases to find information on their topic.

Session Outline  Libraries Homepage (Lecture/Demo) Ask a Librarian Connect from off-campus  ANTPAC catalog (Lecture/Demo)  Expanded Academic (Lecture/Demo)  Hands-on exploration of assigned databases Sociological Abstracts L/N Congressional Universe United States Newspapers  Review of each database (Discussion)  Evaluations

Early Results

Read each statement and indicate the best resource to use by placing a check in the appropriate column. (Fall, 2003—383 responses) ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find books Find scholarly articles Find scholarly and popular articles Get help Find electronic copy 62% 38% 53% 62% 77%

Let's use Sociological Abstracts as the demonstration database instead of Expanded Academic ASAP. Another Eureka Moment

Read each statement and indicate the best resource to use by placing a check in the appropriate column. (Spring 2004, 809 evaluations) ANTPACExpanded Academic ASAP Sociological Abstracts UC elinks Ask a Librarian Find books Find scholarly articles Find scholarly and popular articles Get help Find electronic copy 73.9% 48.3%52.7% 55.8% 74.7% 64.4%

Student Performance Comparison Fall, 2003Spring, 2004 Use ANTPAC to find books Use Soc Abs to find scholarly articles Use Expanded Academic to find popular and scholarly Ask a Librarian for help Use UCeLinks to find electronic copies

Bonuses Ability to look for themes in student comments What parts of the presentation helped you the most?  Following along step by step (71)  Visual aid/demonstration (73)  Group work (83)

Student comments cont. What helped you the least?  Group work (59)  Not enough time (38)  Nothing/everything helped (202) What could improve the session?  Longer session (184)  Slower and more detail about resources (58)  Speak louder/microphone

Still more student comments I still need to learn more about:  What each database is used for and when to use it for certain searches (78) The other databases we didn't get to explore (59) How to find the actual copies of books and articles in the library (34)

Problems with Paper ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find booksXXXXX Find scholarly articles XXXXX Find scholarly and popular articles XXXXX Get helpXXXXX Find electronic copy XXXXX

Problems with Paper ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find booksXXX Find scholarly articles XXX Find scholarly and popular articles XXXXX Get helpXXX Find electronic copy XXX

Problems with Paper ANTPACExpanded Academic ASAP Sociological Abstracts UCelinksAsk a Librarian Find booksXXX Find scholarly articles XXX Find scholarly and popular articles XX Get helpXXXXX Find electronic copy XXX

Benefits of Automating Evaluation Form  Could only select one response  More comments/longer comments  Results available immediately  Results could be sorted in a variety of ways

Numeric Results  Individual Librarian/Individual Session  Individual Librarian/All Sessions  Overall evaluations by quarter  Overall evaluation by academic year

Individual Performance Improvements Assessment/Feedback methodology  Use of Observation Rubric –Presentation style Use of projected image Can be heard –Session Content Outlines session Pacing –Classroom management Interaction with students Execution of learning techniques  Comparison between Observation and Evaluations

Results Clear, concise presentation + Hands-on + Review of experience Best performance by students on selection of best resource

Example

Another Example

Librarians: All Academic year: Total evaluations: 1733 Q1. Session Content The Library Research Skills session was useful. Strongly agree:873(50%) Agree:778(45%) Neutral:64(4%) Disagree:9(1%) No opinion:1(0%) I learned new skills.Strongly agree:907(52%) Agree:713(41%) Neutral:78(5%) Disagree:18(1%) No opinion:3(0%)

Librarians: All Academic year: Total evaluations: 1733 Q2. Instructors The librarian presented information in a way that I could understand. Strongly agree:970(56%) Agree:689(40%) Neutral:42(2%) Disagree:15(1%) No opinion:4(0%) The librarian encouraged and responded to questions. Strongly agree:960(55%) Agree:636(37%) Neutral:105(6%) No opinion:8(0%) Disagree:8(0%)

Q3. Choose the Best Resource A. I can use this resource to find out if the UCI Libraries own the book Reckoning with Homelessness. Antpac:1315(76%) ASAP:134(8%) Ask a Librarian:74(4%) Sociological Abstracts:84(5%) UC-eLinks:73(4%) D. I can get help with my topic by using this resource. Antpac:165(10%) ASAP:176(10%) Ask a Librarian:1160(67%) Sociological Abstracts:76(4%) UC-eLinks:99(6%) E. I can use this online resource to see if there is an electronic copy of an article available. Antpac:334(19%) ASAP:197(11%) Ask a Librarian:46(3%) Sociological Abstracts:50(3%) UC-eLinks:1042(60%)

Q3. Choose the Best Resource B. I can use this online resource to locate scholarly peer-reviewed journal articles on the topic of homelessness. Antpac:160(9%) ASAP:660(38%) Ask a Librarian:21(1%) Sociological Abstracts:779(45%) UC-eLinks:52(3%) C. This is a good resource to use when I am beginning my research because it covers both scholarly journals and popular magazines. Antpac:211(12%) ASAP:1078(62%) Ask a Librarian:25(1%) Sociological Abstracts:287(17%) UC-eLinks:77(4%)

Future Wishlist  Pre and post testing  Ability to share results of evaluation with students  More nuanced testing of ability to apply concepts learned  Consistent evaluation of learning outcomes from other library research skills sessions

Questions?

Cathy Palmer Contact Me