Demonstrate your value: Choosing and using a blend of methods to assess the effectiveness of information literacy instruction Carrie Leatherman Natural.

Slides:



Advertisements
Similar presentations
Building a Career Portfolio
Advertisements

E ducational T echnology I ntegration & Implementation P rinciples to guide teachers in their instructional decision making Sara Dexter University of Virginia.
Performance Assessment
Getting Started with Learning Outcomes Assessment Purposes, Practical Options, & Impact Megan Oakleaf, MLS, PhD Library Assessment Conference.
Implementation of a Scientific Literacy Project in a Large First Year Biology Class -Fiona Rawle.
Curriculum Development and Course Design
Understanding the ELA/Literacy Evidence Tables. The tables contain the Reading, Writing and Vocabulary Major claims and the evidences to be measured on.
Daniel Peck January 28, SLOs versus Course Objectives Student Learning Outcomes for the classroom describe the knowledge, skills, abilities.
Assessing student learning. Diagnostic - The gathering of information at the outset of a course or program of study to provide information for both the.
Henri Mondschein California Lutheran University January 7, 2010.
BEST PRACTICES IN INFORMATION LITERACY ASSESSMENT Yvonne Mery, Vicki Mills, Jill Newby, University of Arizona Libraries February 11, 2009.
Guidelines and Methods for Assessing Student Learning Karen Bauer, Institutional Research & Planning, Undergraduate Studies; Gabriele Bauer, CTE.
Objectives WRITING WORTHWHILE OBJECTIVES FOR YOUR CLASS.
The NEED  Technical writing is an essential skill for developing and practicing engineers.  Freshman Engineering students have difficulty  Understanding.
Design of Reusable Multimedia Resources to Deepen Information Literacy North Carolina State University Libraries Kim Duckett Principal Librarian for Digital.
Learning Objectives, Performance Tasks and Rubrics: Demonstrating Understanding and Defining What Good Is Brenda Lyseng Minnesota State Colleges.
Rubric Assessment of Student Responses to an Information Literacy Tutorial Megan Oakleaf Librarian for Instruction & Undergraduate Research Steve McCann.
One Library’s Approach Presented by Bruce Kocour Carson-Newman College
ASSESSMENT SYSTEMS FOR TSPC ACCREDITATION Assessment and Work Sample Conference January 13, 2012 Hilda Rosselli, Western Oregon University.
Data-Driven Transformation. What is Assessment? What is the Purpose of Assessment? Why Should We Care?
Winning Strategies for Assessing Student Learning Institutional Planning, Assessment, Research, and Testing (IPART) January 23, 2008.
USING STUDENT OUTCOMES WHEN INTEGRATING INFORMATION LITERACY SKILLS INTO COURSES Information Literacy Department Asa H. Gordon Library Savannah State University.
Analyzing and Improving College Teaching: Here’s an IDEA Alan C. Lacy, Associate Dean College of Applied Science and Technology Illinois State University.
Grappling with Grading Assessment & Rubrics
The model of consistent teaching of information literacy in online courses Faculty liaision librarians tutoring researchers and research students Vilve.
The Impact of Project Based Learning on High School Biology SOL Scores Rhiannon Brownell April 1, 2008 ECI 637 ECI 637 Instructor: Martha Maurno, M.S.
Evaluation 101 Everything You Need to Know to Get Started Evaluating Informal Science Education Media Everything You Need to Know to Get Started Evaluating.
Student Learning Objective – SLO
Developing an Online Credit-Bearing Information Fluency Course: Lessons Learned Rebecca Blakiston Yvonne Mery Leslie Sult University of Arizona Libraries.
Business research methods: using questions and active listening
Formative Assessment.
1 REACH Performance Tasks SY14-15 Library Science June 16, 2014.
Integrating Information Literacy Into the Classroom TLM Institute Technology & Information Literacy Mount Mary College May 30, 2002.
1 Linked Learning Summer Institute 2015 Planning Integrated Units.
Leveraging Your Library: Collaborating with Your Library Staff to Teach Science Research Andy Spinks, Lead Librarian Center for Advanced Studies at Wheeler.
Threshold Concepts & Assessment Ahmed Alwan, American University of Sharjah Threshold Concepts For Information Literacy: The Good, the Bad and the Ugly.
Supporting and Coaching Classroom Facilitators Oudete Taylor August 13, 2015.
Assessment & the Value of Academic Libraries Kelly Lambert.
Stages 1 and 2 Wednesday, August 4th, Stage 1: Step 5 National and State Standards.
Assignment Design I Katy Sullivan, Reference & Instruction Librarian Albin O. Kuhn Library & Gallery Spring 2004.
Bob Knipe Bob Knipe Dean, Learning Technologies Genesee Community College
Student Learning Assessment: An Interactive Exploration Megan Oakleaf, MLS, PhD NELIG Annual Program June 8, 2007.
Library-led Faculty Workshops: Helping Distance Educators Meet Information Literacy Goals in the Online Classroom Robert Miller Ed O’Donnell Neal Pomea.
Assessing Your Assessments: The Authentic Assessment Challenge Dr. Diane King Director, Curriculum Development School of Computer & Engineering Technologies.
A Basic Guide to Academic Assessment Presented by Darby Kaikkonen Director of Institutional Research.
1. Administrators will gain a deeper understanding of the connection between arts, engagement, student success, and college and career readiness. 2. Administrators.
Assessment Information from multiple sources that describes a student’s level of achievement Used to make educational decisions about students Gives feedback.
Using Journals to Build Information Literacy Skills Violet H. Harada University of Hawaii ACEI Annual International Conference April.
Tastes Great, Less Filling: How to Design and Deliver Substantial Instruction to Large Enrollment Classes Without Being Overwhelmed STEPHANIE JH MCREYNOLDS.
Introduction to Academic Assessment John Duffield Office of Academic Assessment Georgia State University September 2013.
Assessing Information Literacy with SAILS Juliet Rumble Reference & Instruction Librarian Auburn University.
 “I have to teach the same information skills each year because students do not learn them.”  “I don’t have time to give tests so I do not assess student.
1 An Evaluation Plan and Tool Kit for the Archibald Bush Innovative Teaching and Technology Strategies Grant Valerie Ruhe and J.D. Walker, Center for Teaching.
21 st Century Learning and Instruction Session 2: Balanced Assessment.
Developing Program Learning Outcomes To help in the quality of services.
Formative Assessment. Fink’s Integrated Course Design.
1 Teacher Evaluation Institute July 23, 2013 Roanoke Virginia Department of Education Division of Teacher Education and Licensure.
Chapter 6 Assessing Science Learning Updated Spring 2012 – D. Fulton.
Incorporating Instructional Design into Library Instruction Classes NEFLIN Live Online July 7, 2011.
Overview of Types of Measures Margaret Kasimatis, PhD VP for Academic Planning & Effectiveness.
Writing Learning Outcomes Best Practices. Do Now What is your process for writing learning objectives? How do you come up with the information?
STRATEGIES TO IMPROVE SYLLABUS DESIGN Mr. Philip Montgomery.
Learning Goals Development & Assessment The Basics of Goals-Based Course Design.
EDSS 540: Literacy in Secondary Schools Kelli Burns.
Learning Objectives for Senior School Students. Failing to plan is planning to fail. / Psychology of Achievement /
Designing Quality Assessment and Rubrics
Critical Information Literacy
Getting Started: Beginning the Assessment Cycle in a New STEM Class
Assessing Biology Student Success
ENGINEERING A BRIDGE TO INFORMATION LITERACY
Presentation transcript:

Demonstrate your value: Choosing and using a blend of methods to assess the effectiveness of information literacy instruction Carrie Leatherman Natural Sciences Librarian Western Michigan University May 17, 2013

Overview Background The Basics Why assess instruction? Planning for assessment Assessment Examples Exercise 2

Background How I got interested in assessment Provide information literacy instruction New emphasis on assessment at my institution Value of Academic Libraries report 3

The Basics Assessment “[S]ystematically examining patterns of student learning across courses and programs and using this information to improve educational practices.” (“What is the difference…,”[2013]) 4

The Basics Formative vs. Summative Assessment “Formative assessment refers to the gathering of information or data about student learning during a course or program that is used to guide improvements in teaching and learning.” (“What is the difference…,”[2013]) 5

The Basics Formative vs. Summative Assessment Summative Assessment: “The gathering of information at the conclusion of a course, program, or undergraduate career to improve learning or to meet accountability demands.” (“What is the difference…,”[2013]) 6

The Basics Direct vs. Indirect Assessment “ Direct assessment is when measures of learning are based on student performance or demonstrates the learning itself.” (“Common assessment terms,”[2013]) 7

The Basics Direct vs. Indirect Assessment “ Indirect assessments use perceptions, reflections or secondary evidence to make inferences about student learning.” (“Common assessment terms,”[2013]) 8

The Basics Learning Outcomes “Learning outcomes are statements of what students will learn in a class... The statements are focused on student learning (What will students learn today?) rather than instructor teaching (What am I going to teach today?). These statements should include a verb phrase and an impact ("in order to") phrase -- what students will do/be able to do and how they will apply that skill or knowledge.” (“Tips on Writing Learning Outcomes,” 2009) 9

The Basics Learning Outcome Example Students will be able to search a database using relevant search terms in order to retrieve articles that are on-target and topic- relevant. 10

Why Assess Instruction? Evaluate the effectiveness of instructional practices Evaluate students' mastery of skills Prove the value of library programs Accountability (“Assessment Issues,” 2013; Oakleaf, 2010) 11

Planning for Assessment Ask yourself: 1.Am I ready to do assessment? 2.Why am I doing assessment? 3.What are stakeholders’ needs? 4.Will the assessment tell me what I want to know? 5.What are the costs of assessment? 6.What are the assessment’s institutional implications? (Oakleaf and Kaske, 2009) 12

Planning for Assessment Think about assessment approaches: Fixed-choice tests Performance assessments Rubrics (Oakleaf, 2008) 13

Planning for Assessment Fixed-choice tests Indirect assessment Measures acquisition of facts Pros Collect a lot of data quickly; easy to score Can use for pre- and post-instruction comparison Cons Don’t test higher-level thinking skills Difficult to construct and analyze (Oakleaf, 2008 ) 14

Planning for Assessment Performance Assessment Direct assessment Measures real-life application of knowledge, skills Pros Captures higher-order thinking skills Supports learning and assessment in authentic contexts Cons More difficult to create, administer, and score (Oakleaf, 2008 ) 15

Planning for Assessment Rubric “A rubric is a scoring tool that explicitly represents the performance expectations for an assignment or piece of work. A rubric divides the assigned work into component parts and provides clear descriptions of the characteristics of the work associated with each component, at varying levels of mastery” (“Common Assessment Terms,” 2013) 16

Planning for Assessment Rubrics Measures quality of student performance Pros Means to create common instruction values Focuses on higher-order thinking skills Produces data that is easy to understand, defend, and covey Cons Design flaws can affect data quality Time-intensive to develop, use, train raters to use (Oakleaf, 2008 ) 17

Example 1: One-Shot Instruction Background Intro biology class for biology majors Past classes used poor-quality sources for class project Class project requirements Group presentation about area of molecular biology Required sources: Empirical research article Review article Three other sources 18

Example 1: One-Shot Instruction Collaborated with instructor and TAs Students’ information literacy skills needed to improve Formulated learning objectives One-shot instruction session 19

Example 1: One-Shot Instruction Desired learning outcomes of instruction Students will be able to: From a detailed research question in order to define key terms, relevant to research topic, to use as article database search terms Identify components of empirical research article in order to locate one relevant to project. Identify components of review article in order to locate one relevant to project. 20

Example 1: One-Shot Instruction Before instruction asked myself 1.Am I ready to do assessment? Yes! 2.Why am I doing assessment? Increase student learning; accountability; improve instruction 3.What are stakeholders’ needs ? Non-anecdotal; unobtrusive 4.Will the assessment tell me what I want to know? Baseline; new info; trustworthy; meets needs 5.What are the costs of assessment? Time; personnel costs 6.What are the institutional implications of the assessment? Information literacy goal of libraries, university What if the instruction isn’t effective? 21

Example 1: One-Shot Instruction Assessment methods selected Pre-, post-instruction quizzes (Fixed-choice) Open-ended questionnaire (Performance, Rubric) Bibliography analysis (Performance, Rubric) All administered within class elearning shell 22

Example 1: One-Shot Instruction Pre-, post-instruction quizzes 23

Example 1: One-Shot Instruction Open-ended questionnaire 24

Example 1: One-Shot Instruction Bibliography Analysis 25

Example 1: One-Shot Instruction Rubric for open-ended questionnaire 26

Example 1: One-Shot Instruction Rubric bibliography analysis 27

Example 2: Online Tutorial Background Anecdotal evidence that online information literacy tutorial outdated Created a new version of tutorial Updated content and interface Same concepts New tutorial More effective? Preferred by students? 28

Example 2: Online Tutorial Desired learning outcomes of instruction Students will be able to: Define an appropriate research question given assignment criteria in order to select a manageable focus within a general topic. Construct effective search queries in order to locate information resources for a given research question. Apply evaluation criteria to resources in order to identify ones most appropriate to research question. Recognize when it is necessary to provide a citation for information sources in order to avoid plagiarism. 29

Example 2: Online Tutorial Before instruction asked myself 1.Am I ready to do assessment? Yes! 2.Why am I doing assessment? Increase student learning; accountability; improve instruction 3.What are stakeholders’ needs ? Non-anecdotal; time consuming but compensation 4.Will the assessment tell me what I want to know? Baseline; new info; trustworthy; meets needs 5.What are the costs of assessment? Time; personnel costs 6.What are the institutional implications of the assessment? Information literacy goal of libraries, university What if the instruction isn’t effective? 30

Example 2: Online Tutorial Assessment methods Quizzes (Fixed-choice; direct method; summative) Hypothetical research project (Performance; direct; summative) Focus groups Satisfaction with new tutorial Not learning outcomes-based assessment! 31

Example 2: Online Tutorial Quizzes 32

Example 2: Online Tutorial Hypothetical research project 33

Example 2: Online Tutorial Rubric for hypothetical research project 34

Miscellaneous Triangulating data via multiple assessment methods Statistical analysis increases validity Research that involves human subjects What campus resources are available? 35

Lessons Learned Assessment Is increasingly important Takes planning Various methods and approaches Is ongoing Is worthwhile! 36

Exercise Think Pick an instructional activity you would like to assess Ask yourself: 1.Am I ready to do assessment? 2.Why am I doing assessment? 3.What do stakeholders want to know? 4.Will the assessment tell us this? 5.What are the costs of assessment? 6.What are the assessment’s institutional implications? Pair With a partner’s help, choose an assessment method that would work best for your situation Help your partner choose an assessment method Share! 37

References Association of College and Research Libraries Assessment Issues. Retrieved from Carnegie Mellon Eberly Center for Teaching Excellence and Educational Innovation. [2013]. Common assessment terms. Retrieved from Carnegie Mellon Eberly Center for Teaching Excellence and Educational Innovation. [2013]. What is the difference between assessment and grading? Retrieved from Connaway, L. S., & Powell, R. R Basic research methods for librarians (5th ed.). Santa Barbara, Calif.: Libraries Unlimited. Oakleaf, M Dangers and opportunities: A conceptual map of information literacy assessment approaches. portal: Libraries & the Academy, 8(3), Oakleaf, Megan Writing information literacy assessment plans: a guide to best practice. Communications in Information Literacy 3(2): Oakleaf, Megan Value of Academic Libraries: A Comprehensive Research Review and Report. Chicago: Association of College and Research Libraries. Oakleaf, Megan and Kaske, Neil Guiding Questions for Assessing Information Literacy in Higher Education. portal: Libraries & the Academy 9(20): University Library, University of Illinois at Urbana-Champaign Tips on Writing Learning Outcomes. Retrieved from 38

Questions? 39

Thank you! Carrie Leatherman 40