Assessing Your Instruction Program Texas Library Association April Michelle Millet Information Literacy Coordinator, Trinity University, San Antonio, TX Yvonne Nalani Meulemans Science Librarian, Calif. State University, San Marcos, CA

Overview  History of Assessment and Information Literacy  Best Practices  Assessment at CSU-San Marcos  Assessment at Trinity  Programmatic assessment  Other opportunities

Assessment of IL: a brief history  1. Higher education assessment movement  2. Strategic planning and TQM adoption  3. Evolution of information literacy

Activity 1  What does assessment mean to you?

Best practices documents  ACRL Best Practices Initiative: Characteristics of Programs of Information Literacy that Illustrate Best Practices, 2003 Describes program evaluation and student outcomes as areas of assessment. Describes program evaluation and student outcomes as areas of assessment. Lists some examples of the types of assessment methods. Lists some examples of the types of assessment methods.  Standards for Libraries in Higher Education Covers all areas, but has relevant sections detailing assessment in libraries as well as what instruction programs ought to include. Covers all areas, but has relevant sections detailing assessment in libraries as well as what instruction programs ought to include.

9 Principles of Good Practice for Assessing Student Learning 9 Principles of Good Practice for Assessing Student Learning  From American Association of Higher Education  Overarching principles to guide assessment  6. Assessment fosters wider improvement when representatives from across the educational community are involved.  8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.

What to assess?  Student learning outcomes Refers to the measurement of what a student actually learned in the course. Pre-tests and post-tests Annotated bibliographies Classroom assessment techniques (Angelo and Cross)

What to assess? continued  Student perceptions M easures the perceptions that students have about the course content, the instructor’s knowledge and teaching ability, and what the students feel they learned from the course. Student evaluation forms Surveys Focus groups

What to assess? continued  Progress towards goals and objectives Refers to what some know as ‘inputs and outputs.’ Statistics on number/duration of courses, instruction sessions Number of hits to a web site Student follow-up appointments, drop in visits, s

Small steps can take you far  Consult your institution’s/library’s mission/values/goals documents. Is IL articulated in campus/library documents? Is IL articulated in campus/library documents?  Do you already keep basic statistics on students reached? Time spent preparing classes? This can be valuable evidence. This can be valuable evidence.  What kind of in-class activities do you already do? Collecting student work can be assessment. Collecting student work can be assessment.

Activity 2.  Identify one “small step” that you would like to take next.

Assessment in the CSUSM  CSUs are teaching-centered, primarily undergraduate campuses.  CSUSM has appx 6000 FTEs.  Librarians are faculty and work with instructors in virtually every department.

Guided notes: measuring student learning  Students get a basic outline which they fill in during the class.  Students have a ‘product’ for their time with you.  You can route back to instructor to illustrate what students learned. Psychology 230: Research Workshop Research Tools Discussed PsycInfo: database of psych stuff, some full-text, use thes. Buros’ MMY: a source of psych measures, see psych research guide for e-version Using PsycINFO Thesaurus: identify right words, breaking up=relationship termination Boolean: stress and college, teen* or adolescen* *gets plurals ‘Empirical studies’ and ‘literature reviews’ ES have: experiments, empirical=observed, critique 3 for assignment! LR have: analyzes what is known about topic. Compilation of ES. Questions I have: What are Yvonne’s office hours??

Research chart: measuring student learning

Student evaluation forms: measuring student perception Question Category Strong DisagreeDisagreeNeutralAgree Strong AgreeTOTALMean score cat 1 Respect for students cat 2 Facilitated learning cat 3 Appropriate to Assignment/ class cat 4 Affective/ attitudinal changes

Assessment or Evaluation?  What do we mean by assessment anyway? Assessment focuses on the end result Assessment focuses on the end result Evaluation focuses on the overall process Evaluation focuses on the overall process

Students Instruction/ Pedagogy Instruction Program Learning Outcomes Assessment Student-centered Learning

Assessment and Collaboration  Teaching faculty want to know…  Assessment in the classroom Use it as outreach to the teaching faculty Use it as outreach to the teaching faculty Share results and use as dialog for assignment design Share results and use as dialog for assignment design Build an integrated information literacy program within a distinct discipline Build an integrated information literacy program within a distinct discipline Build tailored first-year experience programs Build tailored first-year experience programs

Activity 3.  Who are your allies? Name three prospective collaborators.

Trinity University and IL  Information Literacy is relatively new  Challenges to change  2400 FTE, primarily liberal arts campus

Trinity’s Answer  Build an IL program and assess along the way What is an IL program? What is an IL program? Assessment was internally and externally focused Assessment was internally and externally focused  The end result is a program, fully integrated into the curriculum

Where do I begin?  How do you know what you are doing/building is effective?  What is most important to administrators?  How can you help those who may be working against you? Example: FYS and Lunches at Trinity (other people’s assessment data!) Example: FYS and Lunches at Trinity (other people’s assessment data!)

First Step  Where are you now? State of Information Literacy and Instruction State of Information Literacy and Instruction  What do you want to do next? Short term and long term goals. Know the difference. Short term and long term goals. Know the difference.  Go for the most impact!

Activity 4.  What is your goal? What do you want to achieve next?

Low Effort/High Impact  Creating programmatic assessment takes time and work Begin with low effort/high impact items Begin with low effort/high impact items Start tracking hours spent preparing for classStart tracking hours spent preparing for class Start tracking NEW classes coming in for instructionStart tracking NEW classes coming in for instruction

Best Practices as a Guideline for Assessment  Great tool to use as a guideline Collaboration/Integration Goal of Document Collaboration/Integration Goal of Document Check off what you’ve done Check off what you’ve done Check what you can do now (short term) Check what you can do now (short term) Which characteristics will be your guide for the future? Which characteristics will be your guide for the future?

Activity 5.  Is your “small step” LE/HI?  If not, do you have an alternative idea?

Creating a Plan  From your long term and short term goals Internal focus Internal focus (teaching, hours, # of classes)(teaching, hours, # of classes) External focus External focus (who are the active faculty, creating outcomes and standards, long-term studies)(who are the active faculty, creating outcomes and standards, long-term studies)  Focus on pieces: It WILL come together!

Activity 6.  One short term goal:  One long term goal:

Outcomes for your Plan  Example: Assessing Program Effectiveness (External) Outcome: Encourage broad faculty participation in instruction in order to ensure information literacy reaches across the curriculum. Outcome: Encourage broad faculty participation in instruction in order to ensure information literacy reaches across the curriculum. Assessment: Collect data concerning overall number of classes and departments using instruction, looking for increases in numbers of classes or departmental participation. Assessment: Collect data concerning overall number of classes and departments using instruction, looking for increases in numbers of classes or departmental participation.

Outcomes Continued  Assessing Program Effectiveness (Internal) Outcome: Hold workshop for librarians about integrating outcomes assessment in classes in order to help them be more effective in their teaching. Outcome: Hold workshop for librarians about integrating outcomes assessment in classes in order to help them be more effective in their teaching. Assessment: Have each librarian create three outcomes for every instruction session…Collect data from their assessments. Assessment: Have each librarian create three outcomes for every instruction session…Collect data from their assessments.

Activity 7.  Write one outcome using either your long term or short term goal. What do you want to know? Why? What do you want to know? Why? Don’t forget “in order to”Don’t forget “in order to”

Program Assessment Can:  Improve Teaching  Improve Student Learning  Provide Validation  Provide Validation

Five Questions  What do you want the instruction program to be able to do?  What factors does the program demonstrate or exhibit if they are meeting the outcomes?  How will you gather your data or evidence?  How will you determine if you have achieved your outcomes?  How will the discussion or evaluation of the data occur? Who will be involved?

Questions Translate Into…  Outcome What does an effective program do? What does an effective program do?  Indicator What will be happening? What will be happening?  Assessment How will the data be collected? How will the data be collected?  Criteria How will you know? How will you know? Immersion 2004

Other Opportunities  SAILS Standardized Assessment of Information Literacy Skills Standardized Assessment of Information Literacy Skills  ETS ICT Project  LibQual+

An Ongoing Culture of Assessment “Assessment is an ongoing process aimed at understanding and improving student learning.” Tom Angelo, AAHE Bulletin, 1996 ACRL Immersion 2004

Assessment Plans: Individual or Group Work

Session Evaluation

Thank You !!