Wartburg College What do First-Year Students Know About Information Literacy and When Do They Know It? IA/ACRL Panel, April 21, 2006

We Make Assumptions About First-Year Students u Is there data to support our perceptions on their level of preparation for college-level research? u Faculty give students’ Information Literacy skills too much or too little credit.

Need for Baseline Assessment of Entering Students’ Skills u Decided to administer a pre-test. u Helps librarians and faculty make informed decisions about info literacy lessons. u Gained a snapshot of what incoming WB students know (or don’t?!) about information literacy.

Practicalities u When to administer the test? u Multiple choice format for scoring 290 tests. u Faculty cooperation required. u Assistance with test design & measurement. u Administrative support via ILAC program. u Started pre-testing in Fall 2000.

Two Parts to the Wartburg Information Literacy Pre-test u Demographic snapshot. u Information Literacy Competency Standards for Higher Education questions.

Demographic Snapshot of Entering Class u Computer Background: –2000: Do they own a computer? –2005: How is time spent on the computer? u Use of sources: –Experts / Books / Magazines / Journals / Internet. u Information-related activities: –Papers / Speeches / Documentation / Research / Evaluation u Required use of bibliographies and footnotes.

New Demographic Questions u Added 2005 u Male and female percentages. u Did you attend an Iowa high school? u If ‘yes,’ did you receive instruction from the school library media specialist?

Information Literacy Competency Standards Quiz u Establish a baseline for future assessment. u Questions based on ACRL’s Information Literacy Competency Standards for Higher Education. u Matched selected standards to questions about info we felt incoming students should know.

Pre-test Demographic Results u Students more wired than we assumed. –93% had access to a computer at home in u Searching for school or personal info outranks downloading or gaming in u Research choices favor online resources. –73% start with the Internet. (rated 5 + 4) –30% use scholarly journals last. (rated 1 + 2)

Demographic Results 2 u Students DO: –write research papers, –give researched speeches, –include reference lists, and –search for personal information. u Yet, 1/3 rd of them do NOT reject a piece of information because the source was unreliable.

Iowa-Specific Results u 2/3rds of the students attended HS in Iowa. u Of those, 38% received instruction from the school library media specialist (2005). u 61% said they did not. –EN111 = 32% yes, 68% no. –EN112 = 41% yes, 58% no. u Did not ask if they “had” a media specialist in their Iowa HS.

Competency Standard Results u EN112 have a better grasp of research skills and terms (more AP exposure, etc.?). u English teachers will be depressed that ½ could not pick out a thesis statement. (22) u Where to start info search? (23) –1/4 th think any “book” is good. –1/4 th admit to starting on the Internet.

Competency Results 2 u Good news on Boolean logic: 73% can recognize recommended search terms from choices. (25) u Narrowing terms (from too many hits) is a problem for 1/3 rd.(24) –Out of the 2/3rds who get it, EN112 = 71%; EN111 = 49%. u Database selection for college research is problematic. (26) –Less than ½ chose the scholarly one from descriptions.

Competency Results 3 u 85% understand the concept of “help” screens (doesn’t measure if they USE them). (28) u Confusion about “scholarly” sources.(29) –Only 46% chose a research journal (EN112 again!) –41% chose a.com website ( u 83% know duplicating a CD is a copyright violation. (36) u See question 38. OUCH! –Is it plagiarism to use the outline of a paper found on the web…?

Competency Results 4 u Citation recognition--poor.(40-41; 44-45) –Less than half could identify a journal article. –Only 21% could identify a portion of a book. u 90% know that a book is found in an online catalog! (42) u Confusion about finding print journal articles: –1/4th want to look in the online catalog. (39 & 43) –1/2 would look for a library-owned journal in the books or online.

Crunching the Numbers u It was a statistically significant sample. –270 tests returned for a first-year class of 491 (55%)—2000. –295 tests in 2005—518 first-year students (57%). u Significant differences between EN 111 and EN 112. –Divided by ACT English section score.

What We Learned About Test Design u 2000: 3 of 15 questions had poor discrimination –Few got the correct answer regardless of ability. –Poor correlation between item and test score vs. simple ignorance. 2005: No significant discrimination problems on pre-test. u Used the resources of Wartburg’s Assessment Center.

Where Pre-testing Led Our Information Literacy Program u Quizlet for second year to compare results. u Ongoing modification to keep Information Literacy relevant to curriculum u Final assessments in departmental capstone classes.

Summary u Work in partnership with faculty, not just a “librarian” thing. –Curriculum mapping. –Inservice to faculty on ILAC. –Use assessment tools to shape ILAC program design. –Assessment data for HLC evaluation and Gen. Ed review. u See Vogel Library Information Literacy web pages (

My Personal Observations u Pre-test results reveal students do not approach information-seeking “strategically” on their own. u State Library EBSCOhost database buy isn’t reaching all students. u In a Google world, students don’t understand the differences between using it or other vetted resources. u Expectations for HS students depend on the information literacy of their teachers. u Teaching librarians are the conduit for raising expectations. u This type of dialogue is crucial!!!