Google and Beyond: What Sources are Students Really Using? Don MacMillan University of Calgary WILU 2007, Toronto May 17, 2007

WILU 2007 | Google and Beyond | MacMillan2 Introduction University of Calgary (Calgary, Alberta, Canada) University of Calgary (Calgary, Alberta, Canada) 30,000 FT and PT Students 30,000 FT and PT Students Long-standing integration of IL skills into the Biological Sciences Curriculum Long-standing integration of IL skills into the Biological Sciences Curriculum Ongoing assessment of IL program Ongoing assessment of IL program Survey questions generated more information than anticipated Survey questions generated more information than anticipated

WILU 2007 | Google and Beyond | MacMillan3 Why? Promote student reflection on research skills Promote student reflection on research skills What sources were upper-level biology students actually using? What sources were upper-level biology students actually using? Have research habits changed over time-why? Have research habits changed over time-why? Do students use different sources at different stages? Do students use different sources at different stages? Is IL instructional content aligned with student needs? Is IL instructional content aligned with student needs?

WILU 2007 | Google and Beyond | MacMillan4 Who? 25 third and fourth year biological sciences students in CMMB421-Virology Fall third and fourth year biological sciences students in CMMB421-Virology Fall 2006 Most students have had at least one instruction session (BIOL231) Most students have had at least one instruction session (BIOL231) More than 20 students considering graduate studies More than 20 students considering graduate studies

WILU 2007 | Google and Beyond | MacMillan5 How? FAST-Free Assessment Summary Tool FAST-Free Assessment Summary Tool Anonymous online survey tool that summarizes students impressions of a course/instructor Anonymous online survey tool that summarizes students impressions of a course/instructor Numerical and textual data easily gathered and collated Numerical and textual data easily gathered and collated 16 question survey administered at the end of instruction session 16 question survey administered at the end of instruction session

WILU 2007 | Google and Beyond | MacMillan6 Table 1 - Resources used by Virology Undergraduate Students (N=25) ResourceRank Biological Abstracts ??? Google/Scholar Library Catalogue Patents PubMed Web of Science

WILU 2007 | Google and Beyond | MacMillan7 Table 1 - Resources used by Virology Undergraduate Students (N=25) ResourceRank Biological Abstracts 2 Google/Scholar4 Library Catalogue 3 Patents6 PubMed1 Web of Science 5

WILU 2007 | Google and Beyond | MacMillan8 Table 2 - Resources used by Virology Undergraduate Students (N=25) ResourceNumberPercentage Biological Abstracts 2080% Google/Scholar1872% Library Catalogue 1976% Patent Databases 28% PubMed2184% Web of Science 520%

WILU 2007 | Google and Beyond | MacMillan9 Table 3 - Purpose of use for each resource Resource Choosing a topic Exploring a topic Finding specific info Finding background info Will use in future Bio Abs Google (Scholar) Library Catalogue 4851 Patents13 PubMed WOS211

WILU 2007 | Google and Beyond | MacMillan10 How do you decide which source to check first? “Usually I use Biological Abstracts and then PubMed”. “Usually I use Biological Abstracts and then PubMed”. “I was using Biological Abstracts because I was most familiar with it, but now I think PubMed and Web of Science might be better because there are a lot of useful features on these databases that I wasn't aware of”. “I was using Biological Abstracts because I was most familiar with it, but now I think PubMed and Web of Science might be better because there are a lot of useful features on these databases that I wasn't aware of”. “I use general sources such as Google Scholar to browse general topics in the field of interest before searching for specific articles”. “I use general sources such as Google Scholar to browse general topics in the field of interest before searching for specific articles”.

WILU 2007 | Google and Beyond | MacMillan11 Which source do you find most useful for your research purposes? “I think PubMed will become my most useful tool”. “I think PubMed will become my most useful tool”. I find PubMed usually very helpful as well as Google Scholar I find PubMed usually very helpful as well as Google Scholar “I anticipate PubMed, Web of Science and the Patents databases will be very useful “ “I anticipate PubMed, Web of Science and the Patents databases will be very useful “

WILU 2007 | Google and Beyond | MacMillan12 Has the way you do library research changed as you have progressed through your studies? If so, how? “More information has become available and more links. Library sessions have been very informative”. “More information has become available and more links. Library sessions have been very informative”. “I often use multiple search engines/databases now to ensure that I have received a more complete picture on my search topic. Before, I would only use one search engine.” “I often use multiple search engines/databases now to ensure that I have received a more complete picture on my search topic. Before, I would only use one search engine.” “The library seems more organized and also the information sessions have gotten more specific and are quite helpful.” “The library seems more organized and also the information sessions have gotten more specific and are quite helpful.”

WILU 2007 | Google and Beyond | MacMillan13 What factors caused the change? “Formal instruction such as this session changed my research strategies” “Formal instruction such as this session changed my research strategies” “Tutorials for Ecology, Biology and now CMMB (Cell Biology)”. “Tutorials for Ecology, Biology and now CMMB (Cell Biology)”. “Library tutorials have given me a lot of useful information, as well as advice from profs and TA's”. “Library tutorials have given me a lot of useful information, as well as advice from profs and TA's”. “Professors, personal research and playing around, informal and formal instruction”. “Professors, personal research and playing around, informal and formal instruction”.

WILU 2007 | Google and Beyond | MacMillan14 What do you know about library research now that you wished you had known earlier? “I like how the index on PubMed works to break down searches into specific categories”. “I like how the index on PubMed works to break down searches into specific categories”. “The idea of "index" I learned today in Pubmed. Given a very general topic, Pubmed was provided more specific topics for me to explore”. “The idea of "index" I learned today in Pubmed. Given a very general topic, Pubmed was provided more specific topics for me to explore”. “That there is so much available online. I would far rather be at home searching for a topic with a cup of tea at my side than wandering through the aisles of a library for hours”. “That there is so much available online. I would far rather be at home searching for a topic with a cup of tea at my side than wandering through the aisles of a library for hours”.

WILU 2007 | Google and Beyond | MacMillan15 How did you learn about the various sources? “CMMB421 tutorial!”. “CMMB421 tutorial!”. “Library sessions. I would honestly recommend them to anyone who has do to any kind of research”. “Library sessions. I would honestly recommend them to anyone who has do to any kind of research”. “Biology 231, 313”. “Biology 231, 313”. “Blind luck and of course this tutorial”. “Blind luck and of course this tutorial”.

WILU 2007 | Google and Beyond | MacMillan16 Do you plan to pursue an advanced degree? 14 students indicated they would be pursuing advanced studies in the biological or life sciences 14 students indicated they would be pursuing advanced studies in the biological or life sciences 10 students selected medicine or ‘other’ as future areas of study 10 students selected medicine or ‘other’ as future areas of study ‘Future’ medical students used PubMed but there was no clear difference between usage patterns for PubMed among ‘pre-med’ and ‘other students’ ‘Future’ medical students used PubMed but there was no clear difference between usage patterns for PubMed among ‘pre-med’ and ‘other students’

WILU 2007 | Google and Beyond | MacMillan17 Unanticipated Results High number of positive responses regarding the value IL sessions High number of positive responses regarding the value IL sessions Students appreciate review of IL skills from course to course Students appreciate review of IL skills from course to course High use of PubMed High use of PubMed No habitual Google Scholar users No habitual Google Scholar users Many predictive statements about changes to research habits based on IL sessions Many predictive statements about changes to research habits based on IL sessions Students recognize value of their IL skills - useful for marketing purposes Students recognize value of their IL skills - useful for marketing purposes

WILU 2007 | Google and Beyond | MacMillan18 Conclusions Students use a variety of strategies to achieve research goals Students use a variety of strategies to achieve research goals Reinforcement of literacy skills throughout degree program highlights the benefits of integrated IL Reinforcement of literacy skills throughout degree program highlights the benefits of integrated IL Students used different tools for different purposes Students used different tools for different purposes Learned about more resources-use resources more efficiently Learned about more resources-use resources more efficiently Survey results benefited students and librarian Survey results benefited students and librarian

WILU 2007 | Google and Beyond | MacMillan19 Future Directions Incorporate PubMed earlier in IL cycle Incorporate PubMed earlier in IL cycle Google and Google Scholar will be split in next iteration of survey Google and Google Scholar will be split in next iteration of survey Add Scopus (Elsevier) in next survey Add Scopus (Elsevier) in next survey Ask follow-up question later in term to ascertain what new sources were used during the term. Ask follow-up question later in term to ascertain what new sources were used during the term.

WILU 2007 | Google and Beyond | MacMillan20 Related Links Link to WILU 2007 presentation: Link to WILU 2007 presentation: CMMB 421 IL Survey Questions: CMMB 421 IL Survey Questions: CMMB 421 IL Survey Results: CMMB 421 IL Survey Results:

WILU 2007 | Google and Beyond | MacMillan21 Related Links Related Readings: Related Readings: FAST Survey Tool: FAST Survey Tool: CMMB 421 (Virology) IL Web Page:

WILU 2007 | Google and Beyond | MacMillan22

WILU 2007 | Google and Beyond | MacMillan23

WILU 2007 | Google and Beyond | MacMillan24

WILU 2007 | Google and Beyond | MacMillan25

Thank You! Don MacMillan Librarian for Biological Sciences, Physics and Astronomy University of Calgary