1. Library Services within the Total User Experience: Discovery and Digital Scholarship William Mischo Grainger Engineering Library Information Center University of Illinois at Urbana-Champaign National Institute of Technology Silchar January 21, 2015 Library Services within the Total User Experience

2. Core Library Services Activities that support the campus research and instructional experience –Discovery gateway systems that offer “smart” search assistance that mimics the actions of a trained reference librarian –Dynamic digital environments for active engagement with students and faculty The role of the academic library; the importance of establishing key campus and external partnerships Library Services within the Total User Experience

3. Role of the Library Core values: library’s changing roles in collection development, discovery and delivery, and digital scholarship Library’s role in end-to-end lifecycle of knowledge creation and knowledge management Concept of local collection much less meaningful Re-imagineering the library

4. Discovery and Delivery Libraries at a Crossroads Roger Schonfeld (Ithaka): “this is a good moment for academic libraries to step back to reconfirm (or reconsider) their vision for discovery, to ensure that their visions connect with information-seeking practices and preferences…” Exploring three technologies: –Federated search –Web-Scale Discovery Systems (WSDS) –Hybrid Bento-style approach

5. Full Library Discovery Overarching issue is how to optimize access within the complex information landscape we live in: online catalogs, publisher full-text, institutional repositories, A&I services, local and multi-institutional digital content, learning resources, etc. Full Library Discovery – Lorcan Dempsey Not only collections but services – subject area specialists, library webpages, smart search assistance, pathfinders, *personalization, etc.

6. Search & Discovery Gateway function and full-text is King Federated search: broadcast and merge over remote information targets Web-Scale Discovery Services: single aggregated index harvested from content holders Hybrid bento-style federated search where the screen display is partitioned into tile areas Numerous libraries investigating Bento

7. Search & Discovery University of Illinois at Urbana-Champaign (UIUC) Library discovery efforts: – Easy Search federated search and recommender system – WSDS: Primo Central, 2011-2014; WorldCat Discovery, 2013-; Ebsco Discovery Service API – Bento-style search and display

8. Areas of Focus: --Search Assistance --Information Content Integration --Full Library Discovery OPACs -UIUC VuFind -State VuFind -Voyager -WorldCat Discovery- Easy Search API Direct Links (Best Bets) Libguide Matches Spelling Suggestions Subject Area Identification Author Suggests Limit/Broaden Suggests Added Title Search links Journal AtoZ Matches DOI matches Journal & Article Locator Discovery and Delivery Environment Total User Experience Focus Full-Text Linking: -- CrossRef and DOI -- Local Link resolver -- A&I via OpenURL local Link Resolver -- Google Scholar and Link Resolver Users “I am looking for a specific item” “I need articles, books about X” Federated Search (broadcast), Web-Scale Discovery (aggregation) Hybrid Bento-style Search/Display Subject Area Identification -Open Worldcat Class Label -Recommended targets -Departmental library -Librarian subject specialist Scopus API Ebsco EDS API Dark Target Searches Google Discovery Services APIs Link Resolver A to Z Search LibraryWebsite Search A&I article databases APIs Direct Links Transaction Logs Information target HTTP Connectors Digital Content Management Repository Course Management Systems Multiple Publisher Repositories e-journals, e-books National/International Digital Platforms DPLA, Hathi, LOCKKS, CrossRef, DataCite (datasets) Web-Scale Discovery Systems

9. University of Illinois Library Gateway Revised Gateway portal introduced in September 2007, powered by Easy Search federated search system: o Recommender and discovery system o Employs context-specific and adaptive Search Assistance mechanisms o Helps with search strategy modification and navigation o Takes users into native interfaces at point of completed search o Writes out custom transaction logs

10. Evidence-Based Services We believe in user testing and incorporating user needs in systems User behavior studies and custom transaction log data and user survey input Online Catalog user studies have been contradictory Increasing usage of Gateway as access mechanism for known-item search 10

11. UIUC Library Gateway

13. UIUC Library – Bento Display

15. Digital Scholarship Centers in Libraries Support growing & widespread e-research and digital scholarship Bring together expensive technologies for use by all campus departments Bring together expertise to serve multiple campus departments Support graduate and undergraduate students who desire to create digital projects but do not have access to tools and expertise in their department

16. Library/Campus Partnership Library as place, Place as service Collaboration, e-scholarship, innovation, experiment with new forms of scholarship and publishing Knowledge visualization, data visualization, data analytics, next-gen design, user interaction Lifecycle of scholarly content. Helping the institution become a learning organization

17. Digital Scholarship Center facilities: UCLA

18. Visualization facility at Hunt Library NCSU

19. Brown University Library

21. Georgia State University Library

22. Grainger Engineering Library Information Center: Illinois

23. Data Management Issues U.S. federal grant agencies require Data Management Plans: NSF, NIH, DOE The 2013 White House OSTP mandate and White House memo Need to demonstrate importance to campus of a campus-wide infrastructure for data management Multiple campus stakeholders: library, academic computing center, research offices, colleges, departments

24. UIUC Library Initiative factors There is a growing number of disciplinary and government repositories We wanted to lead the efforts for a campus- wide program Wanted to show upcoming compliance issues, dangers of current practices Approached offices of: Vice-Chancellor for Research and Provost

25. NSF Data Management Plans Data Management Plans (DMPs): required element in NSF proposals, January 2011 July 2011: the Library, working with the campus Office of Sponsored Programs and Research Administration (OSPRA) began an analysis of DMPs in submitted NSF grant proposals Currently, looked at 1,600 grants with 1,260 in the analysis.

26. Reasons for DMPs Make research data available and sharable Use of data for verification of results and reproducibility of research Federal agency can show significant return on investment to justify funding Important to know storage venues and mechanisms for sharing and reuse Use of local templates and local campus resources such as institutional repository

27. Reasons for Analysis What storage venues and mechanisms for sharing and reuse are being used? Are the PI’s using local templates and local campus resources such as the IDEALS? Is the campus in danger of losing data? Of losing grants because of compliance issues?

28. Analysis Analysis attempts to characterize and classify DMPs into categories DMPs assigned multiple categories 1,260 DMPs from July 2011 to November 2013

29. Data Venue and Risk Data Location Submitted ProposalsFunded Proposals Risk of Loss/Corruption/ Breach n=1260 n=298 PI Server/Website 64%High61%High Departmental Server/Website 11.2%Medium to High7%Medium to High Campus-Wide Resource 52.9% Low 45% Low IDEALS (Institutional Repos.) 21.9%19.8% NCSA 4.3%16.4% Disciplinary Repository/Cloud 25.8%Medium to Low21.4%Medium to Low Remote Repository 28%Medium to High22.8%Medium to High Optical Disk, Specimens, Analog 19.4%Out of Scope11%Out of Scope

30. Analysis Any differences in storage venue or technologies between the unfunded proposals and the funded proposals? Any differences between the proposals from the first year and the more current proposals? Other differences in proposal categories between funded and unfunded 734 active NSF awards, $861.8 million

31. Implications and Conclusions 1.No significant differences between funded/unfunded proposals in storage venues - no advantage in specifying IDEALS or Disciplinary 2.More recent proposals suggest IDEALS and disciplinary repositories included at a significantly higher level Questions remain: what is the role of the library? The campus? The subject discipline?

32. Linking of Data and Article Literature The standards are CrossRef DOIs and DataCite DOIs: we need bi-directional links Role of publishers in data storage and linking Linking is of interest to publishers, authors, grant agancies Potential for a Linked Open Data application Potential for data repository recommender service

33. Linking of Data and Article Literature Example is search of Scopus to retrieve an article by an author or on a topic The article DOI (CrossRef DOI) is extracted and a link to DataCite using this CrossRef DOI is constructed The matches at DataCite are displayed along with their links from the DataCite DOI to the dataset results landing page/producer page

34. Follow-on Vice-Chancellor for research and Provost have funded a campus-wide Research Data Service - RDS Assist in compliance with federal agencies and assist researchers with DMPs, DOIs, storage mechanisms both short-term and preservation Develop important partnerships with campus units and national entities

35. Illinois Environmental Scan Subscribe to 200+ specialized article and report databases 65K online journals Collection budget of $15 million Online is, in many cases, the exclusive access point Have a legacy online catalog (search over metadata, holdings of books, journals, +) We operate in a distributed, heterogeneous information environment 35

36. Evidence-Based Services We believe in user testing and incorporating user needs in systems User behavior studies and custom transaction log data and user survey input Online Catalog user studies have been contradictory Increasing usage of Gateway as access mechanism for known-item search 36

37. We Do Tools and Services Debate over role of Library in “discovery” Desire for one-stop-shopping We look at custom transaction logs over Gateway and ES Advisory Committee Easy Search recommender and search assistance tool Known-item access important Problem with blended displays and full-text searching 37

38. Today’s Library Gateways Academic libraries are moving from federated search systems to web-scale discovery systems (WDS) with centralized aggregated indexing to Bento-style hybrid search and discovery WDS feature single-entry search boxes with faceted limiting operations’ is faceting useful? As we transition to WDS & Bento, understanding and supporting user search behaviors is important

39. Retrieval Issues Little use of advanced systems; facet use is moderate, about 30% Full-text search retrieval and relevancy ranking issues Large number of known-item cut-and paste searches 39

40. Web-Scale Discovery Features Hosted and in the networked Cloud Aggregated index of licensed publisher-based bibliographic content – journal articles, newspaper articles, book chapters, dissertations, and local OPAC and digital content Single entry search box Blended display format with limiting facet operations & other Web 2.0 features Central aggregated index contrasts with the federated search/broadcast search approach 40

41. Why are Libraries Embracing WDS? Rival to Google; hope to bring back users Extend the OPAC and integrate local digitized content & Institutional Repository Negative experiences with federated search/broadcast search One-stop shopping approach attractive Next Big Thing – keep up with others 41

42. The Big Issues Understanding and supporting user search behaviors is important The effectiveness of full-text search and relevancy ranking is under question Some modules (e.g. OPAC) may be better than central index search/discovery Important to distinguish between discovery features and known-item access mechanisms 42

43. Easy Search Custom Transaction Log We record user actions, system suggestions, search reformulations, and clickthroughs 2010-2011 study of 1.4 million searches and 1.5 million clickthroughs 2014 study of 1 million searches, 1.1 million clickthroughs over 10 month period May 2013 to March 2014 Initially supported by NSF and IMLS OPAC studies ambiguous results; web search engine behaviors different

44. Our Study of Custom Transaction Log Data Conducted an evidence-based study of user behavior in the Easy Search University of Illinois Gateway Looked at 1.4 million searches and 1.5 million clickthroughs performed by users in Fall 2010 and Spring 2011 semesters; now updating Our transaction log studies were supported by National Science Foundation and Institute of Museum and Library Studies

45. UIUC Library Gateway

47. UIUC Library – Bento Display

49. Examples Competent Jerks, Lovable Fools, and the Formation of Social Networks The virtual supermarket: An innovative research tool to study consumer food purchasing Mothers and fathers : a study of the development and negotiation of parental behaviour Matouschek, Kellis, Serrano, Fersht Nature Kinsella and Phillips 2005 Mowen, A. J., Kyle, G. T Jackowski, M. (2007). Citizen preferences for the corporate sponsorship of public sector park and recreation organizations. Journal of Nonprofit and Public Sector Marketing, 18(2), 93 Hemenway, D. (2010). Why We don't Spend Enough on Public Health. New England Journal of Medicine, 362(18), 1657 university, need for money J. Quant. Spectrosc. Radiat. Transfer.

50. 2014 Transaction Log Analysis Breakdown of searches from the main Gateway: – Easy Search and associated tabs: 902,420 76.5% the gateway Easy Search default tab: (55.1%) – advanced search: 56,170 4.8% – departmental library page searches: 158,750 13.5% – myeasysearch: 18,159 1.5% – search suggestion searches: 42,708 3.6% Total: 1178207 50

51. Search Assistance Features Context-specific suggestions and prompts for: –Spelling changes –Redo as author search –Direct links to frequent or special search terms or Libguides –Links to matching e-journal title(s) –Links to citation linker for full-text retrieval –Limit results to exact phrase, title word(s), or title phrase –Dark background searches displayed if needed

52. 2014 Transaction Log Analysis Gateway searches total: 902,420 – Easy Search tab: 649,320 – Books search tab: 106,375 – Article tab: 76,375 – Journal Title tab: 70,350 2007: 49.4% of all searches were known item 2011: 51.2% of searches in 54% of sessions were known-item 52

53. 2014 Transaction Log Analysis Sample of 974,137 user-entered searches: – average words per query is 5.11 words – This is up from earlier studies: 3.58 words per query in 2008; 3.76 words per query in 2009; 4.33 words per query in 2011 93,642 one word, 158,054 greater than 8 words 49.6% were 3 words or less, 29.4% are 6 words or more longest search was 299 words 53

54. 2014 Transaction Log Analysis 20.98% of all searches in 29.13% of all sessions used a search assistance suggestion or custom added search result link 57.2% of the 489,272 search sessions were single query sessions 6% of the sessions contain 6 or more queries, 1,157 sessions contain more than 20 queries, and 96 search sessions contain more than 50 queries 54

55. Primo Clickthrough Study May-June 2014 Primo clicks 15,068 times between 05-01-2014 and 06-21-2014. Total of 147,326 clickthroughs during that time Looked at a sample of 478 of the 15,068 searches 245 known-item searches (51.8%) and 228 topical searches (48.2%) Of the 245 known-item searches, 159 were judged successful - they brought back the expected results in the Primo first page or as the top item 55

56. Primo Clickthrough Study (2) Of the 159 ‘successful’ searches, all of them were also successful in Scopus, Ebsco, CrossRef, WorldCat Discovery, Web of Knowledge, VuFind, IShare, Google, or Google Scholar searches Removing the Google results left 9 searches that were successful in Primo but not via Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, or WorldCat Discovery. Removing CrossRef and WorldCat Discovery resulted in 35 searches successful in Primo but not successful in Scopus, Ebsco, Web of Science, VuFind, or IShare 56

57. Primo Clickthrough Study (3) Found that CrossRef and WorldCat Discovery good at retrieving successful search results for full citation searches 48 known-item searches that were unsuccessful in Primo but successful in one of the other Easy Search targets - Scopus, Ebsco, Web of Science, CrossRef, VuFind, IShare, WorldCat Discovery, or Google/GS 57

58. Primo Clickthrough Study (4) Evaluated sample against Arizona State’s Summon. In one case, a known-item search that was unsuccessful in Primo yielded a first page match in the AZU Summon. However, there were 50 known-item searches that were successful in our Primo that were unsuccessful (item not on top or first page) in the AZU Summon 58

59. Easy Search and Bento approach user survey 120 survey results that users of Easy Search provided in October 2014 43 were daily users and 53 indicated they were weekly users 99 surveys where users provided us with comments on their Easy Search experiences. In 77, users made either a very favorable (28) or favorable comment about Easy Search 59

60. Easy Search and Bento survey (2) 91 surveys with comments about the Bento approach with 66 of these providing very favorable or favorable comments 40 users with Very favorable/favorable views of ES who also held Very Favorable/Favorable views of the Bento search 17 user surveys where ES was regarded as Very Favorable/Favorable and the Bento version was regarded as Unfavorable or Neutral 60

61. Easy Search and Bento survey (3) 21 surveys where ES was regarded as Unfavorable or Neutral and the Bento version as Very favorable/Favorable Overall, this shows a significant acceptance of the Bento search and display approach Note that one of the Bento options is to re- perform the user search in “classic” Easy Search 61

62. Bento Table Table comparing the feature sets of 17 Bento search/display implementations deployed by university libraries, including our version Number of Bento versions that do not do spell checking and do not provide direct links from the bento display page to full-text None of the others employ the one-click to full- text PDF that we do 62