From Shoeboxes to Mashups: ERMs and Decision Support Tim Jewell University of Washington NISO Conference: “Understanding the Data Around Us” November 2d Magnolia Hotel, Dallas

The “Shoebox”

“Shoebox” Data Elements Journal Call Number Priority (1, 2 or 3) Price Location Duplicated? Selector Fund Faculty consulted Other Selectors Consulted

Collection Development Context: Financial Chronic pressure on University and Library budgets Increased importance of accountability and evidence to support decision-making Market E-journal “packages” Licensing issues Cancellation limits Long-term access/archiving Publisher consolidation Organizational: the end of rugged individualism?

Collection Development Support Applications “Serials Cancellation” projects “Databases” Journals Print Electronic Analysis of E-journal packages Proposals/Offers Periodic Re-analysis Refocus Budgeting/Spending All these tending to merge

UW Serials Review Toolkit

“Local” Journal Metrics Cost/payment Usage Print Electronic COUNTER Cost per Use ILL use Faculty data Publications Citations Requests Priority judgments

“Global” Journal Metrics Retail/List Price ISI Impact Factor Emerging Journal Cost-effectiveness Eigenfactor Usage Factor MESUR “150-200 Metrics” Article: “The new metrics of scholarly authority” (Michael Jensen, Chronicle of Higher Education, June 15, 2007)

But Don’t Forget Fund Code, Selector Package Information Publisher Cancellation Limits Print vs. Online Archiving/continuing access rights Portico, LOCKSS coverage? Subscribed vs. Nonsubscribed titles

What reports or “views” do we want, with what comparisons? Different for types of resources Databases Journals Journal to Journal Comparisons Package Comparisons E-books Different levels of analysis “Selector” or Department “Fund Group” or Broad Area Campus University Consortium Trends?

From Usage Data Management to Serials Decision Support: Questions and Design Criteria What data/metrics do we want? Everything available/ “I want it all?” How do we want to display/report it? In all sorts of ways/ ”My way” Where should data be organized and stored for optimal use? Where best support for #1and #2 is available/”I want it easy”

The Metrics Landscape: Products and Services “Serials Service” Vendors Serials Solutions Ulrich’s Serials Analysis 360 Resource Manager SWETS Scholarly Stats Thomson Scientific Journal Citation Reports Journal Use Reports ILS Vendors ERMs from Ex Libris, III, SIRSI/Dynix

Some quick takes on “metrics services” ISI Journal Citation Reports Impact Factor well-established But open to question, reports not integrated with other data Ulrich’s Serials Analysis System Provides current pricing and subject categories But only provides links to Impact Factor Serials Solutions 360 Resource Manager Provides peer comparisons/benchmarks Not yet clear what else is provided Journal Cost-Effectiveness Relative Cost Index an interesting idea, But sustainable? EigenFactor Interesting alternative using “page rank” approach, But validated, sustainable? ISI Journal Citation Reports Impact Factor a well-established metric, reasonably good reporting capability, but no connection to other data and limited flexibility EigenFactor Free/low cost, promising alternative metric, interesting reporting capability, no integration with other data Ulrich’s Serials Analysis System Good overlap analysis, etc., but have to upload holdings data, only get links to Impact Factor, etc. Serials Solutions 360 Resource Manager May provide “benchmarking” comparisons to peer libraries

Scholarly Stats Features Issues Relatively low-cost way to aggregate usage data Well-designed reports Most- and least-used journals Analysis by package/platform SUSHI support Issues No tie to payment information Price paid Fund codes No tie to other metrics Comprehensiveness and pricing

ISI Journal Use Reports Features Includes multiple metrics: Impact Factor COUNTER Usage data Publications Citation Flexible reporting “Profiles” Budget/fund codes? Reports can be exported(?) Issues Must upload serials holdings data Lacks some local information: Prices Paid License Information Too tied to ISI journal coverage? Lacks other metrics Pricing model? FEATURES JUR is the only resource that brings together this unique combination of data to ensure fully-informed decision-making: Institutional COUNTER-compliant journal usage reports from publishers and vendors -- show the value of a journal to the patrons Journal citation metrics from JCR -- show the value of the journal to the literature Institutional publication data -- let users analyze journal use and institutional publishing patters, creating profiles by department, section, and budget code Article level data from Web of Science? -- reveal citation activity at the researcher and departmental level BENEFITS Until now, librarians who wanted to analyze journal use had to collect and assemble data from many different vendors. The data couldn't be easily collected, combined and analyzed. JUR makes the process easy and fast. It combines usage data for all journals in a collection, along with citation data from Journal Citation Reports?* and Web of Science. JUR enables librarians and administrators to use the integrated and customized data to: Acquire a better understanding of departmental needs Analyze usage to spot trends by citations, usage or both Spot collection gaps or research trends Support curriculum as well as collection development And more *Subscription to JCR is required.

The DLF Electronic Resource Management Initiative, Phase I

Trial Assess need/budget License terms Price Evaluate Order, Register Catalog Digital Registry Proxy server Gateway WebBridge Usage stats Review alternatives Review problems User feedback Investigate Evaluate Monitor Provide Access Contact info Provide Support Administer Inform users Track problems Troubleshoot Manage changes Provide Training Payment, manage financials Setup contacts Customize interface Holdings management Set up usage statistics

The DLF Electronic Resource Management Initiative, Phase II License Expression Professional Training in License Term Mapping (ARL/DLF collaboration) ERMI/ONIX E-Resource Usage Statistics Protocol for automated delivery (SUSHI) ILS/ERM Interoperability

Integrating usage statistics into a collection assessment tool via Innovative’s ERM module NWIUG 2007 Created by Hana Levay levay@u.washington.edu and Diane Carroll carroldi@wsu.edu

ERM Beta Can now accept SUSHI feeds UW using ScholarlyStats to gather Easy to generate COUNTER-style reports within ERM Brings in data from order records to calculate cost per use

Project Muse Statistics Export Average cost per title = $26121/354 titles =$73.79

A Close-Up View of Statistics Export CPU per title = Cost per title / Use for title CPU: Africa Today = $73.79/156 = $0.47

Project Muse Order Record

A Close-Up View: Project Muse

WSU Serials Decision Database Database designed for selectors to make a decision to add, cancel or renew a title The complete serials decision database contains subscribed and unsubscribed titles This example contains only subscribed titles so the same set of titles can be compared to results obtained only from Millennium Handout 1: WSU Serials Decision Database Row 1 – Source of the information Row 2 – Label for data found in the columns

What sources were included for assessment of subscribed titles? Millennium Title, ISSN, Order record #, location, vendor, fund code, cost Access provider Ejournal coverage, Archival access and license restrictions Subscription agent Title, format purchased, subscription period, publisher, ISSN Scholarly Stats and publisher website Ejournal use statistics Calculated Cost per use

What sources included subscribed and unsubscribed titles? ILLiad Interlibrary loans borrowed (ILLiad), ISSN Web of Science Number of authored papers Number of papers referenced by institution’s authors Calculated information on the priority of the titles

Priority of the journal High Use (1) Medium Use (2) Low Use (3) Number of articles by WSU authors in the journal from 2004 to 2006 >10 5-10 2-4 Number of articles cited by WSU authors from 2004 to 2006 >40 10-40 5-9 Full text/PDF downloads >250 40-250 <40 Interlibrary loan requests over the last 12 months 6-9 5 These are OR statements

Steps in Creating WSU Reports Create list of current subscriptions using Checkin Record Create list of order records Merge Checkin and Order records Create list for Resource and License Record data Add usage statistics and cost per use

Create List challenges Can not do one create list that will pull data from the Bib, Order, Checkin, Resource and License records. Must do this report in stages and merge data to get information on each subscribed title on one line on a spreadsheet.

Making Millennium more friendly for journal collection assessment Greater integration of data Need to be able to download journal, provider, licensing restrictions, coverage and use data in a create list Expanded use of import function introduced with ERM Can already load coverage information Need the ability to upload data such as Web of Science use data, priorities, other local use information and link it to the checkin records

III Millennium ERM Features Issues Includes fund codes Can include license terms Includes price paid Includes subscription period SUSHI Support (automated intake) Generates cost per use Issues Complex reporting steps Limited flexibility Cost per use not always meaningful Support for alternate metrics?

Revised Serials Collection Assessment Design Criteria “I want most of it”, but especially . . . Prices paid Local “budget structure” data COUNTER Stats Alternate metrics “I want it my way” Customizable reporting By “fund code” By “platform” or “package” Peer library comparisons/benchmarking Sort, recombine, display as needed “I want it easy” Or at minimal time investment/ inconvenience . . .

SUSHI Generalized? User Consolidated Statistical Reports Reports Interface Data Consolidation Consolidated Data Import (manual) SUSHI (automatic) ERM knowledge base Web interface Web interface Web interface SUSHI SUSHI SUSHI Usage Metric A Metric B Usage Metric C Metric D