1. From Shoeboxes to Mashups: ERMs and Decision Support
Tim Jewell
University of Washington
NISO Conference:
“Understanding the Data Around Us”
November 2d
Magnolia Hotel, Dallas

2. The “Shoebox”

3. “Shoebox” Data Elements
Journal
Call Number
Priority (1, 2 or 3)
Price
Location
Duplicated?
Selector
Fund
Faculty consulted
Other Selectors Consulted

4. Collection Development Context:
Financial
Chronic pressure on University and Library budgets
Increased importance of accountability and evidence to support decision-making
Market
E-journal “packages”
Licensing issues
Cancellation limits
Long-term access/archiving
Publisher consolidation
Organizational: the end of rugged individualism?

5. Collection Development Support Applications
“Serials Cancellation” projects
“Databases”
Journals
Print
Electronic
Analysis of E-journal packages
Proposals/Offers
Periodic Re-analysis
Refocus Budgeting/Spending
All these tending to merge

6. UW Serials Review Toolkit

7. “Local” Journal Metrics
Cost/payment
Usage
Print
Electronic
COUNTER
Cost per Use
ILL use
Faculty data
Publications
Citations
Requests
Priority judgments

8. “Global” Journal Metrics
Retail/List Price
ISI
Impact Factor
Emerging
Journal Cost-effectiveness
Eigenfactor
Usage Factor
MESUR
“ Metrics”
Article: “The new metrics of scholarly authority” (Michael Jensen, Chronicle of Higher Education, June 15, 2007)

9. But Don’t Forget Fund Code, Selector Package Information Publisher
Cancellation Limits
Print vs. Online
Archiving/continuing access rights
Portico, LOCKSS coverage?
Subscribed vs. Nonsubscribed titles

10. What reports or “views” do we want, with what comparisons?
Different for types of resources
Databases
Journals
Journal to Journal Comparisons
Package Comparisons
E-books
Different levels of analysis
“Selector” or Department
“Fund Group” or Broad Area
Campus
University
Consortium
Trends?

11. From Usage Data Management to Serials Decision Support: Questions and Design Criteria
What data/metrics do we want?
Everything available/ “I want it all?”
How do we want to display/report it?
In all sorts of ways/ ”My way”
Where should data be organized and stored for optimal use?
Where best support for #1and #2 is available/”I want it easy”

12. The Metrics Landscape: Products and Services
“Serials Service” Vendors
Serials Solutions
Ulrich’s Serials Analysis
360 Resource Manager
SWETS
Scholarly Stats
Thomson Scientific
Journal Citation Reports
Journal Use Reports
ILS Vendors
ERMs from Ex Libris, III, SIRSI/Dynix

13. Some quick takes on “metrics services”
ISI Journal Citation Reports
Impact Factor well-established
But open to question, reports not integrated with other data
Ulrich’s Serials Analysis System
Provides current pricing and subject categories
But only provides links to Impact Factor
Serials Solutions 360 Resource Manager
Provides peer comparisons/benchmarks
Not yet clear what else is provided
Journal Cost-Effectiveness
Relative Cost Index an interesting idea,
But sustainable?
EigenFactor
Interesting alternative using “page rank” approach,
But validated, sustainable?
ISI Journal Citation Reports
Impact Factor a well-established metric, reasonably good reporting capability, but no connection to other data and limited flexibility
EigenFactor
Free/low cost, promising alternative metric, interesting reporting capability, no integration with other data
Ulrich’s Serials Analysis System
Good overlap analysis, etc., but have to upload holdings data, only get links to Impact Factor, etc.
Serials Solutions 360 Resource Manager
May provide “benchmarking” comparisons to peer libraries

14. Scholarly Stats Features Issues
Relatively low-cost way to aggregate usage data
Well-designed reports
Most- and least-used journals
Analysis by package/platform
SUSHI support
Issues
No tie to payment information
Price paid
Fund codes
No tie to other metrics
Comprehensiveness and pricing

15. ISI Journal Use Reports
Features
Includes multiple metrics:
Impact Factor
COUNTER Usage data
Publications
Citation
Flexible reporting
“Profiles”
Budget/fund codes?
Reports can be exported(?)
Issues
Must upload serials holdings data
Lacks some local information:
Prices Paid
License Information
Too tied to ISI journal coverage?
Lacks other metrics
Pricing model?
FEATURES
JUR is the only resource that brings together this unique combination of data to ensure fully-informed decision-making:
Institutional COUNTER-compliant journal usage reports from publishers and vendors -- show the value of a journal to the patrons
Journal citation metrics from JCR -- show the value of the journal to the literature
Institutional publication data -- let users analyze journal use and institutional publishing patters, creating profiles by department, section, and budget code
Article level data from Web of Science? -- reveal citation activity at the researcher and departmental level
BENEFITS
Until now, librarians who wanted to analyze journal use had to collect and assemble data from many different vendors. The data couldn't be easily collected, combined and analyzed.
JUR makes the process easy and fast. It combines usage data for all journals in a collection, along with citation data from Journal Citation Reports?* and Web of Science.
JUR enables librarians and administrators to use the integrated and customized data to:
Acquire a better understanding of departmental needs
Analyze usage to spot trends by citations, usage or both
Spot collection gaps or research trends
Support curriculum as well as collection development
And more
*Subscription to JCR is required.

16. The DLF Electronic Resource Management Initiative, Phase I

17. Trial
Assess need/budget
License terms
Price
Evaluate
Order, Register
Catalog
Digital Registry
Proxy server
Gateway
WebBridge
Usage stats
Review alternatives
Review problems
User feedback
Investigate
Evaluate
Monitor
Provide Access
Contact info
Provide Support
Administer
Inform users
Track problems
Troubleshoot
Manage changes
Provide Training
Payment, manage financials
Setup contacts
Customize interface
Holdings management
Set up usage statistics

18. The DLF Electronic Resource Management Initiative, Phase II
License Expression
Professional Training in License Term Mapping (ARL/DLF collaboration)
ERMI/ONIX
E-Resource Usage Statistics
Protocol for automated delivery (SUSHI)
ILS/ERM Interoperability

19. Integrating usage statistics into a collection assessment tool via Innovative’s ERM module
NWIUG 2007
Created by Hana Levay and Diane Carroll

20. ERM Beta Can now accept SUSHI feeds
UW using ScholarlyStats to gather
Easy to generate COUNTER-style reports within ERM
Brings in data from order records to calculate cost per use

22. Project Muse Statistics Export
Average cost per title = $26121/354 titles =$73.79

23. A Close-Up View of Statistics Export
CPU per title = Cost per title / Use for title
CPU: Africa Today = $73.79/156
= $0.47

24. Project Muse Order Record

25. A Close-Up View: Project Muse

26. WSU Serials Decision Database
Database designed for selectors to make a decision to add, cancel or renew a title
The complete serials decision database contains subscribed and unsubscribed titles
This example contains only subscribed titles so the same set of titles can be compared to results obtained only from Millennium
Handout 1: WSU Serials Decision Database
Row 1 – Source of the information
Row 2 – Label for data found in the columns

27. What sources were included for assessment of subscribed titles?
Millennium
Title, ISSN,
Order record #, location, vendor, fund code, cost
Access provider Ejournal coverage,
Archival access and license restrictions
Subscription agent
Title, format purchased, subscription period, publisher, ISSN
Scholarly Stats and publisher website
Ejournal use statistics
Calculated
Cost per use

28. What sources included subscribed and unsubscribed titles?
ILLiad
Interlibrary loans borrowed (ILLiad), ISSN
Web of Science
Number of authored papers
Number of papers referenced by institution’s authors
Calculated information on the priority of the titles

29. Priority of the journal
High Use (1)
Medium Use (2)
Low Use (3)
Number of articles by WSU authors in the journal from 2004 to 2006
>10
5-10
2-4
Number of articles cited by WSU authors from 2004 to 2006
>40
10-40
5-9
Full text/PDF downloads
>250
40-250
<40
Interlibrary loan requests over the last 12 months
6-9 5
These are OR statements

30. Steps in Creating WSU Reports
Create list of current subscriptions using Checkin Record
Create list of order records
Merge Checkin and Order records
Create list for Resource and License Record data
Add usage statistics and cost per use

31. Create List challenges
Can not do one create list that will pull data from the Bib, Order, Checkin, Resource and License records.
Must do this report in stages and merge data to get information on each subscribed title on one line on a spreadsheet.

32. Making Millennium more friendly for journal collection assessment
Greater integration of data
Need to be able to download journal, provider, licensing restrictions, coverage and use data in a create list
Expanded use of import function introduced with ERM
Can already load coverage information
Need the ability to upload data such as Web of Science use data, priorities, other local use information and link it to the checkin records

33. III Millennium ERM Features Issues Includes fund codes
Can include license terms
Includes price paid
Includes subscription period
SUSHI Support (automated intake)
Generates cost per use
Issues
Complex reporting steps
Limited flexibility
Cost per use not always meaningful
Support for alternate metrics?

34. Revised Serials Collection Assessment Design Criteria
“I want most of it”, but especially . . .
Prices paid
Local “budget structure” data
COUNTER Stats
Alternate metrics
“I want it my way”
Customizable reporting
By “fund code”
By “platform” or “package”
Peer library comparisons/benchmarking
Sort, recombine, display as needed
“I want it easy”
Or at minimal time investment/ inconvenience . . .

35. SUSHI Generalized? User Consolidated Statistical Reports
Reports Interface
Data Consolidation
Consolidated
Data
Import
(manual)
SUSHI
(automatic)
ERM knowledge
base
Web interface
Web interface
Web interface
SUSHI
SUSHI
SUSHI
Usage
Metric A
Metric B
Usage
Metric C
Metric D