1. IOTA: Improving OpenURL Through Analytics Nettie Lagace (@abugseye) NISO Associate Director for Programs CEAL Workshop on Electronic Resources Standards and Best Practices March 25, 2014 Thanks to Oliver Pesch of EBSCO for much of this slide material.

2. Simple example assuming equal element weights ElementDescriptionWeightThis OpenURL ATitleArticle title1 AuLastAuthor’s last name1 DateDate of publication1 ISSN 1 IssueIssue number1 SPageStart page1 TitleJournal Title1 VolumeVolume number1 TOTAL 8 1 1 1 1 1 5 Completeness Score... (Total for This OpenURL) Total Weights 5 / 8 =.625 Completeness Score... (Total for This OpenURL) Total Weights 5 / 8 =.625

3. IOTA Recommended Practice Defines a technique for determining element weights Tested with real link resolvers and real OpenURLs Based on research which looked for a correlation with data elements on the OpenURL and “success” of the OpenURL

4. Select a set of “perfect” OpenURLs – include all key data elements and resolve to full text Perform step-wise regression – Test failure rates for each element by removing that element Use failure rates as basis for weights Use weights to calculate Completeness Scores and to test for correlation between weights and success for larger sample A Statistical Approach to Determining Element Weights

5. Element removed from the OpenURL DescriptionFailure Percentage ATitleArticle title.74% AuLastAuthor’s last name.07% DateDate of publication.4% ISSN ISSN (either online or print ISSN) 22.02% IssueIssue number20.27% SPageStart page33.27% Title Journal Title (either Title or Jtitle).61% VolumeVolume number74.14% Volume is most critical Author’s last name is least important Date is surprisingly low Failure Rates from 1500 OpenURL test sample

6. ElementDescriptionWeight* ATitleArticle title1.87 AuLastAuthor’s last name0.83 DateDate of publication1.61 ISSN ISSN (either online or print ISSN) 3.34 IssueIssue number3.31 SPageStart page3.52 Title Journal Title (either Title or Jtitle) 1.78 VolumeVolume number3.87 *Element weight calculation: log10 (failure-rate-per-10,000 OpenURLs) Calculated Element Weights

7. Correlation Coefficient.80 Tests conducted on sample of 15,000 OpenURLs randomly pulled from IOTA database Results

8. IOTA Online Tool 23.3+ million OpenURLs processed Reporting interface – Analyze data elements (metrics) across vendors or database (Source) – Analyze (Source) for all data elements

10. Analysis of vendors by element (metric)

11. Analysis of elements by vendor

12. How to use IOTA materials The Technical Report provides suggestions for improving OpenURLs The interactive tool offers a means to pin-point irregularities in data provided on OpenURLs The Recommended Practice describes how to create a Completeness Index Completeness Index allows OpenURL quality problems to be quantified

13. Why you should use IOTA Link resolver vendors can implement the Completeness Index in their products to help identify problematic OpenURL sources Librarians can use suggestions and Completeness Index to more effectively communicate quality problems to content providers Content providers can use the online interactive tool to identify problems with the data they provide

14. Thank you! Questions? nlagace@niso.org @abugseye I Want To Go To Magnolia Mountain I Want To Go To Magnolia Mountain by lilomooney is licensed under CC-BY 2.0