1. Data-Informed Ebook Discovery: An Analytical Approach to the User Experience
Therese - Good morning, (Introductions)
I am Therese Triumph and I am up here with my colleagues Angela Bardeen and Libby Wilcher
This morning we will be discussing our findings from a project that the E-book Working Group has been involved with since last year. This project was the work of many librarians including Megan Kilb and Kristina Spurgin. In addition much of the manual checking was done by our student workers.
This project looks at the discoverability of several ebook collections through the library's discovery service and the catalog.
The team identified a dozen e-book packages across disciplines and analyzed the titles to determine their availability via our Summon Discovery Service, the extent of indexing, the success rate with full-text linking and whether titles can be found in the catalog.
The team also evaluated vendor-provided catalog records, identified data present in the records, investigated how the records interacted with Syndetic Solutions data, and compared our inventory of catalog records against vendor-provided title lists.
University of North Carolina at Chapel Hill Libraries
Angela Bardeen, Therese Triumph & Libby Wilcher
2017 NC Serials Conference

2. Background UNC has ~5.2 million full text e-books
Several hundred e-book collections
We are focusing on the collections today
UNC has 5.2 million full text e-books
Several hundred e-book collections
We are focusing on the collections today

3. In the literature Cataloging demand Subscription – changing title list
Low usage, low quality books included in packages
Missing titles that had been purchased across the disciplines
Changes in list = more cataloger time
Helen Georgas writes in her article "the case of the disappearing ebook...” that they (nor  us) have the staff (I.e. catalogers) to check ebook packages for each title and MARC record – even a "spot check" (like we have done for this study) is time consuming."
Georgas also mentions that title lists given at purchase may not be accurate or may change over time
And similar to "serials" in a big package a purchase of an ebook collection includes low use/and or low quality ebooks.
William Waters book "Ebooks in Academic Libraries" (2013) states that use of vendor-supplied MARC records is not always straightforward…the number of titles don’t always match up with the amount of MARC records given.
Any changes from the publishers with the title list either adding or subtracting can then require more catalogers’ work.

4. Purpose
Better understand availability and discoverability of E-book titles 
Improve processes/flow
Identify problematic packages/ publishers 
In order to
Improve our users' experiences 
Identify areas requiring further investigation
Communicate issues to vendors
Advocate for improved access (when necessary)
Story ~ The Science liaison librarians are e-preferred when it comes to purchasing books so the discoverability of ebooks is very important to UNC libraries.
One of the reasons we embarked on this project is that faculty and students were contacting us when they were not able to find a specific ebook either in our discovery service or the catalog. Many times librarians checked the online publishers websites…the ebooks were there…and available…so why weren’t they showing up for the students.
Was this just anecdotal ? Or did we have a problem? Were we getting the ebooks we paid for?
So began the “e-book collection discovery project”!
Better understand availability and discoverability of E-book titles 
Improve processes/flow – how much follow-up do we need to do from the purchasing of the collection to availability in the Summon/catalog
Identify problematic packages/ publishers - we wanted to have information to talk to the publishers and if needed get some changes made.
And all this knowledge we are gaining is so that we can
Improve our users' experiences 
Identify areas requiring further investigation
Communicate issues to vendors
Advocate for improved access (when necessary)

5. The Project Examined discovery/access through Summon
Examined access through catalog
Evaluated MARC Records and Syndetics
As Therese mentioned we undertook a pilot project to answer these questions

6. E-book Collections Social Sciences I Social Sciences II Sciences I
Sciences III
Health Sciences I 
Humanities I 
Humanities and Social Sciences I
Multidisciplinary I 
Random sample of titles generated from select e-book packages

7. Methods: Summon and Catalog Discovery
Conducted searches in Summon for a sample of titles from select packages
Is content from this e-book collection appearing in Summon search results? 
Is content from this e-book collection indexed in Summon at the full-text level?
How does Summon try to pass you through to the full-text for content in this collection? Do you arrive directly at the full text?
Is content from this e-book collection linking at the book-level page or a chapter-level page?
Is it accessible through the catalog?
Angela

8. Humanities and Social Sciences I – Summon and Catalog
Title list 9787 
9787 catalog records  
looked at a sample of 200 – revised 143
Content appearing in Summon search results  
         ~70%  (100/143)  /  ~30% not appearing 
Content indexed in Summon at the full-text level 
         ~37% (53/143) of total sample  indexed in Summon at full text-level 
         ~53% (53/100) that appear in Summon are indexed at the FT level 
Full-text links for the content in this collection working 
       95% linked to full text 
       However... of the titles available in Summon only 16% had direct links to the full text  
Content appearing in Catalog search results  
       100 % (143/143)

9. Sciences I- Summon and Catalog
Science I  
Title list 1316 
1319 catalog records  
looked at a sample of 200 
Content appearing in Summon search results  
        ~80% 160/200 /  ~20% not appearing 
Content indexed in Summon at the full-text level 
     ~79% (159/200)  indexed in Summon at full text-level 
Full-text links for the content in this collection working 
         157 out of 160 / 2% not linking 
Content appearing in Catalog search results  
       ~95%  191/200 (191/200) / ~5% not appearing 

10. Summon and Catalog Overall

11. Methods: MARC Records and Syndetics
Evaluated Catalog Records, including:
Data present in MARC records (whether there are classification numbers, LCSH, etc.)
Whether catalog records get enhanced with Syndetics Solutions data in the public catalog
Comparison of # of catalog records in our system against # of titles in entitlement data from content providers\
Libby

12. Methods: Automating Evaluation
Step 1. Create a tab-delimited text file indicating the collections to be evaluated. Each line must have the "Host Item" title from the 773 field, followed by a TAB and a 'shorthand' name for the collection.
The title used in these 773 fields are locally established (and controlled) collection titles. We establish them at the level of granularity at which we need to be able to a) manage/reconcile MARC catalog records; and b) report out stats to various partners.

13. Methods: Automating Evaluation
Step 2. Run Script #1 using the text file as input to obtain a list of unsuppressed bib records included in the catalog for each specified collection. The resulting list is saved to a comma separated values (CSV) file.
ILS Database now called Sierra DNA

14. Methods: Automating Evaluation
Step 3. Run Script #2 using the CSV file as input to pull subject headings data from the ILS for each bib record. The resulting list is saved to a new CSV file.

15. Methods: Automating Evaluation
Step 4. Run Script #3 using the CSV file from Script #2 as input. It queries the Endeca Web Services API and Syndetics Solutions for XML to parse for data points. Boolean values and counts are appended to each row of bib records and saves as a new CSV file.

16. Methods: Automating Evaluation
Step 4. Run Script #3 using the CSV file from Script #2 as input. It queries the Endeca Web Services API and Syndetics Solutions for XML to parse for data points. Boolean values and counts are appended to each row of bib records and saves as a new CSV file.

17. Methods: Automating Evaluation
Step 4. Run Script #3 using the CSV file from Script #2 as input. It queries the Endeca Web Services API and Syndetics Solutions for XML to parse for data points. Boolean values and counts are appended to each row of bib records and saves as a new CSV file.

18. Methods: Automating Evaluation
Step 4. Run Script #3 using the CSV file from Script #2 as input. It queries the Endeca Web Services API and Syndetics Solutions for XML to parse for data points. Boolean values and counts are appended to each row of bib records and saves as a new CSV file.

19. Methods: Automating Evaluation
Step 4. Run Script #3 using the CSV file from Script #2 as input. It queries the Endeca Web Services API and Syndetics Solutions for XML to parse for data points. Boolean values and counts are appended to each row of bib records and saves as a new CSV file.
The reason we took this multi-step approach is that we couldn’t answer some of the specific questions that our colleagues had using only data in the Endeca public catalog. To distinguish between LCSH, MeSH, and other (including uncontrolled) subject headings, and to only count LC classification/call numbers, we needed the granularity afforded by the MARC standard. This granularity is lost (at least to some extent) when the data is mapped into the Endeca catalog’s more general data model.

20. Humanities and Social Sciences I-MARC

21. Sciences I- MARC

22. MARC Overall

23. Conclusion Summary Next Steps Questions for audience
Workflow – communications between subject librarians (public services) and ESM 
Communicating across public services
Next Steps
Communicate to Vendors
Further study of packages
Questions for audience
Assessment of collections 
What is important
Summary
We have learned that some of the ebook packages require more work than others and we will need to keep that in mind when we make further purchases.
It has lead to an Improve in the Workflow from the subject librarians purchasing the ebook packages and Electronic Recourse librarians
We realize the need to follow up with our public services librarians to keep them informed on what we learned
We also want to keep our users in mind when we make our purchases of these ebook collections
Next Steps
Communicate to Vendors
Further study of packages
Questions for audience
Have you done any Assessment of your ebook collections? 
What is important to you and your users?
This project looks at the discoverability of several ebook collections through the library's discovery service and the catalog.
The team identified a dozen e-book packages across disciplines and analyzed the titles to determine their availability via our Summon Discovery Service, the extent of indexing, and the success rate with full-text linking.
The team also evaluated vendor-provided catalog records, identified data present in the records, investigated how the records interacted with Syndetic Solutions data, and compared our inventory of catalog records against vendor-provided title lists.

24. Thank you Angela Bardeen, bardeen@email.unc.edu
Therese Triumph,
Libby Wilcher,
Thank you to Megan Kilb and Kristina Spurgin for their work on this project.