Linked Open Data Project Assessment Methods and Benchmark Data

Assessment methods User Case Studies 4 series of 5 1-on-1 interviews 2 before enhancement and 2 after 2 for Motley assessment and 2 for Kolb-Proust assessment First pair of assessments have been completed Focus Group (pending late April/early May deployment) Web-site Log Metrics Limited to Motley and Portraits of Actors collections Gathered using Google Analytics Consists of 2 rounds of data collection (before and after)

Benchmark Case Studies Motley Benchmark Case Study Consisted of user-task focused interviews of 5 students in theatre-related fields Compared the Motley Collection to the Havard Theatre Collection Some difference in content Motley – costume and theatre designs Harvard – costume and theatre photographs Kolb-Proust Benchmark Case Study Consisted of user-task focused interviews of 5 students in French literature-related fields Compared the Kolb-Proust Archive to the Bovary Manuscript Archive Kolb-Proust – Proust’s correspondence and Prof. Kolb’s scholarly notes Bovary – Flaubert’s Bovary manuscript (various versions) and manuscript notes

Benchmark Case Studies – General Results Users value clear sorting and filtering options on their search results pages; Users highly value facility for faceted browsing options; Users desire more comprehensive metadata to appear in the short descriptions for each item in the search results lists, especially with regards to: The date the original item was created and The creator of the original item; Users desire more prominent collection homepages to: Differentiate the collection from other collections available through the search interface, Provide a richer explanation of the collection, and Provide clearer search examples.

Benchmark Case Studies – Specific Results Motley Collection Give options for changing number of items on a results page; Present a side-by-side view of item’s image with its metadata; Have all advanced search fields available to streamline search process; Have thumbnail images of each item on the search results pages; Have a “Browse All” option; Add the ability to search within results from a previous search; and Flatten the page hierarchies so that it takes fewer clicks to reach a full record of each item Kolb-Proust Archive Have a more prominent search bar and access to “Advanced Search” options Have a summary of the amount of data that is available along with clearer instructions for how to access it.

Google Analytics Collecting Basic Metrics (summarized): Number of Pages Accessed Pageviews Unique Pageviews Average Time Spent on Page Referrers Additional Metrics (not summarized): Browser/OS used Data Devices (i.e., mobile devices used) User locations

Benchmark Traffic Statistics for Motley & PoA Combined number of items: 7,579 (37.6% of total items in Illinois’ digital collections) Total pages accessed: 1,294 (13.2% of total for all Illinois’ digital collections) Total pageviews: 2,473 (9.2% of total for all Illinois’ digital collections) Total unique pageviews: 1,944 (9.6% of total for all Illinois’ digital collections) Average time on page: 35.39 seconds (36.5% less time than pages of other collections)

Benchmark Referrer Statistics for Motley & PoA Combined number of items: 7,579 (37.6% of total items in Illinois’ digital collections) Total Referrals: 489 (6.5% of all referral traffic for CONTENTdm sites) Average Percent New Sessions: 81.45% (1.72% more than all sites) Total New Users: 393 (6.3% of all new users from referrals) Average Pages/Session: 3.90 (5% more than other sites) Average Session Duration: 80.85 seconds (43.2% shorter than other sites)

JCDL Reviewer Feedback The paper addresses many relevant aspects for digital library practitioners. The work carried out so far is interesting and the position/vision of the authors is interesting to follow. But the paper does not make any attempt to support its arguments with studies, evaluations, etc. There is a core of current work related to the use of schema.org for vocabularies and the challenges of mapping to these from the various remarks held in the DC records of the archive collections they were working with. I like their reasoning for using schema.org - basically in looking at how to utilize LOD in the archive to improve discoverability rather than as an inventory mechanism, discoverability across collections - i.e. using the LOD approach (along with the schema.org vocabularies) to improve the interconnections between archive collections. A report of this initial mapping work along with the initial UI work over their ContentDM collection would have made a stronger statement as a short paper. It seems Sections 4.3 and 4.4 are more like proposes based on other similar implementations as the examples used are not based on the project. Please clarify in writing. It would be great to have lessons learned like statements for each steps for more general application of the project experience. One final comment, on a particular point. The abstract and section 3.1 indicate that the metadata in use is Dublin Core based, which we found very misleading. The mapping presented in section 3.2 shows data elements with very detailed semantics, not the general semantics that characterizes Dublin Core. This work would more suitable for presentation at the JCDL2017's practitioners day, as a poster.