1. Evaluation: Preliminary Results from the Server Side Frank A. Settle Elizabeth Blackmer Thomas Whaley The Alsos Digital Library for Nuclear Issues Washington & Lee University Lexington, Va

2. User Server Server log data Who? What? How? Why? Evaluation from the Server Side

3. Preliminary Observations from the Server Side Robots, crawlers, and other automated searches produce over 50% of the visits. While they inflate the number of visitors, they do produce human visits indirectly. Search engines (Google and Yahoo) provide the majority of referrals. Sites that have embedded, directed links to specific bibliographical information provide over 10% of the referrals. Referrals from the NSDL portal as well as other sites that include Alsos in their list constitute a small percentage of referrals. However, users may be using a search engine to follow up on an Alsos link they saw in a list on another site.

4. Month Total visitors (including robots, etc.) Visitors from top referrers per month Prints per month Estimated human visitors per day Visitors from NSDL portal Per month September 2002 1696na41029na March 2003 2619na36444na May 2003 2008na31233na June 2003 1455na26023na August 2003 2489914505435 September 2003 4810147415008918 Overall Usage - September 2002 – September 2003

5. Top Referral Sites – September 2003 Visitors www.google.com 687 search.yahoo.com 289 chemcases.org227 search.msn.com138 www.google.ca 54 aolsearch.aol.com 39 www.atomicarchive.com 38

6. Embedded Link from Atomic Archive to Alsos

7. Preliminary Observations from the Server Side Printing (hardcopy or files) indicates real engagement of the user. However there are also serious users who do not print so this criteria for estimating engaged users is low. Almost all of the visitors listed as referrals from other sites are human users, not robots, crawlers, etc. Based on a typical day (Tuesday, September 30, 2003) a large number of users (~72%) who are referred directly to an annotation by another site, print the annotation and exit Alsos immediately. User access to the six general browse areas (Issues, Warfare, etc.) indicates an equal distribution of interests. This validates the selection of these areas and shows the broad interests of users.

8. No 163643 72% Yes 46 717 28% Total 5 8% 12 20% 43 72% 60 Did user move beyond printing the annotation? HomepageSearch Annotation Entry Point Engaged Users’ Behavior Tuesday, Sept. 30, 2003

9. Distribution of Browse Topics for September 2003 (human visitors)

10. Home Page

11. Browse Issues Area

12. Results for Nuclear Waste

13. Complete Citation with Annotation

14. Processed Information from Server Tabular session displays – summary information for each session Individual session display – tracks the user through a session Suffix statistics – provides information on types and also locations of international users Hit counts for individual references – skewed by robots and multiple hits during a single session

15. Processed Server Log Data

17. Problem – hits are skewed by robots, crawlers, etc.

18. Types of Users Robots, crawlers, etc. Humans who scan topics, site information, etc., but don’t access specific annotations Single reference users referred to Alsos from a search engine or website to a specific annotation. User gets information and exits. Often prints. Human browsers who go to a browse area, obtain a list of references, and then examine specific annotations. May or may not print. Human searchers who enter keywords(s), title, or creator to trigger a search. Most users in this category do not use the advanced search feature.

19. Single reference user from Google

20. Searcher enters keyword “Iran”

21. Browser in People Area

22. Casual Visitor – Scans Information

23. Summary Massive amounts of raw server log data must be transformed into smaller, organized sets of information. This information can provide useful information on who is using the collection and how they use it. This information is useful in improving the utility of the collection and the effectiveness of the user interface. This information is critical in seeking funds to sustain and expand the collection