1. The Web Archiving Service Tracy Seneca California Digital Library California Digital LibraryNew York UniversityUniversity of North Texas National Digital Information Infrastructure Preservation Program Library of Congress and the Web-at-Risk NDIIPP Project

2. Overview 1.Web archiving: what & why 2.Web-at-Risk grant: scope & purpose 3.Web Archiving Service Sample Screens

3. Web archiving: what & why

4. “Web Archiving”: Assumptions Using automated methods to gather web content Building some kind of collection composed of more than one site Intent on preserving captured content Results are searchable –Public access may not be available

5. How is the material at risk? Vulnerability of –Digital publications –Web publications –Government web publications –Local government web publications

6. The Ephemeral Web

7. Issues Unique to Government and Political Web Documents Publication & notification streams Elections, political change Security vs. freedom of information Local agencies often don’t have the resources to archive their own publications

8. Web-at-Risk grant: scope & purpose

9. Grant Scope Jan 2005 – Jun 2009 Build tools to allow librarians to capture, curate and preserve web-based government and political information. –Create topical and event-based archives –Capture individual sites and documents Assess the impact of these tools on traditional collection development practices. Explore web archiving service sustainability.

10. Project Partners

11. Web-at-Risk Collections

12. Beyond the Grant Support web archiving for the University of California –Enable collaboration across campuses –Enable collaboration between librarians and researchers/faculty

13. Web Archiving Service (WAS) Tangible outcome of grant work Being developed and release over a series of pilot tests Pilot test 5 underway until May 23 2008-2009 develop rights management and public access features

14. WAS Production Early summer 2008, Web Archiving Service goes into ‘limited’ production. –Available 24/7 to the curators who have taken part in the pilot tests so far Expand user community within UC as CDL confirms that WAS infrastructure, user support and training is sufficient.

15. Web Archiving Service Workflow and Sample Screens

16. WAS workflow Project > Site > Capture > Collection Set up a project (usually a topic or event) Define the sites to capture Run single or multiple captures of each site Choose which results to add to a single, searchable collection

18. Capture sites individually

19. Set Frequency

20. Add metadata (or not)

22. Sites can be captured in batches

23. When Capture Finishes

25. Display Results (QA capture effectiveness)

26. Display Results: Overview & Reports

27. Display Results: Full Text Search

28. Display Results

29. Display Results (metadata)

31. Create Collection

32. Build Collection (add entire captures)

33. Build Collection

34. WAS features for analysis It’s impossible to know what a web site ‘contains’ until after you capture it! Tools for understanding where the data comes from and how it has changed.

35. What’s the nature of this content?

36. What new publications are in this capture?

37. Build Collection ( Select files from “Compare” screen)

38. How volatile is this site? (Not yet available)

39. Potential We can now capture the “chit chat” – the popular reaction to historic events, in ways never before possible. How will researchers interact with captured content once it is in an archive? –Visualization –Text analysis What is the potential, beyond simple search and display?

40. Web Archive Visualization Doantam Phan – Stanford University

41. Questions? Web-at-Risk Wiki http://wiki.cdlib.org/WebAtRisk You Tube Video: “Web-at-Risk Collections” http://wiki.cdlib.org/WebAtRisk tracy.seneca@ucop.edu