1 WEB ARCHIVING IN THE BRITISH LIBRARY John Tuck Head of British Collections February 2004

2 BRITISH LIBRARY: CONTEXT  Created by British Library Act  National Library of the United Kingdom.  Origins from  One of world’s greatest research libraries.  160 million collection items.

3 BRITISH LIBRARY: COLLECTION DEVELOPMENT  Building as completely as possible the UK national published archive - current and retrospective gap filling; print and electronic.  Collecting research-level English- language material published world-wide in the humanities, social sciences, STM.  Buying foreign-language material selectively  Material acquired through: legal deposit, voluntary deposit from publishers, purchase, donation, exchange.

4 LEGISLATION  Legal Deposit Libraries Act 2003: enabling legislation.  VDEP: Voluntary Deposit of Electronic Publications.

5 DOMAIN.UK  Six-month experiment to select and capture 100 UK web-sites,  audit change, loss, links, etc.  determine next steps.

6 DOMAIN.UK: Why?  Short-lived nature/changing content of many web-sites.  loss of information.  increasing reference to web-sites in research/scholarship.

7 DOMAIN.UK: Voluntary/Rights Cleared Approach  Voluntary.  Requiring explicit agreement of website publishers to take part in pilot.  No public access.

8 DOMAIN.UK: Selection  Websites of historical or cultural significance.  Cross-section of Dewey Decimal Classification.

9 DOMAIN.UK: Process  selected sites for approval and to check whether already archived.  Measure sites for links, size, change, etc.  Frequency of visits: every three weeks or more in some cases.  Supported by those sites approached.  Report recommended scaling up.

10 BRITISH LIBRARY WEB ARCHIVING PROGRAMME  Building on Domain.uk.  BL to play leading role in collecting UK web presence in partnership with other institutions nationally and internationally.  Selective approach.

11 BRITISH LIBRARY WEB ARCHIVING PROGRAMME contd.  Co-ordinate a snapshot of entire UK web presence at occasional intervals.  Achieve more regular capture of limited and well-defined range of sites.  Sites judged to be research-level, whether in terms of stated intentions of sites themselves or of potential to be primary resources for research.

12 WEB ARCHIVING PROGRAMME  Comprises a series of complementary projects and activities.  Based entirely on voluntary, rights-cleared basis pending secondary legal deposit legislation.  Aims to embed web archiving within the BL's overall collection development policy.  Aims to provide the infrastructure to collect, preserve and make accessible web-site material alongside material in other formats.

13 WEB ARCHIVING PROGRAMME STRANDS  Four main strands:  Definition of collection development policy.  UK Web Archiving Consortium.  International Internet Preservation Consortium.  Internet Archive: incunabula of the internet.

14 COLLECTION DEVELOPMENT  Appointment of Curator, Web Archiving.  Extension of policy defined for Domain.uk.  Sites of national, historical and cultural significance.  Research level now/in the future.

15 UK WEB ARCHIVING CONSORTIUM  Two-year project.  Six partners: BL (lead); National Library of Scotland, National Library of Wales, National Archives, Joint Information Systems Committee, Wellcome Library.  Plan to use PANDAS software developed by National Library of Australia.  Rights to use individual sites to be cleared with rights-holders.

16 UK WEB ARCHIVING CONSORTIUM contd.  Procurement exercise in process to recruit supplier to host service.  Intention to let contract in April 2004 and to be operational in summer  Sites to be made accessible to users.  Each partner to collect up to 500 sites per year, i.e. 6,000 during project.

17 INTERNATIONAL INTERNET PRESERVATION CONSORTIUM  Project involving national libraries.  Led by Bibliotheque Nationale de France.  Also includes BL, Library of Congress, Library and Archives of Canada, Nordic countries, Italy, Australia, Internet Archive.

18 INTERNATIONAL INTERNET PRESERVATION CONSORTIUM contd.  Aims to develop automated web-crawler mechanism.  Open-source tools to search web at regular intervals matching agreed collection development policies.  Working groups in: access tools; content management, deep web, framework, metrics and test-beds, researcher requirements.  Developmental at this stage.

19 INTERNET ARCHIVE  Collecting and saving sites since  Wayback machine.  Legal, technical and procurement issues.

20 SOME CHALLENGES  Defining UK.  Rapid technology change.  Third party rights (not always subject to UK law).  Libel/defamation issues.  Software issues / which platform?  Validity of a snapshot.

21 SOME CHALLENGES contd.  Formats for archiving.  Metadata standards.  Archiving ‘look and feel’.  Authenticity.