Danish Legal Deposit on the Internet: Current Solutions and Approaches for the Future ECDL, September 2001 by Birgit N. Henriksen Head of Digitization and Web Department The Royal Library, Denmark

Presentation outline Since 1998 selection based archiving (production) netarchive.dk (new project, multiple archiving strategies, ) Nordic Web Archive (project , access to web archives) Three different initiatives:

The Danish Legal Deposit Law 1697: All printers in royal and ducal lands must deposit 1703: Only printers in Copenhagen have to deposit 1781: All printers in royal and ducal lands must deposit 1902: All printed materials to be deposited 1927: Posters and some types of ephemera excluded 1997: All published works to be deposited

The law from 1997 covers any work published in Denmark regardless of medium “work”: a delimited quantity of information which must be considered a final and independent unit “published”: when … copies of the work have been placed on sale or otherwise distributed to the public

Types of Net Publications Static included (only periodically updated) monographs periodicals Dynamic excluded (continuously updated) Databases homepages

How do we get the material? Download based on notification NOT Harvesting the Danish domain Delivery of works (a collection of files) from the individual publishers

Registration WHO the person in charge of the technical completion of the digital copy HOW by filling out a form at

Registration Form - Monographs

Download - workflow The staff at the Danish Department, The Royal Library determines whether a publication is covered by the law if yes, downloads all files belonging to the work checks downloaded work catalogues and classifies the work in the OPAC (only periodicals) transfers work to archival server (server mirrored nightly to State and University Library, Århus)

Plug-ins

System Environment

Domain names in.dk domain # of sub-domains Registered in.dk May 12’th Registered in.dk June 12’th Represented in archive June 12’th 2001 < 1000

Volume in archived material June 1999June 2000June 2001 # net publications #Representatio ns Repr./net pub #Files – total Files/net pub #Bytes – total1,66 Gbyte12,0 Gbyte18,2 Gbyte

Monographs vs Periodicals Before July 1st 1999 Before July 1st 2000 Before July 1st 2001 #%#%#% Monograph s Periodicals (issues)

Public vs. Private Publishers June 1999June 2000June 2001 #%#%#% Public ,5 Private ,5

Staff resources Man YearsPaid hours per publication Comments 19982,312,75System being developed and set up 19991,91,2Downloading, cataloguing and classifying all publications 20001,30,6Downloading all, cataloguing and classifying periodicals

MimeType Statistics – % of collected files June 1999June 2000June 2001 TEXT/ HTML 56,058,6 %59,3 % Image (GIF, JPEG, PNG) 41,8 %38,4 %37,9 % PDF1,3 % 1,6 %1,7 % Other formats 0,9%1,4 %1,1 %

Three generations using the internet 1st (age 74)2th (age 40)3th (age 10-15) Professionel life (Work/ school related) Professional online periodicals /portals Product information Institutions and organisations Newsgroups Uncritical all available material Entertainment Just surfing aroundAuctions Game services Bizarre websites Newsgroups Events Game services Gimmicks Chat services Searching for information Search engines News Municipal sites Search engines (including cashed web pages) News and media/portals State- and municipal sites Product databases Search engines Special interests Homebanking Stock exchange Homebanking and info related to family economy E-commerce Organisations Seasonal interests Sport clubs (results) Live role play

The modifications from 1902 Brochures and advertisements Catalogues Election campaign material Club/organisation magazines Songs Scouting magazines, church newsletters Maps Portraits Art prints Brouchers Online services like krak.dk Organisation websites Newsletters/minuts on websites Product databases/portals Net Art

Problems related to the notification concept Lack of notification of multiple representations of a publication Lack of notification of new versions

Problems related to technical issues Errors or inconsistencies in the published files Java applets – no solution at the moment Found solutions on previous problems: Documents with java scripts Data behind forms Data behind username/password logins Cookies-based session handling SSL encryption

Gains if harvesting is used Better coverage of Denmark outside the public sphere Updated versions – also for static publications New trends on the net as soon as they appear

Why not only harvesting? Programs and plug-ins are difficult to keep track on Harvesting is not always possible (e.g.. streamed and web casted material) Harvesting may not give a useful result - technical problems (java, interactive sites) - personalised sites Harvesting may produce a collection of documents that have never existed on the net Harvesting may not always give the best format for long-time preservation

Net Art

Home banking

Searching the catalogue

Collections made by harvesting Are not complete – previous slides No robot will never be able to make a ’true’ snapshot – the snapshot contains a mix of documents that have never been published together at the same time – a ’fake’

Archive for Danish Literature from 1. October All full texts are structured in XML on work level The XML is loaded to a database The database performs the web publishing in well-formed HTML on a page level What do we prefer to archive and for what purpose?

Birte Christensen-Dalsgaard: Archive Experience, not Data

Web Archiving Conference, CPH June 2001 Focus: User Expectations to webarchiving in DK Brought together : members of the user community, scholars as well as scientits member from the organisations traditionally in charge of preserving oral and written material members with technical knowledge Proceedings (UK) – netarchive.dk

Web Archiving Conference, CPH June 2001 Sholars & scientist: Archive the dynamic part of the web Focus on archiving the content the context the evidence of use Archivists: Use different archiving approaches New methods for archiving dynamic material Budgets for making snapshots and making selective collections are comparable

Birte Christensen-Dalsgaard: 3 dimensions - duration Real time dialog Published, static Signal lifetime Hourly Update Book-like publications Scientific Journals News-sites Chat

Birte Christensen-Dalssgard: 3 dimensions - Permanent value Transient Persistent Permanent Value What is worth preserving? Quality vs. Representative

Birte Christensen-Dalsgaard: Background - Nature of Information Interactivity StaticDynamic Transient Persistent Permanent Value Real time dialog Published, static Signal lifetime Hourly Update

Birte Christensen-Dalsgaard: Domain of different harvesting methods Interactivity StaticDynamic Transient Persistent Permanent Value Real time dialog Published, static Signal lifetime Legal Deposit, DK Hourly Update Accumulative harvesting Snapshot

Birte Christensen-Dalsgaard: What is missing? Interactivity StaticDynamic Transient Persistent Permanent Value Real time dialog Published, static Signal lifetime Legal Deposit, DK Hourly Update Accumulative harvesting Snapshot

Accumulative Snapshot netarchive.dk (1) Interactivity StaticDynamic Transient Persistent Permanent Value Real time dialog Published, static Signal lifetime Process Test different archival approaches and the subsequent usability of the archived material for research

netarchive.dk (2) Pilot project testing different archival approaches and the subsequent usability of the archived material for research Project partners: State and University Library, Aarhus Centre for Internet Research The Royal Library With economic support from the Danish Electronic Research Library (DEF) Period: August 2001 – July 2002 Case: Danish municipal elections November 2001

netarchive.dk (3) Which materials with What frequency? Collection method? Which software? How should the collection of materials be organized and how should it be stored? How should obsolescence of data formats be dealt with? How should access be given? Budgets for collecting and storing

netarchive.dk (4) Net material covered by netarchive.dk net activities from existing news media (newspapers, radio, TV (both national, regional and local media)) political parties official pages, national and local individual politicians’ personal pages official (county) municipal pages voters’ personal pages »local themes«- pages special interest organisations portals in the broadest sense opinion polling firms public s/ press releases news groups / usenet net-conferences and chat

How do we catch the missing part? Process rather than material – ‘Filming’ the net through a browser Goal: Catch chronological series of displayed WebPages Tools to take into consideration: Business intelligence tools Tools used in usability laboratories …

Nordic Web Archive (NWA) Establish a Danish test archive in order to participate in NWA Software: NEDLIB robot Status 1/9 2001: Archiving started 20/ mio documents 43 GB uncompressed data

Questions?