HAN Conference © History Data Service The History Data Service : Promoting Good Practice and Standards of Scholarship Cressida Chappell Head of Service Hamish James Collections Manager

HAN Conference © History Data Service Who Are We? Founded in 1993 National service funded by –Joint Information Systems Committee –Arts and Humanities Research Board Based in the UK Data Archive at the University of Essex Part of the Arts and Humanities Data Service Team of historians and IT specialists

HAN Conference © History Data Service What Do We Do? (1) Mission Statement The History Data Service collects, preserves, and promotes the use of digital resources, which result from or support historical research, learning and teaching.

HAN Conference © History Data Service What Do We Do? (2) Provide advice and training about creating, describing, using and preserving historical digital resources Collect and preserve historical digital resources Provide access to a wide-ranging collection of historical digital resources including resources held by other organisations Develop online data and metadata delivery systems to enhance access to this collection Establish thematic special collections, and enrich and enhance selected data collections Promote standards and best practice in the creation, description, use and preservation of historical digital resources

HAN Conference © History Data Service Digitisation: Good Practice A ‘good’ digital resource is one that is flexible and that can support many different uses –Preservation - an accurate representation of the source material. –Research - codes, indexes, categorisation. –Access - many different users in different settings. Who will use the resource? How will they use it? –Are there discipline specific methods of describing, categorising, and coding information that should be used? –Can the resource be searched at appropriate levels of detail? The digital resource should be well documented The digital resource should adhere to standards and avoid reliance on unusual features of software or hardware

HAN Conference © History Data Service Ensuring Preservation The time and resources invested in the creation of digital resources can easily be placed in jeopardy because hardware and software become obsolete, and magnetic media degrade Long-term preservation is essential if this investment is to be safeguarded Digital resources need to be preserved and migrated through changing technologies in order that they will continue to be accessible in the future However, the extent to which a digital resource can be preserved without significant information loss is largely dependent on decisions taken during the data creation process

HAN Conference © History Data Service Supporting Re-use Many historical digital resources potentially have significant and long-term value to the research and teaching community The time and resources invested in their creation can only be fully realised if they are suitable for re-use both by the data creator and by others Such suitability, however, is again largely dependent on decisions taken during the data creation process

HAN Conference © History Data Service Ensuring Long Term Viability To remain usable digital resources must have suitable hardware and software and be well documented Digital technology develops so quickly that resources can become unusable within a few years if they are not actively preserved The digital resource should be hardware and software independent to ensure that it remains usable. Use neutral data formats. These are formats that are widely accepted, are not controlled by a single organisation, and have a publicly available definition Use commonly accepted formats in preference to specialist formats. Avoid relying on special features of particular software or hardware that cannot be adequately replicated in other settings

HAN Conference © History Data Service Why Is Good Documentation Important? The maintenance of comprehensive documentation detailing the data creation process and the steps taken involves a significant but profitable investment of time and resources It is more effective if documentation is generated during rather than after a data creation project Such an approach will result in a better-quality data collection, as well as better-quality documentation, because the maintenance of proper documentation demands consistency and attention to detail The process of documenting a data creation project can also have the benefit of helping to refine research questions and it can be a vital aid to communication in larger projects Good documentation is crucial to a data collection’s long-term vitality Without it the resource will not be suitable for future use and its provenance will be lost Proper documentation contributes substantially to a data collection's scholarly value

HAN Conference © History Data Service What Is Good Documentation? At a minimum, documentation should provide information about a digital resource’s –Contents –Provenance Who created the digital resource and why? How was the digital resource created? Which sources were used to create the data collection? –Structure It needs to be sufficiently detailed to allow the data creator to use the resource in the future, when the data creation process has started to fade from memory It also needs to be comprehensive enough to enable others to explore the resource fully, and detailed enough to allow someone who has not been involved in the data creation process to understand the data collection and the process by which it was created.

HAN Conference © History Data Service Elements Of A Digital Resource The environment of a digital resource often receives the most attention, but it is the users and digital objects that are most important –Hardware and software selection should be based on the needs of the users and the types of digital objects to be use Users Knowledge Experience Culture Environment Hardware Software (OS) (Network) Digital Objects Binary Data Relationships

HAN Conference © History Data Service Digitisation Process Digitisation: Any means of capturing the information content of a non-digital source in binary coded form The digitisation process involves separating the information content of the source from the medium which carries that information The process of digitisation creates a representation of the original source, it does not create a duplicate of the original source Information may be enhanced or damaged, discarded or added during the digitisation process Digitisation forces choices about which aspects of the source will be captured in the digital representation of the source Information content can be anything about a source. Consider a page in a book; the information content includes: the text on the page the size and shape of the characters on the page the layout of text on the page the chemical composition of the paper the number of the page

HAN Conference © History Data Service Source - Digitisation - Resource The ‘input channels’ of digitisation (keyboard, scanner etc.) are narrow and can only capture a small proportion of the source’s information content identify aspects of source to digitise chose digitisation method chose digital format

HAN Conference © History Data Service Source Analysis Simplify the source –ignore unwanted information –exclude certain types of information –Define a sub-set of the remaining information content –select information directly from the source or define a set of summarised information based on the source Model the information content sub-set –break information content into discrete elements of information –describe the characteristics of each information element –describe how information elements relate to each other Successful source analysis requires a good understanding of the source and of the purpose of the digital resource

HAN Conference © History Data Service Data Models Data models are abstract ways of structuring information File formats are specific ways of implementing a data format –A Word97 document and an WordPerfect 8.0 document are different file formats that both implement similar data models of ‘a document’ The information content of a source can usually be represented by a number of different data models –the source and the intended purpose of the digital resource should determine the most appropriate data model to use –once a data model has been selected, it should be possible to store data in a number of file formats as required To be useful, digital data must be: –organised according to an appropriate data model –stored in a file format that can represent the data model –used in an application that understands the data model and file format in the desired way (try opening an HTML file in a web browser and an ordinary text editor, notice the difference)

HAN Conference © History Data Service Overlapping Formats Often one data model can be represented using another data model –the elements of a mark-up document can be stored as fields in a database –a database can be stored using a mark-up DTD –SVG (Scalable Vector Graphics) is an XML DTD for storing vector based images There are usually many file formats that can be used to represent a single data model, selecting the right data model is much more important then selecting a particular file format Choice of file format will follow from choice of software that suits your requirements

HAN Conference © History Data Service Digitisation: A Balancing Act Successful digitisation involves several trade-offs: –Amount and detail vs. time and cost of digitisation –Complexity of the digital resource vs. ease of use and understanding –Flexibility of the digital resource vs. suitability for a specific use –Feasibility of digitisation with current technology vs. future possibilities for digitisation Choices of what to digitise and how to digitise a source should be guided by a firm understanding of the source and the intended purpose of the digital resource –Do not exceed the limits of available support (financial, technical, equipment, labour) –Always try to preserve the information content of the source –Keep information that tracks the origin and history of the digital resource with the digital resource