1. Creating textual and visual resources

2. Overview of this session Types of manuscripts Types of printed documents Types of visual resources Methods of capture Some examples How to guidance for rekeying and OCR OCR exercise

3. Types of documents: largely textual Manuscripts Books Periodicals Newspapers Grey literature Documentary surrogates: microfilm etc

4. Types of manuscripts Huge range Several centuries BC to the present day 2500 years of materials Written on many different materials papyrus, animal skins, lead tablets, stone, paper, etc Many different languages font/script issues Music Images

5. Characteristics of manuscripts Unique even if there are many copies, they will all be different May be fragile May have bindings Will need special handling Will need specialist equipment

6. What do you want from mss? Capture once for all time? Complete record? Covers bindings blank pages glosses erasures palimpsests

7. What do you want from mss? Complete colour fidelity? Enhancements at capture stage? UV Infrared Record of minute details? What is significant? e.g. with parchment, do you want to see the pore marks?

8. Copyright Corpus Christi College, Cambridge

9. DIAMM Mss www.diamm.ac.uk

10. Handling Conservation practice in human handling Special stands and cradles Light levels Dust-free environment Temperature and humidity to be rigorously controlled Heat is a real danger

11. A note on metadata Visual images are very difficult to create intellectual metadata for Describing or categorizing images in words is difficult Can use full text descriptions Or thesauri or classification systems Art and Architecture Thesaurus (AAT), Visual Resources Association (VRA), ICONCLASS

12. Printed items: largely textual Books Periodicals Newspapers Grey literature Documentary surrogates: microfilm etc Miscellaneous materials including musical scores ephemera advertisements cartoons posters, etc

13. Diamond sutra, worlds earliest printed book, AD 868

14. Goettingen British Library TexasKeio, Japan

15. News of the World, June 1851News of the World, June 1918

16. Penny Illustrated, October 1861 Weekly Dispatch, June 1856

20. Chopin First Edition

22. Trade card, 18th C.

23. Advertisement for booksellers`

24. Imperial War Museum Spanish Civil War Collection: Poster

25. Reel of microfilm

26. Microfiche

27. Characteristics of documents: books Printed books can date back to the 1470s Gutenberg Bible Early English Books Online may need to be treated more like manuscript materials

28. Characteristics of documents: books Almost certain to be bound Is it possible to disbind? Will they be discarded after scanning? May be printed on unstable media Different sizes May have image-rich content Likely to have language/font/character set issues

29. Characteristics of documents: books Varied internal structures depending on topic and type recipe books art history books childrens books Some common structural features Table of Contents, index, bibliography, chapters, footnotes, pages

30. Characteristics of documents: periodicals Will have different structures according to type, but structure likely to be regular within a title comics popular magazines trade magazines academic journals Some common features … articles, images, advertisements, columns, diagrams, footnotes, bibliography, Table of Contents, etc

31. Characteristics of newspapers Large in format Prolific in output Designed as essentially ephemeral Fragile Complex and multipart Change over time Many different types of content: text, images, advertisements

32. Characteristics of newspapers Difficult to index Difficult to store because of bulk and volume Inherently unstable paper weak and brittle, deteriorates rapidly Great interest to researchers Difficult to extract information from

33. Characteristics of documents: grey literature Catch-all category Includes many different kinds of un-published or semi-published materials reports personal papers conference papers newsletters Difficult to characterize A collection may have many different formats, periods, conditions Difficult to catalogue

34. Characteristics of documents: microform A good long-term storage alternative but a poor substitute for reading lose sense of the physicality of the original linear small format tiring to read impossible to search harder to scan (by eye) than the originals

35. Visual artefacts Huge category of visual materials paintings and drawings fabrics art objects technical drawings maps 3-D objects

36. Croyland, Lincolnshire, John Sell Cotman

37. Bacchanal, Cecily Brown

38. Suffragette Banner, Womens Library

39. A Cosy Couple, Amanda Francis

40. Technical drawing design

41. 1930 map locating Painswick village inside folded printed change of address flier for Pyllis Barron and Dorothy Larcher

42. Spellman Music Covers Collection, Reading University

43. Types of photographs Wide range Prints Negatives (acetate, nitrate, glass plates, paper) Transparencies Slides Daguerreotypes and other special formats Digital originals

44. Dressmaking class 1936: preparation for dress parade

45. John Ruskin's Daguerreotype of a group of windows in the façade of the Casa degli Zane, Venice

46. Glass plate negative

47. 35mm B&W negative

48. Digital original

49. Characteristics of photographs Multiple versions possible Negative and the print and copy photography Colour and monotones – fidelity is vital May be fragile, dirty and even combustible May be flexible or rigid, mounted or in strips (e.g. albums, slides, negative strips) Will probably need special handling Will benefit from specialist equipment

50. Handling Every single interaction with a fragile original can compromise it Many of these may be hundreds of years old … … we want them to last for hundreds more years So special handling is crucial

51. Handling Conservation practice in human handling Heat levels – most critical due to build up Light levels Dust-free environment

52. Image Quality How do we know if it is good enough? Visual sharpness Laterally reversed images Dirt Skew Image completeness Guidance available from the RLG Publications by Franziska Frey (Rochester Institute of Technology)

53. Capture methods Depends on the nature of the original material Depends upon available resources Depends on the purpose of the digitization? A forensic record of the original? to externalise the textual content? audience delivery options information goals

54. Capture methods Scanning book scanner flat bed scanner drum feed scanner microfilm scanner Sunrise microfilm scanner Zeutschel OS10000 A1 Bookscanner

55. Image Quality How do we know if it is good enough? Visual sharpness Laterally reversed images Dirt Skew Image completeness Guidance available from the RLG Publications by Franziska Frey (Rochester Institute of Technology)

57. Case Study: CVMA Corpus Vitrearum Medii Aevi – medieval stained glass The content is only renderable from photographs of the subject. Comprehensive database with high levels of descriptive metadata. Further additions will include maps and church plans linked to window images.

58. Specifications of CVMA Digitisation Source: 35mm slides, medium & large format transparencies, photographic prints Scanning dpi: 35mm – 2,700 dpi Medium format – 1,200 dpi Large format – 1,000 dpi Print – 600 dpi All 24-bit RGB colour File formats: TIFF master (uncompressed) JPEG for web Courtesy of CVMA Project, Courtauld Institute of Art

59. Case Study: Shetland Isles Museum Glass plate collection - >80,000 items In-house scanning using flatbed scanners 600 dpi, 8-bit greyscale specification Delivered on the web with the option to buy content. Online images are thus relatively small.

62. Digitization issues Preparation of materials Assessing the collection Organization of data resources

63. Scanning into electronic formats Preparation of materials Assess the collection STOP POINT 1

64. Scanning into electronic formats STOP: 2 OCR for indexing STOP: 3 OCR/Rekeying for end user presentation STOP: 5 SGML/XML STOP: 4 Metadata

65. Digitization issues In every case you have to: assess the nature of the collection prepare the collection for digitization Decide how to organize the end information resource

66. Creating full text If digital images are scanned with no added value digital microfilm is the result This has many advantages for access But much more is possible...

67. Creating full text There are a number of ways to create manipulable text rekeying OCR (Optical Character Recognition) with correction uncorrected OCR These will be discussed in detail later

68. Rekeying Most costly option But less expensive than it was! Very accurate if done well Can be used instead of providing a digital image Or attached to a digital image as a means of searching

69. Case study: Old Bailey Court Session Papers Largest single digital resource on non-elite peoples. 58,000 pages = >250 million characters rekeyed Rekeying is the most effective way to address the content of the originals XML markup the only way to deliver the content in a structured way

70. www.oldbaileyonline.org

74. OCR Pattern recognition algorithms which can convert images of alphanumeric characters into ASCII code Been around since the 1970s KDEM (Kurtzweil Data Entry Machine), hardware and software very expensive so specialist bureaux offered it as a service move to desktop OCR in the mid-late 1990s See handout for OCR guidance

75. OCR accuracy This depends on the quality of the image being processed 99%+ is possible To what degree is accuracy important? this can depend on the intended use of the captured text

76. Case study: Refugee Studies Centre Library Grey literature collection Earliest documents from the 1960s so copyright a critical issue Making content widely available the key aim Forensic fidelity unimportant Need to capture a large volume

77. Methods: Can do destructive scanning Digitization outsourced Initially uncorrected OCR also outsourced Later, use Olive Software Active Paper Archive OCR for searching, page image for viewing Case study: Refugee Studies Centre Library

78. www.forcedmigration.org

84. How to guidance: Rekeying Single rekeying one pass with checks. Generally 99.5% accurate Double rekeying keyed twice, differences checked. Generally 99.99% accurate Rekeyers should key what they see not what they think! Assume they know nothing Textual layout and structure provide clues for rekeyers Detail all variations, special characters, spellings that you can

85. How to guidance: Rekeying Example From the hand out Note the detail the variations quality assurance

86. How to guidance: OCR Handout Note the need to understand the nature of the document nature of original nature of printing language uniformity text alignment complexity of alignment lines, graphics and pictures handwriting

87. OCR Quiz Look at the examples on screen Make a note of any features you think might affect OCR accuracy Have a guess of what you think the accuracy in % terms might be