1. 1 CS 430: Information Discovery Lecture 7 Automatic Generation of Catalog Records

2. 2 Course Administration Text books are now in the Campus Store. You will need a textbook for Wednesday's reading. Laptop computers - Everybody who has (a) signed up and (b) submitted an assignment should receive an email about collecting a laptop - Laptop surveys will be handed out in class

3. 3 Automatic extraction of catalog data Example: Dublin Core records for web pages Strategies Manual by trained cataloguers [See Lecture 6] - high quality records, but expensive and time consuming Entirely automatic - fast, almost zero cost, but poor quality Automatic followed by human editing - cost and quality depend on the amount of editing Manual collection level record, automatic item level record - moderate quality, moderate cost

4. 4 DC-dot DC-dot is a Dublin Core metadata editor for web pages, created by Andy Powell at UKOLN http://www.ukoln.ac.uk/metadata/dcdot/ DC-dot has two parts: (a) A skeleton Dublin Core record is created automatically from clues in the web page (b) A user interface is provided for cataloguers to edit the record

5. 5

6. 6 Automatic record for CS 430 home page DC-dot applied to http://www.cs.cornell.edu/courses/cs430/2001sp/ continued on next slide

7. 7 Automatic record for CS 430 home page (continued) DC-dot applied to http://www.cs.cornell.edu/courses/cs430/2001sp/

8. 8 Observations on DC-dot applied to CS430 home page DC.Title is a copy of the html field DC.Publisher is the owner of the IP address where the page was stored DC.Subject is a list of headings and noun phrases presented for editing DC.Date is taken from the Last-Modified field in the http header DC.Type and DC.Format are taken from the MIME type of the http response DC.Identifier was supplied by the user as input

9. 9

10. 10 DC-dot applied to http://www.georgewbush.com/ continued on next slide Automatic record for George W. Bush home page

11. 11 DC-dot applied to http://www.georgewbush.com/ Automatic record for George W. Bush home page (continued)

12. 12 Observations on DC-dot applied to George W. Bush home page The home page has several meta tags: [The page has no html ] <META NAME="KEYWORDS" CONTENT="George W. Bush, Bush, George Bush, President, republican, 2000 election and more

13. 13

14. 14

15. 15 Collection-level metadata Several of the most difficult fields to extract automatically are the same across all pages in a web site. Therefore create a collection record manually and combine it with automatic extraction of other fields at item level. For the CS 430 home page, collection-level metadata: See: Jenkins and Inman

16. 16 Collection-level metadata (Example from Lecture 5) Compare: (a) Metadata extracted automatically by DC-dot (b) Collection-level record (c) Combined item-level record (DC-dot plus collection-level) (d) Manual record

17. 17

18. 18 Metadata extracted automatically by DC-dot D.C. Field Qualifier Content title Digital Libraries and the Problem of Purpose subject not included in this slide publisher Corporation for National Research Initiatives date W3CDTF 2000-05-11 type DCMIType Text format text/html format 27718 bytes identifier http://www.dlib.org/dlib/january00/01levy.html

19. 19 Collection-level record D.C. Field Qualifier Content publisher Corporation for National Research Initiatives type article type resource work relation rel-type InSerial relation serial-name D-Lib Magazine relation issn 1082-9873 language English rights Permission is hereby given for the material in D-Lib Magazine to be used for...

20. 20 Combined item-level record (DC-dot plus collection-level) D.C. Field Qualifier Content title Digital Libraries and the Problem of Purpose publisher (*) Corporation for National Research Initiatives date W3CDTF 2000-05-11 type (*) article type resource (*) work type DCMIType Text format text/html format 27718 bytes (*) indicates collection-level metadata continued on next slide

21. 21 Combined item-level record (DC-dot plus collection-level) D.C. Field Qualifier Content relation rel-type (*) InSerial relation serial-name (*) D-Lib Magazine relation issn (*) 1082-9873 language (*) English rights (*) Permission is hereby given for the material in D-Lib Magazine to be used for... identifier http://www.dlib.org/dlib/january00/01levy.html (*) indicates collection-level metadata

22. 22 Manually created record D.C. Field Qualifier Content title Digital Libraries and the Problem of Purpose creator (+) David M. Levy publisher Corporation for National Research Initiatives date publication January 2000 type article type resource work (+) entry that is not in the automatically generated records continued on next slide

23. 23 Manually created record D.C. Field Qualifier Content relation rel-type InSerial relation serial-name D-Lib Magazine relation issn 1082-9873 relation volume (+) 6 relation issue (+) 1 identifier DOI (+) 10.1045/january2000-levy identifier URL http://www.dlib.org/dlib/january00/01levy.html language English rights (+) Copyright (c) David M. Levy (+) entry that is not in the automatically generated records

24. 24 Metadata about subjects (a) Classification (usually manual) Dewey Decimal Classification (DDC) 324.973political web site Library of Congress classification system (LCC) E840.8.G65political web site (b) Subject headings (usually manual) Keywords assigned from controlled vocabulary e.g., Medical Subject Headings (MeSH) Library of Congress subject headings (LCSH) Political campaigns - United States (c) Terms extracted from text (automatic) Automatic indexing [CS 430] Methods from computational linguistics [CS 374/474]

25. 25 Dewey Decimal Classification Main classes: 000 Computers, information, & general reference 100 Philosophy & psychology 200 Religion 300 Social sciences 400 Language 500 Science 600 Technology 700 Arts & recreation 800 Literature 900 History & geography

26. 26 Dewey Decimal Classification Hierarchy, e.g.: 600Technology (Applied sciences) 630Agriculture and related technologies 636Animal husbandry 636.7Dogs 636.8Cats Uses: Shelving collections of physical objects so that items on similar subjects are shelved together Crude subject access Scorpion project (OCLC): Automatic subject recognition and assignment of DDC classes

27. 27 IFLA Model Work A work is the underlying abstraction, e.g., The Iliad The Computer Science departmental web site Beethoven's Fifth Symphony Unix operating system The 1996 U.S. census This is roughly equivalent to the concept of "literary work" used in copyright law.

28. 28 IFLA Model Expression. A work is realized through an expression, e.g., The Illiad has oral expressions and written expressions A musical work has score and performance(s). Software has source code and machine code Many works have only a single expression, e.g. a web page, or a book.

29. 29 IFLA Model Manifestation. A expression is given form in one or more manifestations, e.g., The text of The Iliad has been manifest in numerous manuscripts and printed books. A musical performance can be distributed on CD, or broadcast on television. Software is manifest as files, which may be stored or transmitted in any digital medium.

30. 30 IFLA Model Item. When many copies are made of a manifestation, each is a separate item, e.g., a specific copy of a book computer file [Works, expressions, manifestations and items are explored in CS 502, Computing Methods of Digital Libraries.]

31. 31 Cataloguing Objectives Functions of catalogs: finding collocating (recall and precision) choosing acquiring navigating... among items in a bibliographic universe Compare use cases in software design.

32. 32 Cataloguing Principles User convenience common usage Representation Sufficiency and necessity parsimony Avoid using one device to serve multiple functions (e.g., to disambiguate and order)