1ETD 2008_Morgan_The SPECTRa-T Project Extracting and re-using research data from chemistry e-theses: the SPECTRa-T project Peter Morgan SPECTRa-T Project Director Head of Medical and Science Libraries Cambridge University Library

2ETD 2008_Morgan_The SPECTRa-T Project Outline Why SPECTRa-T? Getting started Mining the text –PDFs –.docx Workflows Further thoughts

3ETD 2008_Morgan_The SPECTRa-T Project Why SPECTRa-T?

4ETD 2008_Morgan_The SPECTRa-T Project theses should be semantic and interactive -Peter Murray-Rust (ETD 2007 keynote address)

5ETD 2008_Morgan_The SPECTRa-T Project SPECTRa-T background SPECTRa-T = Submission, Preservation, & Exposure of Chemistry Teaching and Research data from Theses) SPECTRa-T funded by JISC Digital Repositories Programme 1 year project (April 2007 – March 2008) partners: –University of Cambridge (Chemistry + Library) –Imperial College London (Chemistry + ICT) team had previously worked together on SPECTRa

6ETD 2008_Morgan_The SPECTRa-T Project Why SPECTRa-T? research chemists produce experimental data (materials, reactions, properties = recipes) these data are the basis of further research theses are a rich source of data –c.10k chemistry papers p.a. worldwide –a typical thesis contains preparations –20% will be published in research papers –80% are not published

7ETD 2008_Morgan_The SPECTRa-T Project Why SPECTRa-T? text-mining can retrieve these data two basic data types: –Named Chemical Entities (NCEs) (e.g. words/phrases describing properties, procedures, instruments, etc) –Chemical Objects (COs) (e.g. molecules, spectra) our Semantic Web aim: –extract both data types –create RDF triples and chemical objects –link them to enable semantic querying

8ETD 2008_Morgan_The SPECTRa-T Project RDF triples RDF triples are statements containing a subject (resource), predicate (property), and object (value) water boils at 100 degrees Celsius the value of one property can be used as the resource for another

9ETD 2008_Morgan_The SPECTRa-T Project Getting started

10ETD 2008_Morgan_The SPECTRa-T Project Test material 100 PDF chemistry theses from CalTech, MIT, St Andrews & Stirling –some MIT theses OCR-derived (later removed from analysis because of misassigned characters) 20 Word chemistry theses from Cambridge (converted to Office Open XML.docx mark-up format)

11ETD 2008_Morgan_The SPECTRa-T Project Software OSCAR3 (Open Source Chemistry Analysis Routines) as text-mining tool –developed by SciBorg Project (Cambridge) –natural language processing to identify chemical terms –converts human-readable text into XML marked-up content that machines can manipulate –prefers SciXML documents –uses ChEBI Ontology for chemical name recognition

12ETD 2008_Morgan_The SPECTRa-T Project OSCAR3 parsing Highlighted experimental procedures created by OSCAR3

13ETD 2008_Morgan_The SPECTRa-T Project Mining the text

14ETD 2008_Morgan_The SPECTRa-T Project PDF... wraps text in simple high-level elements is optimized for human, not machine, readability produces poor SciXML –line breaks = loss of continuous text and paragraph structures –chemical drawings replaced by text and disconnected lines –loss of subscript and superscript characters –non-printing characters –OCR-derived text produces erroneous character assignment (e.g. i,l,1)

15ETD 2008_Morgan_The SPECTRa-T Project PDF processing SPECTRa-T tools... –removed line-breaks –removed non-printing characters –removed text fragments resulting from broken drawings –used UTF-8 Unicode to preserve Greek characters (lost in ASCII) (note: PDF/A can avoid some but not all such problems) text then converted to SciXML

16ETD 2008_Morgan_The SPECTRa-T Project SciXML from PDF OSCAR retrieves Named Chemical Entities OSCAR creates SAFXML (Standoff Annotated Format XML) output NCE metadata transformed by XSL stylesheets into RDF triples RDF triplestore can be queried BUT... OSCAR cannot identify Chemical Objects

17ETD 2008_Morgan_The SPECTRa-T Project processing Word theses converted to Office Open XML (.docx) using MS Word 2007 XML is converted into rich SciXML SciXML structure enables OSCAR3 to identify Experimental sections and extract Chemical Objects XML converted to CML (Chemical Markup Language) URIs assigned to CO metadata & associated with NCEs CML COs deposited in lightweight data repository RDF triplestore and CO data repository, linked by URIs, can now be queried semantically

18ETD 2008_Morgan_The SPECTRa-T Project Workflows

19ETD 2008_Morgan_The SPECTRa-T Project PDF workflow THESIS Input PDF document (text) SAFXMLSciXMLRDF SPECTRa-T text processing tools OSCAR3 Triplestore (NCEs) XSL stylesheet Processing of PDF e-theses to yield named chemical entities in a queryable RDF Triplestore (Text and lines in red indicate SPECTRa-T tools) PDF flow Query

20ETD 2008_Morgan_The SPECTRa-T Project workflow THESIS Input.docx document (XML markup) SAFXMLSciXMLRDF SPECTRa-T text processing tools Triplestore (NCEs) XSL stylesheet Processing of DOCX e-theses to yield named chemical entities and linked chemical objects in a semantically queryable linked RDF triplestore and data repository (Text and lines in red indicate SPECTRa-T tools) Add URI link Data XML Create URI CML Chemical Objects URI Data Repository (COs) Semantic Query DOCX flow OSCAR3

21ETD 2008_Morgan_The SPECTRa-T Project Further thoughts

22ETD 2008_Morgan_The SPECTRa-T Project Caveats SPECTRa-T a proof-of-concept approach restricted to a few chemistry sub-disciplines investigated only 2 file formats dangerous to generalise too far but our specific observations raise questions about broader implications...

23ETD 2008_Morgan_The SPECTRa-T Project File formats PDF has some value for text-mining born-digital PDF is better than OCR-derived PDF/A will resolve some problems but both still contain broken text and unreliable structure for text-mining –(and most legacy material is still only in PDF) XML better at providing structured documents for text-mining –(and may be good for preservation as well)

24ETD 2008_Morgan_The SPECTRa-T Project Role of institutional repository preservation versus re-usability? should a central IR require both PDF and Word/XML versions of a thesis? which file format(s) should be openly accessible? –cf. UKPMC XML policy for research papers should subject data be held in subject-specific data repositories managed by domain experts? can subject-based departmental repositories co-exist with a central IR? how can librarians and repository managers understand researchers needs?

25ETD 2008_Morgan_The SPECTRa-T Project IPR institutions can best realise the value of their research data assets by encouraging their discovery facts cannot be copyrighted derived data and databases raise complex legal issues ownership and licensing issues need urgent clarification

26ETD 2008_Morgan_The SPECTRa-T Project Fit for purpose? need to be clear why we collect theses are they intended to be fully re-usable? what does this entail for each subject? do librarians understand researchers? do thesis regulations ensure appropriate formats and submission processes? do IPR policies facilitate re-use? in short, are our e-theses fit for purpose?

27ETD 2008_Morgan_The SPECTRa-T Project Thanks... thanks to my colleagues on the Project team –at Cambridge: Jim Downing, Peter Murray-Rust, Diana Stewart, Alan Tonge, Joe Townsend –at Imperial College London Matt Harvey, Henry Rzepa thanks to the Joint Information Systems Committee (JISC) for funding the project (see for Final Report)... and thanks to you for listening!