A DTD for Qualitative Data: Extending the DDI to Mark-up the Content of Non-numeric Data Libby Bishop and Louise Corti, UK Data Archive, ESDS, University.

Slides:



Advertisements
Similar presentations
OLAC Metadata Steven Bird University of Melbourne / University of Pennsylvania OLAC Workshop 10 December 2002.
Advertisements

DLI Training Nesstar Workshop
Data Documentation Initiative (DDI) Workshop Carol Perry Ernie Boyko April 2005 Kingston Ontario.
1 Metadata Tools for JISC Digitisation Projects of still images and text Ed Fay BOPCRIS, Hartley Library University of Southampton.
HTML I. HTML Hypertext mark-up language. Uses tags to identify elements of a page so that a browser such as Internet explorer can render the page on a.
RSS. March HB/The Data Archive. The RSS Working Group on Data preservation and sharing: standards for documenting data for preservation and secondary.
Putting together a METS profile. Questions to ask when setting down the METS path Should you design your own profile? Should you use someone elses off.
ESDS Qualidata: Qualitative Data Preparation and Use John Southall ESDS 26 November 2003.
New Services for Users Enhanced User Support and Enhanced Access to Data Angela Dale, Head ESDS Government Melanie Wright, Head ESDS Access & Preservation.
Smart Qualitative Data: Methods and Community Tools for Data Mark-Up (SQUAD) Louise Corti UK Data Archive, University of Essex QUADS Demonstrator Workshop.
The Economic and Social Data Service (ESDS) Kevin Schürer ESDS/UKDA ESDS Awareness Day 5 December 2003.
Using Atlas-ti to explore qualitative data Libby Bishop and Louise Corti, UK Data Archive, ESDS, University of Essex IASSIST 2004 workshop.
Depositing Data for Archiving Libby Bishop ESDS Qualidata, University of Essex Changing Families, Changing Food Meeting University of Sheffield 15 March.
Setting the scene: the ESRC and JISC vision for access to qualitative data Louise Corti, ESDS Qualidata Economic and Social Data Service, UK Data Archive.
Smart Qualitative Data: Methods and Community Tools for Data Mark-Up SQUAD Libby Bishop Online Qualitative Data Resources: Best Practice in Metadata Creation.
Using secondary qualitative data in interdisciplinary contexts Libby Bishop ESDS Qualidata, University of Essex Working Across Boundaries: 2 nd NCRM Summer.
Introduction to ESDS Qualidata: Creating and delivering re-usable qualitative data Libby Bishop and Louise Corti ESDS Qualidata RC33 Amsterdam August 2004.
ESDS Qualidata and QUADS Coordination Louise Corti Online Resources Day 15 November 2005, London.
ESDS Qualidata. Qualitative Data Collections Data from National Research Council (ESRC) individual research grant awards Data from ESRC Programme research.
New Services for Data Creators and Providers Louise Corti, Head ESDS Qualidata/ Outreach & Training Alasdair Crockett, ESDS Data Services Manager.
A Common Standard for Data and Metadata: The ESDS Qualidata XML Schema Libby Bishop ESDS Qualidata – UK Data Archive E-Research Workshop Melbourne 27 April.
HAND OUTS DExT Project UK Data Archive September 2007.
Qualitative Data Resources: Qualidata UKDA Libby Bishop ESDS Qualidata, University of Essex Timescapes, University of Leeds St Catherines College, Oxford.
ESDS Qualidata Libby Bishop, ESDS Qualidata Economic and Social Data Service UK Data Archive ESDS Awareness Day Friday 5 December 2003Royal Statistical.
New features for ESDS Qualidata Online Libby Bishop UK Data Archive, University of Essex QUADS Demonstrator Workshop 28 September 2006.
Nesstar, ESDS International and ESDS Qualidata online demonstrations ASLIB visit to the UK Data Archive Wednesday 24 November 2004 Louise Corti, Associate.
Data Exchange and Conversion Utilities and Tools (DExT) Louise Corti, Angad Bhat, Herve LHours UK Data Archive CAQDAS Conference, April 2007.
QUADS Co-ordination Louise Corti QUADS Director, UKDA 28 September 2006.
Secondary analysis of qualitative data: what is it and can it help your research? Libby Bishop ESDS Qualidata, University of Essex Department of Sociology.
The Economic and Social Data Service (ESDS) Karen Dennison UK Data Archive Improving access to government datasets 18 January 2007.
Christina Silver (c) CAQDAS Networking Project Using software to facilitate the analysis of multi-media data © Dr Christina Silver CAQDAS Networking Project.
Metadata and the UK Data Archive CESSDA Expert Seminar Odense September 2008 Margaret Ward Lenin Ageer.
Metadata Management at GESIS-ZA Reiner Mauer GESIS – Data Archive and Data Analysis CESSDA-Expert Seminar Odense, September 11th 2008.
New Directions for ESDS Qualidata: 2003 and beyond Louise Corti, Head ESDS Qualidata Economic and Social Data Service UK Data Archive IASSIST 2003.
DOCUMENT TYPES. Digital Documents Converting documents to an electronic format will preserve those documents, but how would such a process be organized?
Metadata at ICPSR Sanda Ionescu, ICPSR.
A Common Standard for Data and Metadata: The ESDS Qualidata Document Type Definition (DTD) Libby Bishop Online Qualitative Data Resources: Best Practice.
Qualitative Data Preparation and Use Jack Kneeshaw ESDS Psychology Department-U of Essex 4 December 2003.
INTER-UNIVERSITY CONSORTIUM FOR POLITICAL AND SOCIAL RESEARCH Social Science Data and Resources for Researchers Converting Legacy Documentation to DDI:
EAD in A2A Bill Stockting, Senior Editor A2A and EAD Working Group: Central Archives of Historical Records, Warsaw, 26 April 2003.
Arja Kuula: The DDI and Qualitative data IASSIST2001 Amsterdam, May 2001 Finnish Social Science Data Archive.
Smart Qualitative Data: Methods and Community Tools for Data Mark-Up SQUAD Louise Corti IASSIST, Edinburgh May 2005.
Smart Qualitative Data: Methods and Community Tools for Data Mark-Up (SQUAD) Louise Corti and Libby Bishop UK Data Archive, University of Essex IASSIST.
1 Archiving Workflow between a Local Repository and the National Library Archive Experiences from the DiVA Project Eva Müller, Peter Hansson, Uwe Klosa,
Publishing Workflow for InDesign Import/Export of XML
DATA IN Qualitative Data Acquisitions Process Louise Corti ESDS Qualidata, UKDA IASSIST WORKSHOP 27 May 2003.
Archived Qualitative Data: Accessing, Searching and Using Libby Bishop ESDS Qualidata Ph.D. Methods Mini-Course 30 January 2004.
Guest Lecture LIS 656, Spring 2011 Kathryn Lybarger.
Data Exchange Tools (DExT) DExT PROJECTAN OPEN EXCHANGE FORMAT FOR DATA enables long-term preservation and re-use of metadata,
Dr. Kurt Fendt, Comparative Media Studies, MIT MetaMedia An Open Platform for Media Annotation and Sharing Workshop "Online Archives:
DExT PROJECT Louise Corti UK Data Archive University of Essex Colchester, Essex CO4 3SQ Tel: +44 (0) URL:
1 XML as a preservation strategy Experiences with the DiVA document format Eva Müller, Uwe Klosa Electronic Publishing Centre Uppsala University Library,
TEXT ENCODING INITIATIVE (TEI) Inf 384C Block II, Module C.
Chuck Humphrey Data Library Co-ordinator University of Alberta May 16, Capitalising on Metadata Tool development plans IASSIST 2007.
Smart Qualitative Data: Methods and Community Tools for Data Mark-Up (SQUAD) Louise Corti UK Data Archive, University of Essex ASC Conference 29 September.
UK DATA ARCHIVE-NLP COLLABORATION Louise Corti and Claire Grover UK Data Archive University of Essex Colchester, Essex CO4 3SQ
Smart Qualitative Data: Methods and Community Tools for Data Mark-Up SQUAD Libby Bishop Language and Computation Day University of Essex 4 October 2005.
Introduction ESDS Qualidata John Southall ESDS Creating and delivering re-usable qualitative data 24 June 2004.
Using XML to store Descriptive Metadata Richard Murphy Rosarie O’Riordan Central Statistics Office Ireland.
REPORT BACK FROM THE DDI QUALITATIVE WORKING GROUP ……………………………………………………….………………………………
An exercise in preservation and applied technology Making an Electronic Text.
Quads.esds.ac.uk/squad THE PROJECT SMART QUALITATIVE DATA: METHODS AND COMMUNITY TOOLS FOR DATA MARK-UP SQUAD aims to explore methodological and technical.
Exporting WaterML from the Earth System Modeling Framework Xinqi Wang Louisiana State University NCAR SIParCS Program August 4, 2009.
Metadata “Data about data” Describes various aspects of a digital file or group of files Identifies the parts of a digital object and documents their content,
Metadata and Meta tag. What is metadata? What does metadata do? Metadata schemes What is meta tag? Meta tag example Table of Content.
TEI presentation for IS 590 Robert Patrick Waltz July 10 th, 2012.
An Overview of Data-PASS Shared Catalog
Powerful access to qualitative data: What’s behind the UK QualiBank
CSE591: Data Mining by H. Liu
Palestinian Central Bureau of Statistics
Presentation transcript:

A DTD for Qualitative Data: Extending the DDI to Mark-up the Content of Non-numeric Data Libby Bishop and Louise Corti, UK Data Archive, ESDS, University of Essex IASSIST Conference May 2004

need a standard –that includes both file-level metadata and content-level metadata enables more precise searching/browsing extends to linking between sources (e.g. text, annotations, analysis, audio etc) need one customised to social science research that: –meets generic needs of varied data types –is more analytical than ones adapted from TEI speech schema (e.g. oral history projects) –is less granular than ones for conversational analysis (highly detailed) Why another DTD?

Specific applications marking up data to an XML standard for data providers to publish to online systems, such as ESDS Qualidata Online (formerly Edwardians) meet needs of researchers requesting a standard they can follow encourage more qualitative data analysis software companies to pursue XML- outputs (and import/export tools) based on this standard

Hybrid of two standards for the metadata – the DDI Standard for study, file and variable level Level 1: DDI Document description Level 2: DDI Study description Level 3: DDI Data file description –file contents; format; data checks; processing; software) Level 4: DDI Variable description: –for study survey data (mixed methods) or numeric outputs from qualitative data: demographic profile of sample other quantified responses to qualitative data (attributes or thematic classifications often assigned (coded) in CAQDAS software) Level 5: DDI Other Study related materials Level 6: TEI-based qualitative content

TEI for content mark-up standard for text mark-up in humanities and social sciences Elements for the header for a TEI-conformant DTD: standard bibliographic ref to text Mandatory =

Four components of a TEI DTD core tag set – available to all TEI docs base tag set – Transcription of speech additional tag sets – optional –linking –analysis –certainty and responsibility –transcription –names and dates –corpora entity tag sets – not needed

Issues this DTD resolves multiple speakers turn taking researcher annotations of transcripts thematic coding (as well as is possible with XML) name and place references compatibility with existing XML-enabled qualitative data analysis software (e.g. Atlas.ti output) As always, formatting elements handled with style sheets, not in the DTD

Much work remains… Further integration of DDI and TEI required elements Define the DTD for an individual case (e.g. transcript) or a collection, or both? Elements selected: not too many, not too few – assign mandatory and optional How elements are used: follow existing norms, set standard where necessary Need DDI specialist interest group/DDI structural reform group to help define and refine a suitable DTD

Proposed elements and samples See Table of Proposed Elements Sample case-level XML (transcript) marked up with a subset of proposed elements Sample study-level XML using DDI standard (levels 1-3 and 5) Draft DTD soon available on ESDS Qualidata website

Excerpt from interview transcript

Excerpt with XML mark-up … My father was, in the daytime he was a boilermaker on the old North Staffordshire Circular Railway and then every night he played in the theatre orchestra. And sometimes even after the theatre he would go on and play for an hour or two at a dance, well they called them balls in those days. And he 'd to go to had got to be at work at six the next morning! Cornet player.

Thematic coding: Stand-off Architecture in XML Challenges for developing an XML application included the multiple hierarchies in the transcript texts and overlapping fields or elements: dialogue structure v thematic content Conventional mark-up of these structures in a single document violates nesting rules of XML Solution - stand-off annotation approach whereby data and coding stored in different documents (annotation linked by Xlink and Xpointers) Proven utility as method for annotating multi-coded dialogue corpora. Allows for: –multiple coding schemes –overlapping elements –easily extendable

Base-line text unit: utterances ( ) Theme: politics Theme: household Theme: work attributes: id speaker … start time (audio file) end time (audio file) Example of Stand- off XML Architecture

In-house tool for coding themes Permits import and export, not relying on any proprietary CAQDAS package.

Selected elements from Atlas for codes (themes) and pointers <code name="A Formula" id="co_5" au="Thomas M" cDate=" T14:30:57" mDate=" T13:19:42" cCount="0" qCount="1" > <q name="And the name of the star is ca.. id="q1_1" au="Admin" cDate=" T13:27:48 mDate=" T21:45:00" 27, 27"/>

What does the DTD enable? ability for data producers to publish data in multiple formats using style sheets/using web-based systems e.g. ESDS Qualidata Online – brief demo ore/transcriptsmultiple.asp enable data exchange and data sharing across dispersed repositories (c.f. Nesstar) Enable the development of import/export functionality for CAQDAS software

Need for publishing tools Once DTD is more devloped, next step is to develop publishing tools to automate as much of mark-up as possible Currently using simple scripts to find and mark and ; much work still done manually Looking into options for automatic mark-up of some components (e.g. natural language processing and information extraction): –Brill tagger –Gate architecture –Customising existing NLP tools at Sheffield and Edinburgh

Collaborators Oxford Computer Centre (TEI) NLP team at Sheffield NLP team at Essex NLP team at Edinburgh Atlas.ti developers (Berlin) Cardiff Ethnography Group E-social science programme text mining groups Academics in UK who wish to use standard FSD US and rest of world? DDI, IASSIST, CESSDA

Selected References ESDS Qualidata Qualidata Online website Barker, E. and Corti, L. (2002) Enhancing access to qualitative data: Edwardians On-line. ASLIB Journal, Assignation, 20, pp Carmichael, P. (2002) Extensible mark-up language and qualitative data FSQ 3(2), research.net/fqs-texte/2-02/2-02carmichael-e.htm Derose, S. (1999) XML and the TEI. Computers and the Humanities. 33, pp Kuula, A. (2002) Making qualitative data fit the Data Documentation Initiative or vice versa? FSQ 1(3) e.htm Muhr, T. (2000) Increasing the reusability of qualitative data with XML. FSQ 3(1) texte/3-00/3-00muhr-e.htm#g42 Muller, E. et al. Using XML for long-term preservation. Sperberg-McQueen, C.M.. and Burnard, L. (eds.) (2002). TEI P4: Guidelines for Electronic Text Encoding and Interchange. Text Encoding Initiative Consortium. XML Version: Oxford, Providence, Charlottesville, Bergen)

For more information ESDS Qualidata introduction.asp ESDS Qualidata Online