Approaching Utopia: Approaching Utopia: the maturing of qualitative data archiving services and secondary analysis in the UK Louise Corti Associate Director, UK Data Archive, University of Essex Symposium, Grenoble, 3-4 November 2005

Overview of talk the beginning of a national qualitative data service brief history of service key issues in building model supporting user needs current directions synergy with UK qualitative research arena

ESDS Qualidata function of the Economic and Social Data Service (ESDS) specialist service led by the UK Data Archive at the University of Essex acquires, provides access to, and support for, a range of qualitative datasets on a national scale responsible for enhancing qualitative data and documentation provides information and training resources for re-analysing qualitative data

Economic and Social Data Service a more integrated national approach to data archiving and dissemination provides more seamless and easier access to a range of disparate social science data resources dedicated functions: –Management and Co-ordination Function –Core Data Archiving and Preservation Service –Government Data Service –International Data Service –Qualitative Data Service –Longitudinal Data Service

Back in time …a brief history academic pilot study (1991/2) –availability of academic data and willingness to share 5 year contract for small Qualidata service (1994 – 99) development of archive and user service, and procedures (1995-) incorporation into UK Data Archive (2000-) dedicated support service for UK Economic and Social data Service

Qualidata funding ( ) £1994.7m over 5 years (ESRC) £ k for establishing NSPSCA (JRF) £ m funding for 3 years (ESRC) £ k for preservation project (MRC) £ k for 1 year (ESRC) £ k for 2 years (ESRC) UKDA £2003share of total 7m for 5 years staff 2.2 FT} staff 3 FTE} No technical staff staff 5 FTE} in house staff 3 FTE} staff 3.25 FTE

Data Archives and Digital Libraries Research Groups Traditional Archives and Libraries Individual researchers Who is archiving qualitative data?

Researchers and Archives Research groups and organisations 3universities, public and private sectors 3storage by default 3for own organisational purposes University Archives 3where strong social science research tradition 3where already significant collections of older research materials Libraries and Museums 3where strong historical, ethnology or oral history collections Data Archives/Libraries 3Proactive acquisition of digital research resources

Initial issues for a qualitative data service defining data acquisition priorities providing systematic procedures for preparing or ‘processing’ data supplying usable data in desirable formats — digital, text, multimedia across range of themes and methods selecting appropriate access conditions for sensitive data getting tailored legal clarification on consent and copyright

Initial issues for a qualitative data service creating high quality documentation (metadata) for effective resource discovery building a sustainable user community offering training for researchers, students and teachers creating and adding to methodological debate pleasing your funders!

Key developments and innovation no existing procedures to meet specific needs developing data processing procedures over first 3 years then continuous refining – archiving and technical pushed hard for formalised ESRC national Datasets Policy to include qualitative data persuaded other funding bodies to encourage archiving and data sharing proving service viability beyond pilot period

Key developments and innovation building small user community & recognition of secondary analysis as a robust method promoting and encouraging academic debate – new methods –outreach and publishing creating synergy and harmonisation with quantitative UK Data Archive gaining recognition by international data archiving community

Gaining initial advice Very little literature to guide back in Liaison with: traditional archives community oral history community UK Data Archive Murray Centre, USA international network of data archive – IASSIST and CESSDA legal experts computing consultancy for early web catalogue

UK Data Archive Merger revised policy of retaining deposited qualitative datasets in-house: –digital resources and learning materials access to organisational infrastructure and resources e.g. computing and programming staff less isolated more scope for cross training and collaboration greater access to data archiving ‘club’ – infrastructure

Transition cultural shift for existing staff adapting and harmonising procedures lengthy process cross-working and broader data appreciation juggling multiple roles for senior staff overall data acquisitions balance across all social science data agreeing in-house funding allocations and balance re-establishing of priorities

Qualitative data collections Key sources of data: data from National Research Council (ESRC) individual and programme research grant awards data from ‘classic’ social science studies other funders of social science existing archived collections that can be enhanced for social research use

Qualitative data criteria relative importance or impact of the study eg. had a major influence in its field and/or representing the working life of a significant researcher complementary to existing data holdings popularity of the study topic (health, criminology, social policy) data that have further analytic potential than the original investigation. based on national samples mixed methods data

The paper mountain most new collections are born digital but much older data in paper format will digitise paper: –scan and OCR samples of key data –scan as image files to enable faster throughput may digitise sound bytes from audio cassette facilitate archiving of larger non-digital collections in traditional other archives across the UK –but may selectively digitise ‘highlights’

Locating data …obtaining data often very long lead time between locating data and acquiring data record = 9 years (data promised in 1994) letting go of data can be a painful experience…. may require extensive ‘negotiation’ about certain issues …and longer term counselling but also news travels – ‘I hear you have Prof. Xs data, don’t you want mine?”

‘Difficulties’ cited in re-using data problem of the implicit nature of qualitative data collection and analysis – context and reflexivity lack of time to get fully acquainted with research materials created by someone else constraints of informed consent insecurity about the exposure of one’s own research practice; IPR or threat of misinterpretation lack of publicly available research data

Encourage users researchers to consult existing data sources consider both substantive and methodological issues in data re-use teachers to use ‘real’ data for teaching and learning students to explore original data sources

Facilitating secondary analysis acquire a broad range of rich qualitative data make obtaining data more straightforward producing enhanced data and documentation: –user guides, thematic intros and digital samplers –exemplars and case studies of re-use –online access to qualitative data active promotion, outreach and publishing providing range of user support and training activities But need adequate time to get to know data well - new grants, phDs

Areas of best practice offered: data creation consent and confidentiality copyright recording equipment transcription methods data management — formats, software, version control, security, back up, web standards data preparation for archiving documentation, metadata, contextualisation data mark up

Promoting best practice in data creation help desk for individual discussions, new grant applicants – and phone informative web site printed materials data creator and depositor roadshows break-out discussion groups surgeries working with research funding bodies to implement data archiving policies

Areas of best practice offered: data use consider multiple sources – qual and quant sign up to user agreement provide project use details provide case study of use –or talk at workshops on experiences non-selective download of transcripts – take the whole help prepare tailored teaching datasets

Online service development Have additional R&D funding to support: the development of online service the development of XML mark-up standards and tools using natural language processing tools to enable semi-automated data preparation have a draft mark up schema (DTD) based on TEI/DDI currently in circulation QUADS and SQUAD grants

Qualitative data for national data archives? already acquiring qualitative data 3may acquire qualitative data 3 evaluating policy 3 feasibility studies 7have not considered/will not consider acquiring 7no time /resources to investigate 7think an impossible task 7no interest in these kinds of data

Summing up: requirements a data archiving infrastructure that is adequately funded mandatory data sharing policies and a clear data acquisitions strategy robust procedures for data prep. and delivery leadership and direction: reputation and innovation effective management – staff and project highly skilled staff –research, technical and support skills access to research and technical expertise networks access to skilled users

Recent ESRC qualitative research initiatives offering important opportunities for collaboration: Archiving and Sharing Demonstrator scheme National Qualitative Longitudinal Study National Centre for Research Methods National Centre for e-Social Science