Comparing the Quality & Validity of Voice Recognition Software & Professional Transcription Services Sarah Ayres (University of Bristol) & Ian Stafford (Cardiff University) /default.shtml

Introduction This paper provides an overview of a small scale evaluation of voice-recognition (VR) software compared to professional transcription services Key Questions: –Why use voice VR software for transcribing qualitative field data? –What methodology was used to evaluate VR software? –How did the ‘quality’ of transcripts produced by VR software compare with those produced by a professional transcription service? –What are the advantages to using VR software as part of a working project?

The Rationale ESRC bid - to make use of cutting edge technologies & improve efficiency & reliability Identified digital recorders & Dragon V9 VR software Why use VR software to transcribe digital recordings of interviews? –Professional transcripts - errors & formatting concerns –Ensuring sensitivity to the data –Transcription is time consuming & arduous - software will be quicker –Costing - time & money

Evaluating VR software ESRC bid detailed a small evaluation: A purposive sample of three digitally-recorded interviews –A high degree of technical language –A recording with high background noise –An interview with more than one interviewee (3 voices to identify) The three interviews were: –Transcribed by a professional transcriber –Transcribed by a RA using VR software The transcripts were given to three independent assessors to rate the quality of the transcripts based on: –Visual appearance/formatting –Accessibility - e.g. conveys a clear message –Accuracy - e.g. technical terms –Spelling & punctuation In addition a cost comparison has been calculated

Key findings InterviewVisual Appearance/ Formatting AccessibilityAccuracySpelling/ Punctuation Interview Total (60) PROVRPROVRPROVRPRO VRPROVR Technical Interview Noisy Interview Multi-actor Interview Criteria Total (45)

Comparing the costs The professional transcription service charged £300 (inc VAT) for the three recordings Hourly rate of RA on the project was £21.47 (approx £25k at time of appointment in 2006) The RA transcribed the three interviews in the following times: –Technical interview (2 hours, £42.94) –Noisy interview (1.55 hours, £41.15) –Multi-actor interview (2.53 hours, £61.91) Total cost of RA transcriptions - £146 (£154 cheaper than professional transcription)

Conclusions THE FINDINGS ARE HIGHLY CONTEXT SPECIFIC & THE EFFECTIVESS OF VR SOFTWARE WILL DEPEND ON PROJECT SPECIFIC VARIABLES Nonetheless: –The software successfully managed the tasks as set out in the original project brief –The technology performed better than anticipated –Transcribing using VR software is faster than manual typing –Transcripts produced using VR software are better & cheaper than those produced by professional transcription services –Assistance & training from VR expert is essential –We would use the technology again in a working project