1. Voice Recognition in the Electronic Health Record
Diane Luedtke
Nursing Informatics, NSG600INA
November, 2010

2. Speech Recognition Definition
The process of converting an acoustic signal, captured by a microphone or a telephone to a set of words.

3. History 1952 - Recognition of single digits
1964 – Device exhibited at NY World’s Fair
1980’s – 1,000 to 20,000 word vocabularies
Early 90’s – Accuracy 10% to 50% and “discrete” voice recognition
1997 – Recognition of normal speech
Early 2000’s – Accuracy 80%

4. Types of Speech Recognition
Isolated - pause between words
Continuous – no pause between words
Spontaneous – extemporaneous – most difficult to recognize

5. Properties Speaker enrollment Speaker independent Finite state network
General language models
Perplexity
External parameters

6. Variables Phonemes Acoustic variables Within speaker variables
Across speaker variables
Zue, V., Cole, R., Ward, W. Speech recognition. Retrieved from on 10/6/2010.

7. http://www. google. com/imgres. imgurl=http://static. howstuffworks

8. Speech Recognition in Health Care
Earliest users – radiologists
Most successful early users – radiologists, pathologists and emergency physicians
Photo source:

9. Other Healthcare Settings
Primary care clinicians
Psychiatrists
IV nurses - AccuNurse

10. Primary Care Trial at US Army Medical Command in 2009
10,000 copies of voice recognition software
Installed 42 healthcare facilities
Software tutorial and face-to-face training offered
“Champions” trained
Accuracy rated 90% by all participants
Not used with patient in exam room, but used immediately after seeing patient by majority of users
Hoyt, R., & Yoshihashi, A. (2010, Winter). Lessons learned from implementation of voice recognition for documentation in the military electronic health record system. Perspectives in Health Information Management, 7(Winter). Retrieved from

11. Primary Care
Clinic from Wellspan Health implemented electronic health records with voice recognition included
Voice recognition treated as component of EHR
Used in exam room with patient
Baker, R.H. (2010, May). Voice recognition assists clinicians. Health Management Technology. Retrieved from

12. The VA Early trial in late 1990’s Cost $2,000 per work station
Compare 3 word recognition systems using 12 physicians
Evaluation from scripted charting
Error rate ranged from 6.6% to 14.6%
Estimate current use by 7000 nurses and physicians
Devine, E.G., Gaehde, S.A., & Curtis, A.C. (2000, Sept-Oct). Comparative evaluation of three continuous speech recognition software packages in the generation of medical reports. Journal of the American Medical Informatics Association, 7(5),

13. Psychiatry Health record includes dense narrative
In mandatory implementation, providers who do not type notes more inclined to accept voice recognition
Providers would not dictate in front of patient
Providers found no perceived benefit in speech recognition
Half of the evaluators favored the use of speech recognition
Derman, Y.D., Arenovich, T, Straus, J. (2010). Speech recognition software and electronic psychiatric progress notes: physicians’ ratings and preferences. BMC Medical Informatics and Decision Making, 10:44. Retrieved from

14. IV Nurses Used at Butler Memorial Hospital, Butler, PA
Pilot project with 3 IV nurses
Lightweight headset and pocket sized wireless device
Computer entry of IV needs sent to nurse’s headset
On completion of patient care, nurse uses voice recognition system to record what was done in patient’s record
Receive next order over headset for next patient
McGee, Marianne Kolbasuk. (2009, September 17). Voice recognition tools make rounds at hospitals. InformationWeek Healthcare. Retrieved from

15. Patient Interactive Voice Response System
Automated telephone calls made to patients on day following surgery
Patients respond to questions via speech
Speech recognition software updates database based on to patients response
If response indicates follow-up telephone call by nurse, nurses will be prompted to complete contact
System reported to be 97% accurate
Foster, AJ; LaBranche, R; McKim, R; Faught, JW; Feasby, TE; Janes-Kelley, S; Shojania, KG; van Walraven, C. (2008). Automated patient assessments after outpatient surgery using an interactive voice response system. The American Journal of Managed Care, 14(7),

16. Benefits of Speech Recognition
Reduction of transcription expense
Improved patient care
Reduction in time documenting care
Increase per patient revenue
Allows physician to dictate in their own words
Does not add recurring labor costs

17. Barriers to Speech Recognition
Capital cost of EHR with speech recognition
Cost in time (users)
Security or confidentiality issues
Costs to maintain EHR
Interference with doctor-patient relationship
Difficulty with learning new technology
Lack of tech support
Lack of perceived benefit

18. Problems with Speech Recognition
Accuracy rate approximating 90% requires editing
Upgrade of processor speed and/or random access memory may be required
Change in method of documenting encounter notes
Not all users receiving appropriate training

19. Thank You!