Effects of structured template reporting on the quality of radiology reports Juan Carlos Batlle, MD Giles W Boland, MD Daniel I Rosenthal, MD Partners.

Slides:

Advertisements

Similar presentations

Student name Student ID Degree program Area of specialization.

Advertisements

SP 225 Lecture 11 Introduction to Hypothesis Testing.

Radiographer Reporting Presented by Chan Tsz Chun d Ng Kwan Ho d Wong Kin Long, Gulliver d.

Significance Testing.  A statistical method that uses sample data to evaluate a hypothesis about a population  1. State a hypothesis  2. Use the hypothesis.

Mark Troy – Data and Research Services –

Where does the money come from in Radiology?

Software Quality Metrics

+ Team Members Evaluation and Implementation of Web 2.0 Technologies in Support of CReSIS Polar and Cyberinfrastructure Research Projects at Elizabeth.

Journal Club Alcohol and Health: Current Evidence July–August 2005.

Insert Title Here No More than 1 or 2 Lines, Point Font Statement of Technology/Best Practice No More Than 2-3 Lines Point Easy to Read.

University of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation Unit 6: Analyzing and interpreting data “There’s a world.

SW388R7 Data Analysis & Computers II Slide 1 Multiple Regression – Basic Relationships Purpose of multiple regression Different types of multiple regression.

 Pages made using data in a database  Use scripting languages  Use of RDMS  Look and feel by templates, CSS  High scaling  Admin pages as back end.

Acute Ischemic Stroke Management: 2004 Emergency Medicine Perspectives.

Educational Research Chapter 13 Post-Analysis Considerations Gay, Mills, and Airasian.

Evaluation and analysis of the application of interactive digital resources in a blended-learning methodology for a computer networks subject F.A. Candelas,

Computing Revision Evening. The Course ( Computing ) The Exam There is one exam paper which is worth 40% of the overall mark and takes 1½ hours Course.

AMOST Experimental Comparison of Code-Based and Model-Based Test Prioritization Bogdan Korel Computer Science Department Illinois Institute of Technology.

Examining the Impact of Non- Resident Births on Systems Limitations in Philadelphia Obstetrical Care Cynthia L. Line, Ph.D. Patricia Morris, MPH Katherine.

Follow-up of consolidation on chest radiographs before and after the introduction of radiology initiated follow-up. Cliffe H, Walsh J, Kon M.

Program Evaluation. Program evaluation Methodological techniques of the social sciences social policy public welfare administration.

Patterns in Child Outcomes Summary Data: Cornelia Taylor, Lauren Barton, Donna Spiker September 19-21, 2011 Measuring and Improving Child and Family Outcomes.

Software Estimation and Function Point Analysis Presented by Craig Myers MBA 731 November 12, 2007.

ATIA 2009 Accessible Online State Assessment Compared to Paper-Based Testing: Is There a Difference in Results? Presenters: Linnie Lee, Bluegrass Technology.

WRITING THE PAPER WEEK 8. Writing the intro  Are these essential or optional, in an Intro?:  Gain the reader’s interest by offering an opening statement,

Indices Using Weighted Sums and Averages Readability Indices.

JUSTIFICATION OF COMPUTERIZED TOMOGRAPHY EXAMINATIONS AND RADIATION RISKS IN EVERYDAY RADIOLOGICAL PRACTICE Darka R. Hadnađev, Olivera Nikolić, Sanja Stojanović.

Can a Brief On-line Education Tool Improve Surgical Resident Operative Dictations? A Prospective Evaluation Alicia Kieninger, MD, Yi Wei Zhang, MD, Anna.

Automatic Readability Evaluation Using a Neural Network Vivaek Shivakumar October 29, 2009.

Publishing Seasonally Adjusted Data Anu Peltola Economic Statistics Section, UNECE UNECE Workshop on Seasonal Adjustment 20 – 23 February 2012, Ankara,

PrelimCheck: Software to Facilitate Follow-Up of Resident Preliminary Reports Phillip Cheng, MD MS Assistant Clinical Professor of Radiology.

Audit of Radiology Alert system for critical, urgent and unexpected significant findings K A Duncan K Drinkwater On behalf of CRAC May 2015.

September 11, Assessment Primer A. Defining the Assessment Loop B. Establishing Departmental-level learning goals and objectives C. Assessment Terminology.

Software Architecture Evaluation Methodologies Presented By: Anthony Register.

M Itani a, B Taslakian a, N. Batley b, M. Saliba b, E. Hitti b, F El-Merhi a Departments of Radiology a and Emergency Medicine b American University of.

A Metrics Program. Advantages of Collecting Software Quality Metrics Objective assessments as to whether quality requirements are being met can be made.

NMC Online Conference December 2005 Assessing Learning in a MORPG Patricia Youngblood, PhD Director of Evaluation, SUMMIT (Stanford University Medical.

Exam 2 Review. Example Information A researcher wants to know if using their flashcard program increases the number of items an individual can memorize.

Middle School Science Text Adoption Process. Preparing to Pilot February ‘07 - teacher committee narrows state-approved 7/8 science texts to two top choices:

Assessment of Course-Level Learning Outcomes in Psychology.

Barriers and Facilitators to Computer Use in VA for Implementing Guidelines Brad Doebbeling, MD, MSc VA Indianapolis HSR&D Center of Excellence, Indy VAMC.

Education 793 Class Notes Inference and Hypothesis Testing Using the Normal Distribution 8 October 2003.

POSTER TEMPLATE BY: Background Objectives Psychophysical Experiment Smoothness Features Project Pipeline and outlines The purpose.

ACRIN Breast Committee Fall Meeting CADe study Etta Pisano, MD Martin Yaffe PhD Elodia Cole, MS Zheng Zhang, PhD ACRIN Breast Committee.

ReportDiff: Analysis and Feedback for Resident Dictated Reports Phillip Cheng, MD MS Miriam Romero, MD M. Victoria Marx, MD.

Which Method Is More Effective In Treatment Of Calcific Tendinitis In The Shoulder? - Prospective Randomized Comparison Between US- Guided Needling and.

Cognos BI. What is Cognos? Cognos (Cognos Incorporated) was an Ottawa, Ontario-based company that makes Business Intelligence (BI) and Performance Management.

Radiology responsibilities post NPSA guidelines for nasogastric tubes: A single centre review 1 Kevin Flintham Bev Snaith.

» Give details of local providers able to provide specialist assessment » Give details of the range of provision within the area » Select provision options.

A Healthier Tomorrow Begins Today National Immunization Conference 2004.

Building Capacity in Evaluating Outcomes Unit 6: Analyzing and interpreting data 1 “There’s a world of difference between truth and facts. Facts can obscure.

Choosing Wisely : Radiology Perspective

Quality assurance in radiology reporting:

AP Language and Composition

We gratefully acknowledge the support of the AMDA Foundation and Pfizer for an unrestricted Quality Improvement Award.

Introduction Results Materials and Methods Conclusions References

Employment and Social Affairs Platform

Lights! Camera! Graphs! Photos!

Implementing Common Core 2.0

L.O. – How can we manipulate our comic to clarify meaning?

Focussed on whole of population data set

Patient Education: Are We Getting the Message Across?

Unit 6: Analyzing and interpreting data

University of Wisconsin-Madison

Christopher S. Kiefer MD, Erica B. Shaver MD,

Student name Student ID Degree program Area of specialization

METRO SOUTH EDUCATION DISTRICT Western Cape Department of Education

Title in large easily readable font

Tri-College Teagle Foundation Systematic Improvement Grant Retreat: Sustainable Department-Level Assessment of Student Learning September 11, 2009.

Advance HE Surveys Conference

Presentation transcript:

Effects of structured template reporting on the quality of radiology reports Juan Carlos Batlle, MD Giles W Boland, MD Daniel I Rosenthal, MD Partners Research Retreat March 24, 2009 Originally presented at RSNA December 2, 2008

Background Structured reporting increasingly used as a means of systematizing reports and making them clinician and “computer” friendly Stillman 2008 JACR 51; Taira 2001 Radiographics 21 Transition from free dictation to structured reporting will involve structured templates for free dictation Effects of templates on reporting are uncertain

Purpose To evaluate surrogate objective metrics for assessment of text quality of radiology reports To assess the change in text quality of reports after a department-wide transition to automatic structured templates issued by the department

Brief Methods 1 In early 2007, MGH Radiology transitioned to Commissure platform (Nuance Communications) voice-recognition dictation software Previous dictation software allowed personal templates (“macros”) used in addition to free-dictation Commissure allows automatic “population” of a structured template into the dictation upon opening the patient exam, according to exam code (e.g. CTABP+, CTCH+, etc.). Structured templates for each exam code created by division committee

Brief Methods 2 OLAP (Online Analytical Processing) Cube used to extract dictations of all abdominopelvic CTs interpreted by three randomly-selected radiologists from 2006 and 2007 (n=9594) into Excel. Comparison was made of the text of the reports before (n=5100) and after (n=4494) a transition to automatically populated structured templates. Text was analyzed for word count, sentence structure, and Flesch Reading Ease by semi- automated computer software. Flesch Reading Ease rates text on a 100-point scale based on the average number of syllables per word and words per sentence.

Results After the transition to structured reports: Significant decrease in word count (292 to 226 words, p<0.001) Significant increase in number of words per sentence (4.8 to 5.0, p<0.001) Significant worsening in Flesch reading ease score (30.6 to 24.6, p<0.001)

Word Count Significantly Decreased After Transition

Readability Significantly Decreased After Transition

Follow-up Data Preliminary assessment of two randomly selected radiologists from the thoracic imaging division: Attending 1 Word count decrease from 266 to 231 Words/sentence increase from 4.9 to 5.4 Attending 2 Word count decrease from 246 to 232 Words/sentence increase from 5.2 to 5.5

Limitations Small group of radiologists studied, though large sample size Metrics of text “quality” are imperfect surrogates for true quality measurement

Conclusions Implementation of structured templates led to significant changes as measured in the text of dictations. This may significantly influence overall readability of reports and may be unanticipated Future attention should be paid to optimizing structured templates according to surrogate metrics of text quality Ongoing assessment of these metrics may be performed easily using OLAP cube software