Medical Record Reviews – The Rules of the Road David H. Rubin, MD Department of Pediatrics.

Slides:



Advertisements
Similar presentations
Review bootstrap and permutation
Advertisements

Research Curriculum Session II –Study Subjects, Variables and Outcome Measures Jim Quinn MD MS Research Director, Division of Emergency Medicine Stanford.
STUDY DESIGN CASE SERIES AND CROSS-SECTIONAL
Estimation of Sample Size
Some comments on the 3 papers Robert T. O’Neill Ph.D.
Introduction to Evidence Based Medicine Pediatric Clerkship LSUHSC.
Elements of a clinical trial research protocol
Module 2 Psychology & Science.
Writing a Research Protocol Michael Aronica MD Program Director Internal Medicine-Pediatrics.
Concept of Measurement
PSYC512: Research Methods PSYC512: Research Methods Lecture 16 Brian P. Dyre University of Idaho.
Journal Club Alcohol and Health: Current Evidence January-February 2006.
Chapter 3 Preparing and Evaluating a Research Plan Gay and Airasian
Introduction to Communication Research
Selection of Data Sources for Observational Comparative Effectiveness Research Prepared for: Agency for Healthcare Research and Quality (AHRQ)
EVIDENCE BASED MEDICINE
Cohort Studies Hanna E. Bloomfield, MD, MPH Professor of Medicine Associate Chief of Staff, Research Minneapolis VA Medical Center.
Quality Assurance in the clinical laboratory
Part 3 of 3 By: Danielle Davidov, PhD & Steve Davis, MSW, MPA INTRODUCTION TO RESEARCH: SAMPLING & DESIGN.
Adherence to Sepsis Guidelines and Hospital Stay Elspeth Ferguson SCH Journal Club 6 th November 2012.
Copyright © 2008 by Pearson Education, Inc. Upper Saddle River, New Jersey All rights reserved. John W. Creswell Educational Research: Planning,
17 June, 2003Sampling TWO-STAGE CLUSTER SAMPLING (WITH QUOTA SAMPLING AT SECOND STAGE)
Validity and Reliability Dr. Voranuch Wangsuphachart Dept. of Social & Environmental Medicine Faculty of Tropical Medicine Mahodil University 420/6 Rajvithi.
STrengthening the Reporting of OBservational Studies in Epidemiology
Spring 2015 ETM 568 Callier, Demers, Drabek, & Hutchison Carter, E. J., Pouch, S. M., & Larson, E. L. (2014). The relationship between emergency department.
Statistics in Screening/Diagnosis
Collecting Quantitative Data
Research Techniques Made Simple: Databases for Clinical Research Katrina Abuabara, MD MA David Margolis, MD PhD University of Pennsylvania.
RESEARCH A systematic quest for undiscovered truth A way of thinking
1 Lecture 2: Types of measurement Purposes of measurement Types and sources of data Reliability and validity Levels of measurement Types of scale.
Study Design. Study Designs Descriptive Studies Record events, observations or activities,documentaries No comparison group or intervention Describe.
The potential impact of adherence to a guideline on the utilization of head CT scans in traumatic head injury patients. Frederick K. Korley M.D.
Collecting Quantitative Data
Evidence-Based Medicine 3 More Knowledge and Skills for Critical Reading Karen E. Schetzina, MD, MPH.
Research Designs Murray W. Enns Professor of Psychiatry.
PARAMETRIC STATISTICAL INFERENCE
Turning a clinical question into a testable hypothesis Lauren A. Trepanier, DVM, PhD Diplomate ACVIM, Diplomate ACVCP Department of Medical Sciences School.
Evidence Based Medicine Meta-analysis and systematic reviews Ross Lawrenson.
Bias Defined as any systematic error in a study that results in an incorrect estimate of association between exposure and risk of disease. To err is human.
Example Papers Prospective Validation of the Pediatric Appendicitis Score in a Canadian Pediatric Emergency Department Maala Bhatt, MD, MSc, Lawrence Joseph,
Sampling “Sampling is the process of choosing sample which is a group of people, items and objects. That are taken from population for measurement and.
Literature searching & critical appraisal Chihaya Koriyama August 15, 2011 (Lecture 2)
Criteria to assess quality of observational studies evaluating the incidence, prevalence, and risk factors of chronic diseases Minnesota EPC Clinical Epidemiology.
Title of Scholar Project Month day, year Presenter: Supervisor(s): Critical Care Western.
Clinical Writing for Interventional Cardiologists.
SAMPLING TECHNIQUES. Definitions Statistical inference: is a conclusion concerning a population of observations (or units) made on the bases of the results.
Understanding Study Design & Statistics Dr Malachy O. Columb FRCA, FFICM University Hospital of South Manchester NWRAG Workshop, Bolton, May 2015.
Guidelines for Critically Reading the Medical Literature John L. Clayton, MPH.
EBM --- Journal Reading Presenter :呂宥達 Date : 2005/10/27.
Chapter 2 What is Evidence?. Objectives Discuss the concept of “best available clinical evidence.” Describe the general content and procedural characteristics.
Systematic Synthesis of the Literature: Introduction to Meta-analysis Linda N. Meurer, MD, MPH Department of Family and Community Medicine.
BY DR. HAMZA ABDULGHANI MBBS,DPHC,ABFM,FRCGP (UK), Diploma MedED(UK) Associate Professor DEPT. OF MEDICAL EDUCATION COLLEGE OF MEDICINE June 2012 Writing.
24 Nov 2007Data Management and Exploratory Data Analysis 1 Yongyuth Chaiyapong Ph.D. (Mathematical Statistics) Department of Statistics Faculty of Science.
Topics Semester I Descriptive statistics Time series Semester II Sampling Statistical Inference: Estimation, Hypothesis testing Relationships, casual models.
1 Lecture 10: Meta-analysis of intervention studies Introduction to meta-analysis Selection of studies Abstraction of information Quality scores Methods.
Retrospective Chart Reviews: How to Review a Review Adam J. Singer, MD Professor and Vice Chairman for Research Department of Emergency Medicine Stony.
Corso di clinical writing. What to expect today? Core modules IntroductionIntroduction General principlesGeneral principles Specific techniquesSpecific.
EBM --- Journal Reading Presenter :黃美琴 Date : 2005/10/27.
Statistical Concepts Basic Principles An Overview of Today’s Class What: Inductive inference on characterizing a population Why : How will doing this allow.
Research proposal (Lecture 3) Dr.Rehab F Gwada. Objectives of the Lecture The student at the end of this lecture should Know Identify Target Population.
Impact of Comorbidity on Chemotherapy Use and Outcomes in Solid Tumors: A Systematic Review Linda Lee, Winson Y. Cheung, Esther Atkinson, and Monika K.
Table 1. Methodological Evaluation of Observational Research (MORE) – observational studies of incidence or prevalence of chronic diseases Tatyana Shamliyan.
Critical Appraisal Course for Emergency Medicine Trainees Module 2 Statistics.
Top 5 papers of Prehospital care Recommended by Torpong.
Critically Appraising a Medical Journal Article
Supplementary Table 1. PRISMA checklist
Interventional trials
Reading Research Papers-A Basic Guide to Critical Analysis
The objective of this lecture is to know the role of random error (chance) in factor-outcome relation and the types of systematic errors (Bias)
Sampling Techniques Assist. Prof. Maha A. AL-Nuaimi Ph.D/ Comm. Med
Presentation transcript:

Medical Record Reviews – The Rules of the Road David H. Rubin, MD Department of Pediatrics

INTRODUCTION Any study that uses pre-recorded patient focused data as the primary source of information in a research studyAny study that uses pre-recorded patient focused data as the primary source of information in a research study Physician, nurses notes Physician, nurses notes Ambulance call reports Ambulance call reports Diagnostic tests Diagnostic tests Clinic, administrative, government records Clinic, administrative, government records Computerized databases Computerized databases

WHY SELECT THIS DESIGN? Allows research to address issues that cannot be addressed with prospective studiesAllows research to address issues that cannot be addressed with prospective studies Effect of harmful exposures (no randomization possible) Effect of harmful exposures (no randomization possible) Effect of potentially beneficial exposures Effect of potentially beneficial exposures Occurrence of rare events Occurrence of rare events Studies of patterns of disease or behavior Studies of patterns of disease or behavior Quality assurance studies Quality assurance studies Studies where cases may be shared (trauma database) Studies where cases may be shared (trauma database) Pilot studies for prospective studies Pilot studies for prospective studies

DATA QUALITY “Free form” quality of medical records may increase missing and/or erroneous data“Free form” quality of medical records may increase missing and/or erroneous data Handwriting may be illegible or uninterruptibleHandwriting may be illegible or uninterruptible May miss examining potential casesMay miss examining potential cases Computer vs paper recordsComputer vs paper records Data abstraction techniques require standardizationData abstraction techniques require standardization

MISSING OR CONFLICTING DATA Missing information may lead to nonresponse biasMissing information may lead to nonresponse bias Subjects with missing information may be very different from subjects without any missing information Subjects with missing information may be very different from subjects without any missing information Missing values managementMissing values management Case deletion – bias and reduced sample size are problems Case deletion – bias and reduced sample size are problems Case insertion – use what you have and compare missing/non missing cases for differences Case insertion – use what you have and compare missing/non missing cases for differences

SAMPLE SIZE Usually determined based on the summary measure and the size/width of the confidence interval desiredUsually determined based on the summary measure and the size/width of the confidence interval desired An interval with a greater CI (eg 99% CI v 95% CI) is wider and more likely includes the true population value An interval with a greater CI (eg 99% CI v 95% CI) is wider and more likely includes the true population value The width of the CI depends on sample size The width of the CI depends on sample size

SAMPLING Select all cases within a given time frameSelect all cases within a given time frame For nonconsecutive sampling it is best to choose probability samplingFor nonconsecutive sampling it is best to choose probability sampling Provides equal opportunity for each eligible case to be selected Provides equal opportunity for each eligible case to be selected Use random number generator Use random number generator Triage levelTriage level Incidental sampling – choosing most easily accessible casesIncidental sampling – choosing most easily accessible cases Systematic sampling – choosing every x th caseSystematic sampling – choosing every x th case

RELIABILITY Very importantVery important Any differences in data extraction by 2 different people?Any differences in data extraction by 2 different people? KappaKappa Value ranges from -1 (perfect disagreement) to 1 (perfect agreement) Value ranges from -1 (perfect disagreement) to 1 (perfect agreement) K = [observed agreement (%) – expected agreement (%) / [100% - expected agreement (%)] K = [observed agreement (%) – expected agreement (%) / [100% - expected agreement (%)] Try to achieve kappa of 0.6 or better (60% agreement) Try to achieve kappa of 0.6 or better (60% agreement)

MINIMUM REQUIREMENTS FOR MEDICAL RECORD REVIEWS (Lowenstein, 2005) 1)Explicit protocols for case selection/exclusion 2)Abstractor training 3)Precise definitions of key variables 4)Use of standardized abstraction and coding forms 5)Monitoring of abstractor performance 6)Blinding of abstractors to study hypothesis and patient groups 7)Testing of interrater reliability

QUALITY OF MEDICAL RECORD REVIEWS (Badcock, 2005) Observational study of medical record reviews published in several emergency medicine journalsObservational study of medical record reviews published in several emergency medicine journals 107 articles analyzed107 articles analyzed Clear aim reported in 93%Clear aim reported in 93% Standard abstraction forms: 51%Standard abstraction forms: 51% Interrater reliability: 25%Interrater reliability: 25% Ethics approval: 68%Ethics approval: 68% Sample size/power: 10%Sample size/power: 10%

MEDICAL RECORD STUDIES IN EM RESEARCH (Worster, 2005) From assessed medical record review studies in 6 EM journalsFrom assessed medical record review studies in 6 EM journals 79 (14%) medical review studies in 563 original research articles79 (14%) medical review studies in 563 original research articles Adherence to methodological criteriaAdherence to methodological criteria Sampling method: 99% Sampling method: 99% Abstractor blinding to hypothesis: 4% Abstractor blinding to hypothesis: 4% Interobserver agreement for the 12 criteria ranged from 57% - 95% Interobserver agreement for the 12 criteria ranged from 57% - 95%

ACCURACY IN HOSPITAL DISCHARGE SUMMARIES (Callen 2009) Retrospective analysis of 966 handwritten and 842 electronically generated discharge summaries in AustraliaRetrospective analysis of 966 handwritten and 842 electronically generated discharge summaries in Australia 12.1% of handwritten and 13.3% of electronic summaries contained errors12.1% of handwritten and 13.3% of electronic summaries contained errors Medication omission was biggest problemMedication omission was biggest problem NO difference in training levelNO difference in training level

MISSING INFORMATION IN ED CHARTS (Richason 2009) Examined case reports in 4 ED journals from and used 11 reporting standardsExamined case reports in 4 ED journals from and used 11 reporting standards 1,316 case reports identified1,316 case reports identified Poor reporting ofPoor reporting of Co-morbidities Co-morbidities Outcomes Outcomes Concurrent medications Concurrent medications

METHODOLOGY FOR RETROSPECTIVE REVIEWS IN CHILD PSYCHIATRY Conceive questionConceive question Literature reviewLiterature review Proposal methodsProposal methods Create data abstraction instrument and manualCreate data abstraction instrument and manual Sample sizeSample size Obtain IRB approvalObtain IRB approval Pilot studyPilot study

SAMPLE SIZE (Gearing 2006) Estimate 10 charts per variable (Sackett, 1991)Estimate 10 charts per variable (Sackett, 1991) Others estimate 5-7 charts/variableOthers estimate 5-7 charts/variable Convenience sampling – select cases over specific time periodConvenience sampling – select cases over specific time period Quota sampling – predetermined number sampledQuota sampling – predetermined number sampled Systematic sampling – every “nth” case chosenSystematic sampling – every “nth” case chosen

PRACTICAL ISSUES Check all possible CPT codes for diagnosis or procedure codeCheck all possible CPT codes for diagnosis or procedure code Febrile seizure may have been coded as seizure Febrile seizure may have been coded as seizure Gastroenteritis may have been coded as viral syndrome Gastroenteritis may have been coded as viral syndrome Pilot your Data Abstraction FormPilot your Data Abstraction Form Create detailed “Codebook” for your studyCreate detailed “Codebook” for your study Especially critical if > 1 researcher on study Especially critical if > 1 researcher on study

REFERENCES Worster A, Haines T. Advanced statistics: understanding medical record review (MRR) studies. Acad Emerg Med 2004;11: Worster A, Haines T. Advanced statistics: understanding medical record review (MRR) studies. Acad Emerg Med 2004;11: Lowenstein SR. Medical record reviews in emergency medicine: the blessing and the cure. Annals Emerg Med April 2005;45(4): Lowenstein SR. Medical record reviews in emergency medicine: the blessing and the cure. Annals Emerg Med April 2005;45(4): Babcock D et al. The quality of medical record review studies in the international emergency medicine literature. 2005;45(4): Babcock D et al. The quality of medical record review studies in the international emergency medicine literature. 2005;45(4): Worster A. et al. Reassessing the methods of medical record review studies 2005;45: Worster A. et al. Reassessing the methods of medical record review studies 2005;45: Gearing et al. Methodology for Retrospective chart review in child adolescent psychiatry. J Can Acad Child Adoles Psychiatry 15:3:2006Gearing et al. Methodology for Retrospective chart review in child adolescent psychiatry. J Can Acad Child Adoles Psychiatry 15:3:2006

REFERENCES Callen JJ, McIntosh. Accuracy of medication in hospital discharge summaries. Int J. Med Inform. 2009;Oct 1Callen JJ, McIntosh. Accuracy of medication in hospital discharge summaries. Int J. Med Inform. 2009;Oct 1 Richardson TP et al. Case reports describing treatrments in the ED. BMC Emerg Med June 15:9:10.Richardson TP et al. Case reports describing treatrments in the ED. BMC Emerg Med June 15:9:10.