© 2003 By Default! A Free sample background from Slide 1 Impact of an Automated Test Results Management System on Patients’ Satisfaction of Test Result Communication Michael E. Matheny, MD MS Biomedical Informatics Fellow Brigham & Women’s Hospital, Boston, MA

© 2003 By Default! A Free sample background from Slide 2 Lexington, KY University of Kentucky University of Kentucky –Chemical Engineering Undergraduate –Medical School

© 2003 By Default! A Free sample background from Slide 3 Indianapolis, IN Internal Medicine Residency Internal Medicine Residency –St. Vincent Hospital

© 2003 By Default! A Free sample background from Slide 4 Boston, MA Massachusetts Institute of Technology Massachusetts Institute of Technology –Master of Science Biomedical Informatics Biomedical Informatics Fellowship Biomedical Informatics Fellowship –NLM Fellow –Harvard-MIT Health Sciences & Technology Outpatient Urgent Care Outpatient Urgent Care –Brigham & Women’s Hospital

© 2003 By Default! A Free sample background from Slide 5 This Study Strong interest in applying clinical decision support to improve patient safety Strong interest in applying clinical decision support to improve patient safety –Clinical Reminders –Test Results Management AHRQ Grant (David Bates) AHRQ Grant (David Bates) Results Manager Intervention (Eric Poon) Results Manager Intervention (Eric Poon)

© 2003 By Default! A Free sample background from Slide 6 Background Test Result Communication Test result communication between patients and physicians is a critical part of the diagnostic and therapeutic process Test result communication between patients and physicians is a critical part of the diagnostic and therapeutic process However, follow-up of test results in the primary care setting is often challenging: However, follow-up of test results in the primary care setting is often challenging: –High volume of test results –Test results arrive when physician not focused on the patient –Lack of systems to ensure reliability and efficiency

© 2003 By Default! A Free sample background from Slide 7 Background Test Result Communication Problems 31% of women with abnormal mammograms did not receive care consistent with established guidelines 31% of women with abnormal mammograms did not receive care consistent with established guidelines 39% of abnormal TSH at BWH were not followed up within 60 days 39% of abnormal TSH at BWH were not followed up within 60 days 36% of abnormal pap smear were lost to follow-up 36% of abnormal pap smear were lost to follow-up

© 2003 By Default! A Free sample background from Slide 8 Background Physician Workflow 33% of physicians reported they did not always notify patients of abnormal test results 33% of physicians reported they did not always notify patients of abnormal test results ~30% of physicians reported they did not have a reliable method of test result communication ~30% of physicians reported they did not have a reliable method of test result communication 59% of physicians were dissatisfied with how well they managed test results despite spending over an hour a day in this activity 59% of physicians were dissatisfied with how well they managed test results despite spending over an hour a day in this activity

© 2003 By Default! A Free sample background from Slide 9 Background Patient Expectations Patients do not normally discuss their preferences for test result notification with their providers Patients do not normally discuss their preferences for test result notification with their providers Patients preferred telephone notification to regular mail, and found electronic notification to be uncomfortable due to security issues Patients preferred telephone notification to regular mail, and found electronic notification to be uncomfortable due to security issues Patients wanted to be notified of all test results, regardless of whether the results were abnormal Patients wanted to be notified of all test results, regardless of whether the results were abnormal

© 2003 By Default! A Free sample background from Slide 10 Background Patient Satisfaction These problems reduce patient satisfaction with their medical care, and impair future patient-physician interactions These problems reduce patient satisfaction with their medical care, and impair future patient-physician interactions Improving patient satisfaction has been identified as one of the most important issues currently facing healthcare Improving patient satisfaction has been identified as one of the most important issues currently facing healthcare

© 2003 By Default! A Free sample background from Slide 11 Background Current IT Environment Increasing numbers of practices are implementing electronic health records in order to improve documentation, billing, and for the promise of improved patient care Increasing numbers of practices are implementing electronic health records in order to improve documentation, billing, and for the promise of improved patient care While test result viewer applications are available in most commercial and home- grown EHRs, they typically require clinicians to separately keep track of their pending tests, and individually check a patient’s chart for the result While test result viewer applications are available in most commercial and home- grown EHRs, they typically require clinicians to separately keep track of their pending tests, and individually check a patient’s chart for the result

© 2003 By Default! A Free sample background from Slide 12 Background Current IT Environment A few institutions have implemented automated test result notification systems to physicians, but these systems have generally been deployed only in the inpatient setting for critically abnormal results A few institutions have implemented automated test result notification systems to physicians, but these systems have generally been deployed only in the inpatient setting for critically abnormal results The follow-up workflow in the outpatient setting is very different because patients are not in a controlled environment, which presents both communication and compliance barriers The follow-up workflow in the outpatient setting is very different because patients are not in a controlled environment, which presents both communication and compliance barriers

© 2003 By Default! A Free sample background from Slide 13 Objective To evaluate the impact of an EHR-imbedded automated test results notification system on patient satisfaction of test results communication To evaluate the impact of an EHR-imbedded automated test results notification system on patient satisfaction of test results communication

© 2003 By Default! A Free sample background from Slide 14 Methods Study Setting Partners HealthCare System Partners HealthCare System –Brigham & Women’s Hospital –Massachusetts General Hospital –Faulkner Hospital –McLean Hospital –Newton-Wellesley Hospital –Free Standing Outpatient Clinics Longitudinal Medical Record (LMR) Longitudinal Medical Record (LMR) –Released July 2000 –Scheduling –Medication lists –Problem lists –Health maintenance record –Clinic notes (free form & templates)

© 2003 By Default! A Free sample background from Slide 15 Methods LMR Summary Screen

© 2003 By Default! A Free sample background from Slide 16 Methods Study Setting Baseline state of test results management Baseline state of test results management –Test results were embedded directly into the patients’ electronic health record –No automated test results tracking –All test results were mailed to the physician’s clinic office –Physicians were paged directly for critical results

© 2003 By Default! A Free sample background from Slide 17 Methods Patient Test Results Screen

© 2003 By Default! A Free sample background from Slide 18 Methods Intervention Results Manager - an electronic test results management system embedded into the LMR Results Manager - an electronic test results management system embedded into the LMR –Features: Tracks and displays all test results associated with an ordering physicianTracks and displays all test results associated with an ordering physician Prioritizes by degree of test result abnormalityPrioritizes by degree of test result abnormality Facilitates review of test results in context of patient’s historyFacilitates review of test results in context of patient’s history Generates test result lettersGenerates test result letters Allows clinicians to set reminders for future testingAllows clinicians to set reminders for future testing

© 2003 By Default! A Free sample background from Slide 19 Methods Results Manager Summary Screen

© 2003 By Default! A Free sample background from Slide 20 Methods Results Manager Letter Generation Screen

© 2003 By Default! A Free sample background from Slide 21 Methods Study Design

© 2003 By Default! A Free sample background from Slide 22 Methods Enrollment Power Calculations Power Calculations –Two independent proportions –90% Power –75% to 90% Proportion Change in Overall Satisfaction –133 patient encounters in each arm –150 selected as target enrollment

© 2003 By Default! A Free sample background from Slide 23 Methods Randomization Stratified randomization of 26 primary care clinics based on 3 characteristics: Stratified randomization of 26 primary care clinics based on 3 characteristics: –BWH (13) vs. MGH (7) hospital affiliation 6 were free-standing6 were free-standing –Academic (16) vs. Community setting (10) –Low (12) vs. High (14) average patient socioeconomic status Rolling implementation of Results Manager for intervention clinics was completed by March, 2004 Rolling implementation of Results Manager for intervention clinics was completed by March, 2004

© 2003 By Default! A Free sample background from Slide 24 Methods Study Criteria Patients were randomly sampled during eligible pre and post intervention time periods Patients were randomly sampled during eligible pre and post intervention time periods Inclusion Criteria Inclusion Criteria –All patients in participating clinics who had any of the following tests: ChemistryChemistry HematologyHematology PathologyPathology MicrobiologyMicrobiology RadiologyRadiology Exclusion Criteria Exclusion Criteria –Primary care physician determined that patient should not be contacted

© 2003 By Default! A Free sample background from Slide 25 Methods Data Collection Telephone Administration of Survey Telephone Administration of Survey –Internally developed survey –Trained research assistants –Administered 5 to 7 weeks after test result posting date –Up to three attempts were made to contact each patient

© 2003 By Default! A Free sample background from Slide 26 Methods Survey Outcomes (except expectations) were measured by the Likert scale: Outcomes (except expectations) were measured by the Likert scale: –Strongly Agree –Agree –Neither Agree nor Disagree –Disagree –Strongly Disagree All results were dichotomized All results were dichotomized

© 2003 By Default! A Free sample background from Slide 27 Methods Survey Primary Outcome Measure Primary Outcome Measure –Overall satisfaction with test result communication “I am satisfied with the way test results are communicated to me”“I am satisfied with the way test results are communicated to me”

© 2003 By Default! A Free sample background from Slide 28 Methods Survey Secondary Outcome Measures Secondary Outcome Measures –Satisfaction with PCP listening skills “My primary care doctor always listens to my concerns”“My primary care doctor always listens to my concerns” –Satisfaction with information given about treatment and condition “My primary care doctor gives me as much information about my condition and treatment as I wanted”“My primary care doctor gives me as much information about my condition and treatment as I wanted” –Satisfaction with general PCP communication “My primary care doctor and I communicate very well”“My primary care doctor and I communicate very well”

© 2003 By Default! A Free sample background from Slide 29 Methods Secondary Outcome Measure Whether a patient’s expectations were met by the method of test result communication was determined by: Whether a patient’s expectations were met by the method of test result communication was determined by: –Test result type: normal / abnormal Defined as requiring follow-up or a management plan changeDefined as requiring follow-up or a management plan change –Method of test result receipt –Patient’s expected delivery method for test –Hierarchy of test result communication Same Visit > Telephone > Letter > > Next Visit > NeverSame Visit > Telephone > Letter > > Next Visit > Never If receipt was by a more desired method, it was countedIf receipt was by a more desired method, it was counted

© 2003 By Default! A Free sample background from Slide 30 Methods Data Analysis Multivariate logistic regression models Multivariate logistic regression models –Generalized estimating equations (SAS 9.1) –Clustered by primary care physician –Adjusted for Patient AgeAge GenderGender RaceRace Insurance statusInsurance status Self-reported healthSelf-reported health

© 2003 By Default! A Free sample background from Slide 31 Methods Data Analysis Use of interaction term in this study design Use of interaction term in this study design –There were 4 study groups –Multiplicative interaction term of (Pre/Post) * (Control/Intervention) can be interpreted as the relative change in outcome between comparison groups from the baseline to the follow-up evaluation Pre/PostPre/Post Control/InterventionControl/Intervention Post*InterventionPost*Intervention –Reported as a p value only Odds Ratios are reported for both the Control arm and for the Intervention arm after full adjustment Odds Ratios are reported for both the Control arm and for the Intervention arm after full adjustment

© 2003 By Default! A Free sample background from Slide 32 Methods Intention-to-treat analysis: All physicians from intervention practices were considered to be in the intervention arm regardless of RM use Intention-to-treat analysis: All physicians from intervention practices were considered to be in the intervention arm regardless of RM use

© 2003 By Default! A Free sample background from Slide 33 Results Demographics Sample Patient Age (Mean Years)57.3 Patient Sex (% Female)70.0 Patient Race (%) White65.6 Black19.1 Other15.3 Patient Insurance (%) Commercial45.1 Medicare/Medicaid53.9 Self Pay1.1 Totals570

© 2003 By Default! A Free sample background from Slide 34 Results Enrollment # Pre Control150 Pre Intervention141 Post Control158 Post Intervention121

© 2003 By Default! A Free sample background from Slide 35 Results Demographics ControlInterventionp Patient Age (Mean Years) Patient Sex (% Female) Patient Race (%) White Black Other Patient Insurance (%) Commercial Medicare/Medicaid Self Pay Totals313257

© 2003 By Default! A Free sample background from Slide 36 Results Survey Administration 1531 attempted to be contacted 1531 attempted to be contacted –20 had incorrect information –35 had numbers that were out of service –706 did not answer –128 refused –8 had poor mentation or were too ill (self-report) –64 requested callback but were unavailable for future contact 570 successfully administered surveys 570 successfully administered surveys –Response Rate: 37% (570/1531)

© 2003 By Default! A Free sample background from Slide 37 Results Responders vs. Non-Responders RespondersNon-RespondersP Patient Age (Years) <0.001 Patient Sex (% Female)399 (70.0%)664 (65.4%)0.066 Patient Race White374 (65.6%)586 (57.7%)0.002 Black109 (19.1%)215 (21.2%)0.364 Other87 (15.3%)215 (21.2%)0.004 Patient Insurance Commercial257 (45.1%)529 (52.1%)0.009 Medicare/Medicaid307 (53.9%)455 (44.8%)<0.001 Self Pay6 (1.1%)32 (3.1%)0.009 Totals

© 2003 By Default! A Free sample background from Slide 38 Results Primary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with TestControl Result CommunicationIntervention

© 2003 By Default! A Free sample background from Slide 39 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Expectation Met RegardingControl Method of CommunicationIntervention

© 2003 By Default! A Free sample background from Slide 40 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with InformationControl Given for TreatmentIntervention And Condition

© 2003 By Default! A Free sample background from Slide 41 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with PCPControl Listening SkillsIntervention

© 2003 By Default! A Free sample background from Slide 42 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with GeneralControl PCP CommunicationIntervention

© 2003 By Default! A Free sample background from Slide 43 Discussion Outcomes improved with Intervention Outcomes improved with Intervention –Patient Satisfaction with Overall Test Result Commutation –Patient Satisfaction with Diagnosis & Treatment Information

© 2003 By Default! A Free sample background from Slide 44 Discussion Intervention included a number of potential workflow improvements Intervention included a number of potential workflow improvements –Tracking of test results ordered by provider, and concise summary page for management –Template-based results letter generator Can imbed actual test results into letterCan imbed actual test results into letter Improve patient-friendly interpretations of resultsImprove patient-friendly interpretations of results –One-click patient contact information

© 2003 By Default! A Free sample background from Slide 45 Discussion Improvements in satisfaction with discussion of diagnosis & treatment suggested that it was a significant factor in improving overall patient satisfaction Improvements in satisfaction with discussion of diagnosis & treatment suggested that it was a significant factor in improving overall patient satisfaction

© 2003 By Default! A Free sample background from Slide 46 Discussion Limitations Generalizibility Generalizibility –Tool custom built within an internally developed outpatient electronic health record (LMR) –Commercial vendors have been quick to adopt successful new functionality –Number and Variety of clinics should mitigate this problem as well

© 2003 By Default! A Free sample background from Slide 47 Discussion Limitations Telephone Survey Bias Telephone Survey Bias –Method of patient contact bias Distrust of Medical System or SurveyorDistrust of Medical System or Surveyor Poor Health, Mentation, or HearingPoor Health, Mentation, or Hearing SES Bias from Lack of Telephone ServiceSES Bias from Lack of Telephone Service Should be the same bias effect across all armsShould be the same bias effect across all arms –Response Rate Clear differences between responders and non- respondersClear differences between responders and non- responders Likely inflate satisfaction in each of the measurements (for both arms)Likely inflate satisfaction in each of the measurements (for both arms)

© 2003 By Default! A Free sample background from Slide 48 Conclusions An automated management system that provides centralized test result tracking and facilitates contact with patients: An automated management system that provides centralized test result tracking and facilitates contact with patients: –improved overall patient satisfaction with communication of test results –Increased patient satisfaction with the discussion of treatments/conditions

© 2003 By Default! A Free sample background from Slide 49 Acknowledgements Co-Authors Co-Authors –Tejal K. Gandhi, MD MPH –John Orav, PhD –Zahra Ladak-Merchant, BDS MPH –David W. Bates, MD MS –Gilad J. Kuperman, MD PhD –Eric G. Poon, MD MPH Funding Funding –AHRQ U18-HS –NLM T15-LM-07092

© 2003 By Default! A Free sample background from Slide 50 Michael Matheny, MD MS Brigham & Women’s Hospital Thorn Francis Street Boston, MA Michael Matheny, MD MS Brigham & Women’s Hospital Thorn Francis Street Boston, MA The End