1. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 1 Impact of an Automated Test Results Management System on Patients’ Satisfaction of Test Result Communication Michael E. Matheny, MD MS Biomedical Informatics Fellow Brigham & Women’s Hospital, Boston, MA

2. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 2 Lexington, KY University of Kentucky University of Kentucky –Chemical Engineering Undergraduate –Medical School

3. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 3 Indianapolis, IN Internal Medicine Residency Internal Medicine Residency –St. Vincent Hospital

4. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 4 Boston, MA Massachusetts Institute of Technology Massachusetts Institute of Technology –Master of Science Biomedical Informatics Biomedical Informatics Fellowship Biomedical Informatics Fellowship –NLM Fellow –Harvard-MIT Health Sciences & Technology Outpatient Urgent Care Outpatient Urgent Care –Brigham & Women’s Hospital

5. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 5 This Study Strong interest in applying clinical decision support to improve patient safety Strong interest in applying clinical decision support to improve patient safety –Clinical Reminders –Test Results Management AHRQ Grant (David Bates) AHRQ Grant (David Bates) Results Manager Intervention (Eric Poon) Results Manager Intervention (Eric Poon)

6. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 6 Background Test Result Communication Test result communication between patients and physicians is a critical part of the diagnostic and therapeutic process Test result communication between patients and physicians is a critical part of the diagnostic and therapeutic process However, follow-up of test results in the primary care setting is often challenging: However, follow-up of test results in the primary care setting is often challenging: –High volume of test results –Test results arrive when physician not focused on the patient –Lack of systems to ensure reliability and efficiency

7. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 7 Background Test Result Communication Problems 31% of women with abnormal mammograms did not receive care consistent with established guidelines 31% of women with abnormal mammograms did not receive care consistent with established guidelines 39% of abnormal TSH at BWH were not followed up within 60 days 39% of abnormal TSH at BWH were not followed up within 60 days 36% of abnormal pap smear were lost to follow-up 36% of abnormal pap smear were lost to follow-up

8. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 8 Background Physician Workflow 33% of physicians reported they did not always notify patients of abnormal test results 33% of physicians reported they did not always notify patients of abnormal test results ~30% of physicians reported they did not have a reliable method of test result communication ~30% of physicians reported they did not have a reliable method of test result communication 59% of physicians were dissatisfied with how well they managed test results despite spending over an hour a day in this activity 59% of physicians were dissatisfied with how well they managed test results despite spending over an hour a day in this activity

9. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 9 Background Patient Expectations Patients do not normally discuss their preferences for test result notification with their providers Patients do not normally discuss their preferences for test result notification with their providers Patients preferred telephone notification to regular mail, and found electronic notification to be uncomfortable due to security issues Patients preferred telephone notification to regular mail, and found electronic notification to be uncomfortable due to security issues Patients wanted to be notified of all test results, regardless of whether the results were abnormal Patients wanted to be notified of all test results, regardless of whether the results were abnormal

10. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 10 Background Patient Satisfaction These problems reduce patient satisfaction with their medical care, and impair future patient-physician interactions These problems reduce patient satisfaction with their medical care, and impair future patient-physician interactions Improving patient satisfaction has been identified as one of the most important issues currently facing healthcare Improving patient satisfaction has been identified as one of the most important issues currently facing healthcare

11. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 11 Background Current IT Environment Increasing numbers of practices are implementing electronic health records in order to improve documentation, billing, and for the promise of improved patient care Increasing numbers of practices are implementing electronic health records in order to improve documentation, billing, and for the promise of improved patient care While test result viewer applications are available in most commercial and home- grown EHRs, they typically require clinicians to separately keep track of their pending tests, and individually check a patient’s chart for the result While test result viewer applications are available in most commercial and home- grown EHRs, they typically require clinicians to separately keep track of their pending tests, and individually check a patient’s chart for the result

12. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 12 Background Current IT Environment A few institutions have implemented automated test result notification systems to physicians, but these systems have generally been deployed only in the inpatient setting for critically abnormal results A few institutions have implemented automated test result notification systems to physicians, but these systems have generally been deployed only in the inpatient setting for critically abnormal results The follow-up workflow in the outpatient setting is very different because patients are not in a controlled environment, which presents both communication and compliance barriers The follow-up workflow in the outpatient setting is very different because patients are not in a controlled environment, which presents both communication and compliance barriers

13. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 13 Objective To evaluate the impact of an EHR-imbedded automated test results notification system on patient satisfaction of test results communication To evaluate the impact of an EHR-imbedded automated test results notification system on patient satisfaction of test results communication

14. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 14 Methods Study Setting Partners HealthCare System Partners HealthCare System –Brigham & Women’s Hospital –Massachusetts General Hospital –Faulkner Hospital –McLean Hospital –Newton-Wellesley Hospital –Free Standing Outpatient Clinics Longitudinal Medical Record (LMR) Longitudinal Medical Record (LMR) –Released July 2000 –Scheduling –Medication lists –Problem lists –Health maintenance record –Clinic notes (free form & templates)

15. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 15 Methods LMR Summary Screen

16. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 16 Methods Study Setting Baseline state of test results management Baseline state of test results management –Test results were embedded directly into the patients’ electronic health record –No automated test results tracking –All test results were mailed to the physician’s clinic office –Physicians were paged directly for critical results

17. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 17 Methods Patient Test Results Screen

18. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 18 Methods Intervention Results Manager - an electronic test results management system embedded into the LMR Results Manager - an electronic test results management system embedded into the LMR –Features: Tracks and displays all test results associated with an ordering physicianTracks and displays all test results associated with an ordering physician Prioritizes by degree of test result abnormalityPrioritizes by degree of test result abnormality Facilitates review of test results in context of patient’s historyFacilitates review of test results in context of patient’s history Generates test result lettersGenerates test result letters Allows clinicians to set reminders for future testingAllows clinicians to set reminders for future testing

19. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 19 Methods Results Manager Summary Screen

20. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 20 Methods Results Manager Letter Generation Screen

21. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 21 Methods Study Design

22. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 22 Methods Enrollment Power Calculations Power Calculations –Two independent proportions –90% Power –75% to 90% Proportion Change in Overall Satisfaction –133 patient encounters in each arm –150 selected as target enrollment

23. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 23 Methods Randomization Stratified randomization of 26 primary care clinics based on 3 characteristics: Stratified randomization of 26 primary care clinics based on 3 characteristics: –BWH (13) vs. MGH (7) hospital affiliation 6 were free-standing6 were free-standing –Academic (16) vs. Community setting (10) –Low (12) vs. High (14) average patient socioeconomic status Rolling implementation of Results Manager for intervention clinics was completed by March, 2004 Rolling implementation of Results Manager for intervention clinics was completed by March, 2004

24. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 24 Methods Study Criteria Patients were randomly sampled during eligible pre and post intervention time periods Patients were randomly sampled during eligible pre and post intervention time periods Inclusion Criteria Inclusion Criteria –All patients in participating clinics who had any of the following tests: ChemistryChemistry HematologyHematology PathologyPathology MicrobiologyMicrobiology RadiologyRadiology Exclusion Criteria Exclusion Criteria –Primary care physician determined that patient should not be contacted

25. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 25 Methods Data Collection Telephone Administration of Survey Telephone Administration of Survey –Internally developed survey –Trained research assistants –Administered 5 to 7 weeks after test result posting date –Up to three attempts were made to contact each patient

26. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 26 Methods Survey Outcomes (except expectations) were measured by the Likert scale: Outcomes (except expectations) were measured by the Likert scale: –Strongly Agree –Agree –Neither Agree nor Disagree –Disagree –Strongly Disagree All results were dichotomized All results were dichotomized

27. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 27 Methods Survey Primary Outcome Measure Primary Outcome Measure –Overall satisfaction with test result communication “I am satisfied with the way test results are communicated to me”“I am satisfied with the way test results are communicated to me”

28. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 28 Methods Survey Secondary Outcome Measures Secondary Outcome Measures –Satisfaction with PCP listening skills “My primary care doctor always listens to my concerns”“My primary care doctor always listens to my concerns” –Satisfaction with information given about treatment and condition “My primary care doctor gives me as much information about my condition and treatment as I wanted”“My primary care doctor gives me as much information about my condition and treatment as I wanted” –Satisfaction with general PCP communication “My primary care doctor and I communicate very well”“My primary care doctor and I communicate very well”

29. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 29 Methods Secondary Outcome Measure Whether a patient’s expectations were met by the method of test result communication was determined by: Whether a patient’s expectations were met by the method of test result communication was determined by: –Test result type: normal / abnormal Defined as requiring follow-up or a management plan changeDefined as requiring follow-up or a management plan change –Method of test result receipt –Patient’s expected delivery method for test –Hierarchy of test result communication Same Visit > Telephone > Letter > Email > Next Visit > NeverSame Visit > Telephone > Letter > Email > Next Visit > Never If receipt was by a more desired method, it was countedIf receipt was by a more desired method, it was counted

30. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 30 Methods Data Analysis Multivariate logistic regression models Multivariate logistic regression models –Generalized estimating equations (SAS 9.1) –Clustered by primary care physician –Adjusted for Patient AgeAge GenderGender RaceRace Insurance statusInsurance status Self-reported healthSelf-reported health

31. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 31 Methods Data Analysis Use of interaction term in this study design Use of interaction term in this study design –There were 4 study groups –Multiplicative interaction term of (Pre/Post) * (Control/Intervention) can be interpreted as the relative change in outcome between comparison groups from the baseline to the follow-up evaluation Pre/PostPre/Post Control/InterventionControl/Intervention Post*InterventionPost*Intervention –Reported as a p value only Odds Ratios are reported for both the Control arm and for the Intervention arm after full adjustment Odds Ratios are reported for both the Control arm and for the Intervention arm after full adjustment

32. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 32 Methods Intention-to-treat analysis: All physicians from intervention practices were considered to be in the intervention arm regardless of RM use Intention-to-treat analysis: All physicians from intervention practices were considered to be in the intervention arm regardless of RM use

33. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 33 Results Demographics Sample Patient Age (Mean Years)57.3 Patient Sex (% Female)70.0 Patient Race (%) White65.6 Black19.1 Other15.3 Patient Insurance (%) Commercial45.1 Medicare/Medicaid53.9 Self Pay1.1 Totals570

34. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 34 Results Enrollment # Pre Control150 Pre Intervention141 Post Control158 Post Intervention121

35. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 35 Results Demographics ControlInterventionp Patient Age (Mean Years)57.157.70.648 Patient Sex (% Female)64.676.30.003 Patient Race (%) White65.965.30.929 Black19.1 1.000 Other14.915.70.816 Patient Insurance (%) Commercial48.441.20.092 Medicare/Medicaid50.657.60.109 Self Pay1.01.11.000 Totals313257

36. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 36 Results Survey Administration 1531 attempted to be contacted 1531 attempted to be contacted –20 had incorrect information –35 had numbers that were out of service –706 did not answer –128 refused –8 had poor mentation or were too ill (self-report) –64 requested callback but were unavailable for future contact 570 successfully administered surveys 570 successfully administered surveys –Response Rate: 37% (570/1531)

37. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 37 Results Responders vs. Non-Responders RespondersNon-RespondersP Patient Age (Years)57.452.3<0.001 Patient Sex (% Female)399 (70.0%)664 (65.4%)0.066 Patient Race White374 (65.6%)586 (57.7%)0.002 Black109 (19.1%)215 (21.2%)0.364 Other87 (15.3%)215 (21.2%)0.004 Patient Insurance Commercial257 (45.1%)529 (52.1%)0.009 Medicare/Medicaid307 (53.9%)455 (44.8%)<0.001 Self Pay6 (1.1%)32 (3.1%)0.009 Totals5701016

38. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 38 Results Primary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with TestControl90820.6820.345-1.3470.012 Result CommunicationIntervention85932.3801.068-5.306

39. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 39 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Expectation Met RegardingControl61470.4690.326-0.6740.001 Method of CommunicationIntervention44581.3940.827-2.350

40. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 40 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with InformationControl95870.7110.261-1.9370.034 Given for TreatmentIntervention94963.6711.300-10.37 And Condition

41. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 41 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with PCPControl99931.0240.062-16.790.501 Listening SkillsIntervention99982.9620.760-11.54

42. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 42 Results Secondary Outcome GroupPrePostOR95% CIInteraction p Satisfaction with GeneralControl96911.3730.404-4.6680.367 PCP CommunicationIntervention97962.8600.957-8.547

43. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 43 Discussion Outcomes improved with Intervention Outcomes improved with Intervention –Patient Satisfaction with Overall Test Result Commutation –Patient Satisfaction with Diagnosis & Treatment Information

44. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 44 Discussion Intervention included a number of potential workflow improvements Intervention included a number of potential workflow improvements –Tracking of test results ordered by provider, and concise summary page for management –Template-based results letter generator Can imbed actual test results into letterCan imbed actual test results into letter Improve patient-friendly interpretations of resultsImprove patient-friendly interpretations of results –One-click patient contact information

45. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 45 Discussion Improvements in satisfaction with discussion of diagnosis & treatment suggested that it was a significant factor in improving overall patient satisfaction Improvements in satisfaction with discussion of diagnosis & treatment suggested that it was a significant factor in improving overall patient satisfaction

46. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 46 Discussion Limitations Generalizibility Generalizibility –Tool custom built within an internally developed outpatient electronic health record (LMR) –Commercial vendors have been quick to adopt successful new functionality –Number and Variety of clinics should mitigate this problem as well

47. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 47 Discussion Limitations Telephone Survey Bias Telephone Survey Bias –Method of patient contact bias Distrust of Medical System or SurveyorDistrust of Medical System or Surveyor Poor Health, Mentation, or HearingPoor Health, Mentation, or Hearing SES Bias from Lack of Telephone ServiceSES Bias from Lack of Telephone Service Should be the same bias effect across all armsShould be the same bias effect across all arms –Response Rate Clear differences between responders and non- respondersClear differences between responders and non- responders Likely inflate satisfaction in each of the measurements (for both arms)Likely inflate satisfaction in each of the measurements (for both arms)

48. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 48 Conclusions An automated management system that provides centralized test result tracking and facilitates contact with patients: An automated management system that provides centralized test result tracking and facilitates contact with patients: –improved overall patient satisfaction with communication of test results –Increased patient satisfaction with the discussion of treatments/conditions

49. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 49 Acknowledgements Co-Authors Co-Authors –Tejal K. Gandhi, MD MPH –John Orav, PhD –Zahra Ladak-Merchant, BDS MPH –David W. Bates, MD MS –Gilad J. Kuperman, MD PhD –Eric G. Poon, MD MPH Funding Funding –AHRQ U18-HS-11046 –NLM T15-LM-07092

50. © 2003 By Default! A Free sample background from www.powerpointbackgrounds.com Slide 50 Michael Matheny, MD MS mmatheny@dsg.harvard.edu Brigham & Women’s Hospital Thorn 309 75 Francis Street Boston, MA 02115 Michael Matheny, MD MS mmatheny@dsg.harvard.edu Brigham & Women’s Hospital Thorn 309 75 Francis Street Boston, MA 02115 The End