1. The OpenNotes Patient and Family Reporting Tool:
Engaging Patients and Families as Partners in Safety and Quality
Technologies to Engage Consumers in Information Management
S04
Fabienne Bourgeois MD, MPH
Boston Children’s Hospital
Twitter: #AMIA2017

2. Disclosure
I and my spouse/partner have no relevant relationships with commercial interests to disclose. Funding: CRICO Quality Grant
AMIA | amia.org

3. Learning Objectives
After participating in this session the learner should be better able to:
Understand the benefits of allowing patients and families to provide feedback on visit notes
Understand the types of safety concerns most frequently identified by patients and families in visit notes
Understand the impact of patient/family feedback on prompting updates to the medical record or care
AMIA | amia.org

4. Background OpenNotes Initiative Launched in 2010
Leverage patient portals to provide online access to visit notes
Increasing push toward transparency
Empower patients and families to engage in their care by reviewing visit notes
Goal of 50 million Americans with access to visit notes by 2018
Currently ~19 million patients with access nationwide
AMIA | amia.org

5. Background How does access to notes impact care?
Delbanco, T et al. Ann Intern Med. 2012
3 Open Notes sites (MA, PA and WA)
82% of patients opened at least 1 note (84% at BIDMC, 82% at GHS, and 47% at HMC).
77% to 87% reported feeling more in control of their care
60% to 78% of those taking medications reported increased medication adherence
1% to 8% reported that the notes caused confusion, worry, or offense
99% of patients wanted open notes to continue and no doctor elected to stop
AMIA | amia.org

6. Background
Can patients and families help improve the accuracy and quality of visit notes?
Difficult to identify and quantify ambulatory safety events
Few processes to recognize documentation errors
Invite patients and families to identify inaccuracies in notes
AMIA | amia.org

7. Background Bell SK, et al. BMJ Qual Saf 2016
6225 adult patients seen at 2 primary care practices
Invited to provide feedback on their online notes
Survey link at the bottom of the notes
260 reports submitted by patients
Reviewed by Patient Relations Team
23% reported a potential safety concern
Inaccuracies in the note or inability to understand the care plan
64% of reports found to be a definite or possible safety concern
57% of concerns resulted in a change to the medical record or care
99% of the patients found the tool valuable
Improved or no change in patient-provider relationship
AMIA | amia.org

8. Objective
To quantify and characterize patient and parent reports of potential documentation inaccuracies and safety concerns among ambulatory primary care and subspecialty visit notes available through online patient portals at 2 urban academic hospitals
AMIA | amia.org

9. Study Design Design: Setting
Prospective cohort quality improvement initiative
Setting
2 Urban academic hospitals:
Adult medical center (Beth Israel Deaconess Medical Center)
Children’s medical center (Boston Children’s Hospital)
AMIA | amia.org

10. Study Design Participants BIDMC BCH
Adult patients with active patient portal accounts and
Eligible ambulatory visit note (starting 8/2014)
2 primary care practices and OG/GYN practices
BCH
Parents/legal guardians and patients ≥13 years of age with
Active patient portal accounts and
Eligible ambulatory visit note (starting 6/2016)
Primary care and Subspecialty clinic notes
Except:
Confidential ambulatory clinic notes
High risk clinics: Adolescent, Gynecology, Adolescent Substance Abuse Program, Child Protection Team, Psychiatry
Parents of minor children and legal guardians of adult patients
Higher risk of managing confidential information and notes are defaulted to confidential
AMIA | amia.org

11. Intervention Reporting Tool
Co-developed with members of the Patient and Family Advisory Committee
10 item online survey:
How well did the note describe the visit
How well did the respondent understand the care plan
Were there any inaccuracies in the note
Was there any language that the respondent found bothersome
Voluntary positive feedback for the provider
Reports followed a QI algorithm
Reviewed and managed in real-time by Patient Relations representatives
AMIA | amia.org

12. Outcomes Potential Safety Event
“No” response to the query related to understanding or following the care plan
“Yes” or “Unsure” response to the query related to finding a possible inaccuracy
Safety Concerns were reviewed by Patient Relations
Coded based on outcomes
Amendment made to medical record or care
Amendment declined by provider
Referred to patient relations
Intervention declined by patient
AMIA | amia.org

13. Results 1145 Reporting tool surveys submitted BIDMC : 741 surveys
22% reported possible inaccuracies
7% reported challenges with the care plan
7% reported bothersome language
72% provided positive feedback
BCH: 404 surveys
36% reported possible inaccuracies
9% reported challenges with the care plan
8% reported bothersome language
42% provided positive feedback
94% complete by parents, 6 % by patients
AMIA | amia.org

14. Results Types of Inaccuracies BCH N=217 % BIDMC N=247 Medications 9.6
16.6
Description of Chief Complaint 11
14.2
Problem List
14.7
Social History
2.3 4
Family History
8.3
7.3
Description of Physical Exam 6
Other Missing Information
10.6
13.8
Names of Healthcare Providers
0.8
Appointment Scheduling
2.8
0.4
Other
27.2 19
AMIA | amia.org

15. Results Safety Concern Outcome BCH N=87 N (%) BIDMC N=99
Amendment Made to Medical Record or Care
55 (63)
77 (79)
Amendment to Medical Record Declined
8 (9)
3 (3)
Deferred to Patient Relations
5 (6)
15 (15)
Intervention Declined by Patient
 8 (9)
3 (3) 
No Response
13 (15)
1 (1)
22% of reports with safety concern : BCH
13% with safety concern BIDMC
AMIA | amia.org

16. Feedback for Providers
“Dr. Y was great.  My first visit with her: I felt as though she listened thoroughly and treated me with great respect.”
“AMAZING appt. The ease of testing was greatly in part to the attention and entertainment from the doctors. Your confidence and knowledge is obvious-it left us feeling confident and concrete on our dx and decisions for future care. You all were meant to work with children--we are grateful we found you!”
“Dr Z communicates in a way that is caring and informative and motivating.”
“Dr […] was so thorough during the exam and took in so much information regarding my daughter's health issues. She was superb.”
AMIA | amia.org

17. Feedback about Notes
“I love this feature of being able to view the note! It perfectly summarized the visit and was nice to be able to reference and share with my husband who wasn't able to make the appointment. I appreciate Dr […] taking the time to write such a detailed update.”
“We are so grateful to have the notes to review of [Patient’s] apts It is overwhelming as a parent to absorb all of the information at the time of the visit so the notes are very helpful to review.”
“We are so grateful to have the notes to review of [Patient’s] apts It is overwhelming as a parent to absorb all of the information at the time of the visit so the notes are very helpful to review.”
AMIA | amia.org

18. Conclusions
Patients and Parents provide important feedback on documentation inaccuracy
Few patients or parents were bothered by language in the notes
Patients and Parents provided positive feedback regardless of reported inaccuracies
Patients and Parents are important partners in improving the quality and safety of ambulatory care
AMIA | amia.org

19. Project Teams
BIDMC:
Sigall Bell
Tom Delbanco
Jan Walker
Alan Fossa
Macda Gerard
Patricia Folcarelli
BCH: Marvin Harper Dan Nigrin Dianne Arnold Monique Mello Jenny Sadler Irene Chen Elizabeth Bennett Amanda Martino
AMIA | amia.org

20. Thank you!