1. Chart Reviews Improvement Facilitator Training Session 1 Day 2

2. Chart Review - Overview Create a shared understanding of the purpose of the chart review and how it differs from the Model for Improvement Describe the timeline and process of the chart review Respond to providers concerns about privacy Describe measures of success List next steps and supports related to chart reviews 2

3. Why Chart Review Analyzes current performance Provides information about documentation practices Does not require provider have EMR Does not require documentation standards to be in place prior to review Ability to aggregate results (de-identified) 3

4. Purpose of Chart Review What is the purpose of a chart review? A chart review intends to measure current performance against guideline care to identify opportunities for improvement 4

5. The ASaP Intervention 5 Panel Identification Focused Improvement Build on Success Document process to ID patient/provider attachment Generate patient lists for screening Choose + document screening methods: opportunistic and/or outreach Choose + document screening maneuvers Define & document team roles & responsibilities Test small change (PDSA) Standardize processes Measure reliability of processes Apply for CME credits Identify other clinical improvement opportunities Baseline Chart Review & Current Screening Process Assessment 4-Month Follow Up Chart Review & Screening Process Assessment Sustainability Chart Review & Continued Follow-up Reviews 30 days 60 days Ongoing

6. Process of Chart Review 1.Consent 2.Selecting a Bundle 3.Agreeing on standards 4.Collecting Data 5.Analysing Data against standards 6.Feeding back results 7.Implementing changes 8.Allowing time for changes to embed before re-review 9.Collect second set of data 10.Analyse second set of data 11.Feeding back second set of results 12.Repeat 7 – 11 as needed 6

7. Chart Review - Consent 7

8. Chart Review – Health Information Act 8

9. Privacy and Confidentiality - Implications Provider Data – Information about the provider is shared with TOP to support the PCO with the ASaP program – Collection, Usage, and Reporting of this information is covered in the consent process – Personally Identifiable Data will be shared between the provider, the PCO, and TOP CPA – De-identified data will be aggregated for the purposes of PCO reporting 9

10. Privacy and Confidentiality - Implications Patient Data – Patient consent is not required – No identifiable patient information will be recorded or shared – Identifiable patient information will be viewed Chart reviewers will need to assess their organizations Privacy Impact Assessment to determine if this activity has been documented Affiliate Agreement may be necessary if not addressed in the PIA Oath of Confidentiality endures beyond the ASaP initiative 10

11. Chart Review – Selecting Bundle Identification of Clinical Maneuvers by provider 11

12. Chart Review – Selecting Bundle Table of sampling error by number of maneuvers and charts reviewed Key Message:  # of charts reviewed and/or  # of maneuvers increases precision Key Message: Number of chart reviews determined by PCN or other primary care organization Number of Maneuvers 10 Charts20 Charts30 Charts 56.0 – 13.84.2 – 9.73.5 – 7.9 84.8 – 10.93.4 – 7.72.7 – 6.3 104.3 – 9.83.0 – 6.92.4 – 5.6 123.0 – 8.92.7 – 6.32.2 – 5.4 12

13. Agreeing on Standards Screening and Prevention of a patient is dependent on three factors – Patient compliance – Ability of health system to perform – Provision of offer 13

14. ASaP – Agreeing on Standards In Alberta, what are some activities focused on improving screening and prevention for… – Patient compliance? – Ability for health system to perform? – Provision of offer? 14

15. ASaP – Agreeing on Standards 15 ManeuverAge (years)Interval Blood Pressure18+Annual Weight18+Annual Height18+Once lifetime Exercise Assessment18+Annual Tobacco Use Assessment18+Annual Alcohol Use Assessment18+Annual Influenza Vaccination/ Screen18+Annual Pap Test Females 21 – 69 3 years Plasma Lipid Profile Males 40 – 74 Females 50 – 74 3 years CV Risk Calculation Males 40 – 74 Females 50 – 74 3 years Diabetes Screen One of: - Fasting Glucose - Hgb A1c - Diabetes Risk Calculator 40+3 years Colorectal Cancer Screen One of: - FOBT/FIT - Flex Sigmoidoscopy - Colonoscopy 50 – 74 2 years 5 years 10 years MammographyFemales 50 - 69 (74*)2 years

16. Collecting Data 16

17. Initial Chart Review Does the provider have a panel process – YES, then sample entire panel by creating a list of patients and review records by patient last name starting with the random letter skipping identical last names already reviewed. – NO, then chart audit must be delayed until a panel process is established for at least 30 days. – MAYBE, if the provider can generate a partial list of patients that have been confirmed using their new process for a portion of the entire panel sample this list otherwise see NO 17

18. Collecting Data - Bias Peter and Mary both work for the Rocky Peaks Primary Care Network as chart reviewers. Peter reviews 30 of Dr. Doe’s charts (patients of Dr. Doe aged 18 and older) and finds that all the charts (30/30) have a tobacco use assessment charted in the record. Mary reviews the same 30 records and finds that Dr. Doe has a documented assessment for tobacco use on 12 out of the 30 records reviewed. What could explain the difference? BMFC  18

19. Collecting Data - Bias How could we reduce these sources of error? 19

20. Collecting Data – Reducing Bias Reviewer Bias – Preferably the same reviewer for initial and follow up chart reviews – Preferably one or few reviewers for the primary care organization (e.g. PCN) – Availability of realistic examples and indications of documented offers or exemptions – Single point of contact for questions not addressed Outcome Bias – Record only documented actions – Record only actions which have occurred (not intended future actions) 20

21. Collecting Data – Bias Peter and Mary both work for the Rocky Peaks Primary Care Network as chart reviewers. Dr. Doe from the Blue Meadow Family Clinic has agreed to participate in ASaP. Peter has reviewed the charts of Dr. Green already but Peter will be on vacation for 3 weeks during the planned follow up chart review time for Dr. Doe. Who should review Dr. Doe’s charts? BMFC  21

22. Collecting Data – Bias Peter reviews 30 patients of Dr. Doe who presented to the clinic for a complete medical exam between January and February of 2013. Dr. Doe wants to know about his performance in screening for his entire panel. What types of bias are present in Peter’s method? BMFC  22

23. Collecting Data – Reducing Bias How could we reduce the sources of error in Peter’s approach? 23

24. Collecting Data – Reducing Bias Selection Bias – Sample drawn from the whole – Randomization of selection – Familial Bias http://www.randomlettergenerator.com Selection Bias – Sample drawn from the whole – Randomization of selection – Familial Bias 24

25. Collecting Data Doctor Doe at Blue Meadows clinic has 2238 patients according to his EMR. He started a Panel Process 30 days ago as prework for the ASaP initiative. He has 345 patients on his panel since starting this new process. Peter reviewed patients from the panel listing of 2238 and found a screening rate of 22% and then reviewed only patients on the shorter 345 patient list and found a screening rate of 38%. What would account for the differences in screening rates and which panel list should he use? BMFC  25

26. Collecting Data You are reviewing the chart of Sally a 55 year old patient of Dr. Doe for Mammography. There is no notation in the encounter notes that she was offered mammography in the past 2 years. The scanned documents does not have a requisition for ‘mammo’ either. Though you note a surgical report for bilateral mastectomy 5 years ago. How do you record this information? Would the answer be different if this information was recorded in surgical history? BMFC  26

27. Chart Review – What does an ‘Offer’ look like? Completion Decline Exclusion/Exemption 27

28. Data Collection – Evidence of Offer 28

29. Screening Offers Documentation 29 BMFC 

30. Chart Review – Data Collection & Analysis 30

31. Chart Review Report 31

32. Chart Review Report Dr. Doe calls you and wants to know what the 38% screening rate means. What is the denominator? – We randomly selected 30 patient records from your list of patients who were between 18 and 79 and reviewed their record for the 11 maneuvers you selected. What is the numerator? – We looked for documented evidence that the screening and prevention for all maneuvers were addressed which included: results, patient declines, and exclusion/exemptions within the appropriate time frame. 32

33. Chart Review Report – Key Messages How can I improve? – Goal is not 100% – Based on ‘GUIDELINE’ care which has exceptions – Records ‘DOCUMENTED’ actions not ‘actual’ actions – What do you (provider/team) think can be improved? – Comment on what was observed E.g. It was difficult to find exclusions E.g. Patients with chronic disease were not screened 33

34. The ASaP Intervention 34 Panel Identification Focused Improvement Build on Success Document process to ID patient/provider attachment Generate patient lists for screening Choose + document screening methods: opportunistic and/or outreach Choose + document screening maneuvers Define & document team roles & responsibilities Test small change (PDSA) Standardize processes Measure reliability of processes Apply for CME credits Identify other clinical improvement opportunities Baseline Chart Review & Current Screening Process Assessment 4-Month Follow Up Chart Review & Screening Process Assessment Sustainability Chart Review & Continued Follow-up Reviews 30 days 60 days Ongoing 60 day wait

35. Chart Review – Follow up Sampling Opportunistic Method If the provider used an opportunistic approach to offer screening: – Identify the patients who could have been offered screening by selecting panel patients who were seen between chart review (follow up) and completion of the changes. 35

36. Chart Review – Follow up Dr. Green has a panel of 2153 patients. The team implemented an opportunistic approach to screening and finished implementing changes on May 18 th. How would we create a list of patients to sample for follow up chart review? 36 BMFC 

37. Chart Review – Follow up Outreach Method If the provider used an outreach method to offer screening: – Identify the group of patients that outreach was attempted with by having the provider or team recreate the list of patients who were planned for contact If both methods (Outreach and Opportunistic) were used then select half the sample size from each method (e.g. 10 Opportunistic and 10 Outreach records reviewed) 37

38. Chart Review – Outreach Sampling Dr. Smith has selected 5 maneuvers (Blood Pressure, Height/Weight, Tobacco Use, Colorectal Cancer, and Cardiac Risk) but due his large panel size he has chosen to generate a list of patients who are 50 years old and last name begins with A, B, C, or D. You are preparing for the chart review by generating the list of patients to sample. What is the criteria you would use/ask the superuser to create the list? 38

39. Chart Review – Next Steps Confirm who is conducting chart reviews in your organization Determine if your organization has a Privacy Impact Assessment which contemplates chart reviews Confirm number of charts to be reviewed Complete Maneuvers Checklist Complete Screening Offers Documentation in Clinic for each provider/clinic 39

40. Chart Review Summary Chart reviews provide an objective measure of how the providers system for screening is performing The follow up chart review will occur approximately 4 months after the initial chart review and include patients which the new process could have affected Success is described as an offer of screening, completed result, documented decline, or documented exclusion within the appropriate timeframe Identifiable provider data will be shared with the IF +/- CPA and de-identified data will be aggregated 40