Developing a Metric Set for Measuring and Reporting Ambulatory Quality of Care in the Setting of Health IT with HIE Lisa M. Kern, MD, MPH Rina V. Dhopeshwarkar, MPH Rainu Kaushal, MD, MPH HITEC Cornell University New York-Presbyterian Hospital HITEC September 2008

Background ¾ of states are pursuing development and implementation of health information exchange (HIE) New York State is investing $250 million in infrastructure for health information technology (health IT) and HIE –Largest state-based investment of taxpayer dollars

HEAL NY Program HEAL NY: Healthcare Efficiency and Affordability Law for New Yorkers Capital Grants Program –Funding distributed in waves: 1 st wave in nd wave in 2008 –Projects include both health IT and HIE –Grantees were required to evaluate the effects of their projects

HITEC: The Health Information Technology Evaluation Collaborative A formal collaborative of 4 universities in New York (Cornell, Columbia, SUNY Albany, University of Rochester) Established with the endorsement of the New York State Department of Health Established to conduct rigorous evaluations of HEAL NY projects in order to maximize learning and produce generalizable results

How do you measure the impact on health care quality of health IT with HIE?

Limitations of Existing Metric Sets Existing metric sets developed to evaluate the quality of healthcare delivered in an ambulatory care setting: –Rely on manual chart review (expensive and laborious) –Claims data (lack clinical nuance) –Do not presume communication between health care providers –Not designed to take into account incremental effect of receiving clinical data from outside sources

Specific Aims 1.Develop a modified set of quality metrics that can be retrieved electronically and is sensitive to the types of improvements in quality that health IT with HIE may contribute to the ambulatory care setting 2.Validate the modified quality metrics set through review by a panel of national experts in quality measurement and national experts in HIE 3.To test the reliability of electronic retrieval of the modified quality metrics set, by comparing electronic retrieval to manual retrieval 4.To evaluate the long-term effects of using health IT with HIE on improving health care quality, using the modified metric set

8 Conceptual Framework Electronic Reporting Clinical Data Residing Elsewhere Quality Report Electronic Receipt of Clinical Data by Health Care Provider with an EHR at or Before the Point of Care Medical Decision Making HIE “Sensitivity to EHR with HIE” “Suitability for Electronic Reporting”

9 Overall Methodology in Brief 1.Conduct a literature review for existing ambulatory care quality metric sets. 2.Determine if any of the metrics retrieved should be excluded. 3.Articulate the domains and assumptions upon which each metric would be rated. 4.Rate the existing metrics. 5.Develop novel metrics.

10 Inclusion Criteria for Metric Sets Included metric sets had to be: –Endorsed by A national quality organization, A national professional organization, or A national research organization, OR –Specifically address quality of transitions across health care settings Included metric sets could be general or disease-specific

11 Exclusion Criteria Not in the ambulatory setting –Emergency department care was excluded. Not adult primary care –Obstetrics, pediatrics, cancer care and HIV care were excluded. Provider, practice or health plan characteristics Satisfaction or experience of patients or providers

12 Metric Selection (continued) 17 metric sets = 1064 metrics 925 metrics 502 metrics 139 Duplicates 423 Excluded

13 Rating Process: Round One Each metric was reviewed by 2 raters on 2 dimensions, each on a scale from 0-6 –Impact of HIE on medical decision making –Suitability for electronic reporting Ratings were summed across dimensions and averaged across raters 59 metrics scored high (≥9 out of 12)

14 Rating Process: Round Two Each metric was reviewed by several raters on 5 dimensions, each on a scale from 0-6 –Feasibility of delivering data electronically –Impact on medical decision making –Clinical importance –Feasibility of reporting data electronically –Global rating (4-7 raters for each metric) Ratings were averaged across raters for the global rating 18 scored high (≥4 out of 6)

15 Diseases Represented by Top- Scoring Existing Metrics Asthma (1 metric) Cardiovascular Disease (3) Congestive Heart Failure (1) Diabetes (4) Medication/Allergy Management (2) Mental Health (1) Osteoporosis (1) Prevention (5)

16 Novel Metrics Developed through an iterative process with national quality experts Cover topics related to efficiency and coordination of care –Test Ordering (3 metrics) –Medications (4) –Referrals (2) –Revisits (3)

17 Next Step Test reliability of electronic reporting vs. manual chart review for selected existing metrics and for novel metrics

19 Assumptions for Determining EHR+HIE Sensitivity We assumed the perspective of a primary care physician who has the following characteristics: –Is board-certified and competent –Has been in a community-based practice x 10 years –Has a relatively stable panel of patients –Has an electronic health record (EHR), which is linked only to generalist partners –Is in a practice with the technical capacity to participate in an HIE

20 Factors Relevant for Rating EHR+HIE Sensitivity 1.Whether needed data elements would be missing in the absence of HIE (Relevance) 2.Ease of electronic transmission of data elements to the provider (Feasibility) 3.Impact of electronic transmission (Impact) on:  Processes of care and/or patient outcomes  Utilization

21 Factors Relevant for Rating Suitability for eReporting 1.How commonly this metric appears in other quality metric sets (Importance) 2.How often the data needed for this metric are currently structured (Feasibility) 3.If data are not currently structured, how easy would it be technically to create a structured format (Feasibility)

22 Factors Relevant for Rating Suitability for eReporting 4.How much electronic reporting would rely on providers’ style of documentation (Physician burden) 5.How valid an electronically reported version of this metric would be (Validity)