Quality Improvement Measuring Quality of Care with Electronic Clinical Quality Measures (eCQMs) Welcome to Quality Improvement: Measuring Quality of Care with Electronic Clinical Quality Measures.   Today we are going to discuss the important topic of measuring quality through the use of electronic clinical quality measures a.k.a. eCQMs. No doubt, whatever you do in health information technology, you will be producing reports about performance around quality and patient safety. Many of these reports may be reported externally to payers or for public reporting. It is critical to make sure these reports are accurate and as useful as possible. This material (Comp 12 Unit 10) was developed by Johns Hopkins University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number IU24OC000013. This material was updated in 2016 by Johns Hopkins University under Award Number 90WT0005. This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/. Health IT Workforce Curriculum Version 4.0

Measuring Quality of Care with Electronic Clinical Quality Measures (eCQM) Learning Objectives Review types of quality and safety measures currently in use nationally. Explain the attributes of an effective electronic clinical quality measures (eCQMs) reporting system. Examine the importance of having standardized and structured health information for quality measurement, especially electronic clinical quality measures (eCQMs). Discuss the role of HIT standards and terminologies in electronic clinical quality measures. Discuss how HIT can facilitate data collection and reporting for improving quality of care and patient safety. Describe data quality issues in electronic measures. This unit will introduce the learner to different types of quality measures in use by national programs, including electronic clinical quality measures (eCQMs). We will also discuss how structured data entry and the design of electronic documents and flow sheets have a significant impact on the ability to extract quality measures data from the resulting database(s). The importance of rigorous design and testing of system reports used for quality purposes is emphasized. Sample quality measures that are frequently requested of HIT systems are identified, and questions that guide data extraction are suggested. The objectives for Measuring Quality of Care with eCQMs are to: Understand the various types of quality and safety measures currently in use nationally. Explain the attributes of an effective electronic clinical quality measures (eCQMs) reporting system. Examine the importance of having standardized and structured health information for quality measurement, especially electronic clinical quality measures (eCQMs). Discuss the role of HIT standards and terminologies in electronic clinical quality measures. Discuss how HIT can facilitate data collection and reporting for improving quality of care and patient safety. Describe data quality issues in electronic measures. Health IT Workforce Curriculum Version 4.0

Understanding Different Types of Quality Measurement Based on data sources and types of specifications, the following types of quality measurement are currently in use: Claims. Abstraction. Electronic (eCQM). eCQMs are one of the three main frameworks in which quality measures are developed and implemented. Historically, quality measures used data submitted for billing during the claims filing process. These data generally lacked adequate clinical information for comprehensive quality reporting and could only work in a fee-for-service environment. Retrospective chart abstraction by dedicated staff is rich in clinical content but is expensive and is generally unable to scale. While HIT can be leveraged as a data source for many types of quality measures reporting, eCQMs are an end-to-end completely electronic system, where all components are digitally specified. Health IT Workforce Curriculum Version 4.0

The Promise of eCQMs 10.01 Figure. National Quality Forum, 2011. eCQMs promise increased accuracy (assuming there are no issues with data quality) and greater clinical relevance while reducing the burden of manual data collection through abstraction. Being a completely electronic system, they can scale significantly, as quality measurement is a byproduct of data captured for care delivery. 10.01 Figure. National Quality Forum, 2011. Health IT Workforce Curriculum Version 4.0

eCQM Lifecycle Quality measures, regardless of type, generally follow a lifecycle depicted in this diagram. The eCQM lifecycle is generally similar to other types of quality measures at a high level, however differ significantly in how eCQM specifications are developed, tested, and implemented. This is discussed in more detail in the next several slides. 10.02 Figure. Health IT Workforce Curriculum Version 4.0

Chart Abstraction vs. eCQM: Measurement Specification Development Process Paper-based measure development: Develop measure narrative, numerator/denominator in line with existing administrative data and/or data typically found in patient medical records (these can be paper or electronic charts). Create a list of code sets, data elements, and abstraction definitions to represent the concepts within the measure. Solicit public comment on the measures (program specific). Measure developers conduct complete feasibility, reliability, and validity testing. Measures for use in national programs are generally submitted to the NQF for endorsement. eCQM measure development: Develop measure narrative, numerator/ denominator, workflow, and logic, in line with existing standards (e.g., Blueprint for the CMS Measures Management System [eMeasure Specifications section], Quality Data Model [QDM]). Create value sets, collaborating with the Value Set Authority Center and clinical terminology (e.g., SNOMED-CT, LOINC) stakeholders, as needed. Use the Measure Authoring Tool (MAT) to create the eCQM in Health Quality Measure Format (HQMF). Conduct complete feasibility, reliability, and validity testing, which can include working with EHR vendors to understand data element availability and implementation in the field. Develop the implementation test decks or test cases for the Cypress certification tool. Collaborate with other stakeholders (e.g., Health Level 7 [HL7], the eMeasures Issues Group [eMIG]). Solicit public comment on the measures (program specific). Most quality measures developed for use in national programs have well defined and often NQF (National Quality Forum)-endorsed specifications created by measure development professionals. Since eCQMs are a completely electronic system, eCQM specifications also need to be developed in a standardized and computer-readable format. This slide shows the significant measure development differences between eCQMs and chart abstraction measures. Under the eCQM column, you will see that there are several components that ensure electronic specifications (e-specs) are standardized, to facilitate machine readability without additional human intervention. Health IT Workforce Curriculum Version 4.0

eCQM Specifications XML: Human-readable: Description: a CQM written in Health Quality Measures Format (HQMF) syntax. HQMF is the industry (HL7) standard for representing a CQM as an electronic document. Likely user: EHR system developers and administrators, analysts. Use: to enable the automated creation of queries against an EHR or other operational data store for quality reporting. Human-readable: Description: the human-readable HTML equivalent of the XML file content. Likely user: EHR users. Use: to identify the details of the CQM in a human-readable format, so that the user can understand both how the elements are defined and the underlying logic of the measure calculation. eCQMs specifications are designed to be read by both computers and humans and therefore consist of two components with two distinct use cases. The XML component is designed for “ingestion” by computers. It is used most often used by technical/subject matter expert during eCQM implementation. The human-readable component of the specification is designed for an end user to understand measure intent and application of logic to calculate various eCQM populations (for example: denominators, numerators, exceptions, and exclusions). Health IT Workforce Curriculum Version 4.0

HQMF: Human-Readable Here we see a portion of the human-readable Health Quality Measures Format (HQMF) specification. The reader can read how Boolean logic applied to various data elements (differentiated by font color) and embedded QDM “functions” will produce desired results according to the measures intent when applied to EHR data representing a specific patient population. Excerpt from eCQM Specifications Human Readable file, retrieved March 2012 from CMS’s EHR Incentive Programs eCQM Library: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html Health IT Workforce Curriculum Version 4.0

HQMF: Machine-Readable XML This slide shows the machine-readable XML of the HQMF specification designed to be parsed by a computer and matched up with data from an EHR to score the eCQM results. Excerpt from eCQM Specifications Machine Readable file, retrieved March 2012 from CMS’s EHR Incentive Programs eCQM Library: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html Health IT Workforce Curriculum Version 4.0

eCQM Standards – 1 Measure specification standards: Quality Data Model (QDM). Health Quality Measure Format (HQMF). Measure results reporting standards: Quality Reporting Document Architecture (QRDA). Category I for patient-level data. Category III for aggregate data. eCQMs are highly dependent upon standards used for data collection, measure specification development, and calculation of measure results for reporting. HQMF (Health Quality Measures Format) standard is used to develop eCQM specifications. QDM (Quality Data Model) is a common logical data model that supports development of HQMF specifications using the MAT (Measure Authoring Tool). eCQM “engines” apply HQMF to EHR data to calculate eCQM results. These results are formatted in the QRDA (Quality Reporting Document Architecture) standard for reporting these measure results to internal or external entities. QRDA I formats data at the individual patient level, while QRDA III aggregates data at the measure level. Both these reporting formats are currently in use. Health IT Workforce Curriculum Version 4.0

eCQM Standards – 2 This slide shows how the different eCQM standards work together. The Measure Authoring Tool (MAT) is used by measure developers to generate HQMF specifications. The MAT uses the Quality Data Model and follows all the required conventions. HQMF is then parsed by an eCQM engine and matches up with patient data to generate eCQM results in files according to the Quality Reporting Document Architecture. To view eCQM packages: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html 10.03 Figure. Health IT Workforce Curriculum Version 4.0

Quality Data Model (QDM) Data elements. Relationships. Functions. The QDM is an information model intended to standardize data elements as clearly and consistently defined concepts used in quality measures. This slide shows how the model categorizes data into categories and data types with one or more attributes. The model also defines relationships amongst data elements and functions to allow further specificity/constraints in defining data used in quality measures. The Diagnosis, Active: Diabetes Data Type “value set” shown here is uniquely identified by its OID (Object ID); “members” of the value set are shown in three version specific code sets (SNOMED CT, ICD 9, and ICD 10). This example demonstrates the importance of vocabularies and terminologies from standardized code sets in defining data types and/or attributes as “value sets,” which form building blocks of the Quality Data Elements described in more detail in the next slide. 10.04 Figure. Health IT Workforce Curriculum Version 4.0

Quality Data Elements 10.05 Figure. Quality Data Elements are the basic “atomic” unit of the QDM. These are further defined below: Category: consists of a single clinical concept identified by a value set. Data type: the context in which each category is used to describe a part of the clinical care process. Attribute: provides specific detail about a QDM element. Value set: used to define the set of codes that can possibly be found in a patient medical record for a particular concept. This example is from the Condition/Problem/Diagnosis category. It shows related data types and attributes of the data types — Diagnosis: Active value sets consist of coded lists from standardized code sets/terminologies and are used to provide specificity to data types and attributes. 10.05 Figure. Health IT Workforce Curriculum Version 4.0

eCQM Data Capture Issues Types of Elusive Data Examples of Data Discrete value available in electronic format. Ejection fraction from echocardiogram and PR but usually in devices or standalone special or QT intervals in ECG software systems. Structured data captured but available in different setting of care/EHR system. Ambulatory or long-term care data not available in acute care hospital EHR. Data usually captured on paper and scanned into EHR. Clinician progress notes. Data captured electronically but not as structured. Clinician-transcribed documents, e.g., H&P, consults and discharge summaries. Structured data captured electronically but not as (or mapped to) standardized nomenclature/terminology. Nursing interventions documented as structured data, e.g., application of pneumatic pump for VTE prophylaxis. eCQMs require standardized patient data to generate accurate results. In general, this requires structured data capture of key data elements within the EHR using standardized nomenclature. Structured data captured using local terms needs to be mapped to standardized data elements. Historically, a large amount of patient data is often captured on paper and scanned in or as unstructured text. Conversion of unstructured text into codified structured data using NLP (natural language processing) algorithms offers promise in the future, especially for historical data. Most current EHR implementations are designed to capture at least some of the key data elements as structured data. This slide lists some of the more difficult data capture scenarios. 10.06 Table. Health IT Workforce Curriculum Version 4.0

eCQI Resource Center www.healthit.gov/ecqi-resource-center The eCQI resource center website was created by ONC and CMS to provide a one-stop shop for the most current information on all aspects of eCQMs. It contains content of links to other websites with more specific content areas. It can be accessed through this link: www.healthit.gov/ecqi-resource-center. www.healthit.gov/ecqi-resource-center  eCQI Resource Center [Screen shot]. Retrieved March 2012, from https://ecqi.healthit.gov/ecqm Health IT Workforce Curriculum Version 4.0

USHIK: United States Health Information Knowledgebase The USHIK website provides access to complete eCQM specifications and Meaningful Use legislation. USHIK [Screen shot]. Retrieved March 2012, from https://ushik.ahrq.gov/QualityMeasuresListing?draft=true&system=dcqm&sortField=570&sortDirection=ascending&enableAsynchronousLoading=true Health IT Workforce Curriculum Version 4.0

eCQM Implementation Team Project manager. Quality director/manager. HIT analyst. Database analyst. Vendor support representative. SME consultant. Physician representative. Nurses representative. eCQM implementation requires close collaboration of a multidisciplinary team with representation from quality, IT, clinicians, and EHR/quality reporting software representatives. Subject matter experts with more detailed knowledge of HQMF and HL7 standards may also be required, especially in cases of complicated measures. Health IT Workforce Curriculum Version 4.0

eCQM Infrastructure 10.07 Figure. This slide shows the evolving national eCQM infrastructure to support eCQM development, testing, implementation, and certification of CEHRT quality modules. The components of this system include: QDM: Quality data model, which is the underlying logical data model to support eCQM specifications in Health Quality Measures Format (HQMF). MAT: Measure Authoring Tool is the primary tool used by measure developers to create HQMF-based eCQM specifications. VSAC: the Value Set Authority Center is hosted and maintained at the National Library of Medicine (NLM). It contains all versions of value sets used in eCQMs. VSC: Value Set Harmonization Committee is convened by NQF under contract from ONC to establish guidelines for development and harmonization of high-quality value sets and propose a framework for governance of this process at the national level. ONC Jira: ticketing system maintained by ONC to resolve eCQM-related issues. Cypress: tool used for testing and certification of eCQM products. Bonnie: tool used to create test cases for eCQM measures development and testing. NTC: National Test Collaborative. CMS is facilitating the establishment of a national test collaborative as a public-private partnership to field test eCQMs prior to their release for general use in CMS programs. NQS: National Quality Strategy. Established to prioritize the national quality agenda. 10.07 Figure. Health IT Workforce Curriculum Version 4.0

eCQM Data Quality Factors Structured data capture Coded data captured or mapped Efficient clinician workflows Accurate data extraction eCQM engine programming logic Accuracy of eCQM result depends upon many factors including data captured in structured format with appropriate terminology codes or mapped to terminology codes captured in well designed and efficient clinical documentation workflows. Finally data extraction if needed should ensure all relevant data is pulled properly and finally the eCQM engine should have the correct programming logic conforming to the appropriate standards. Health IT Workforce Curriculum Version 4.0

Measurement is an essential component of quality improvement. Measuring Quality of Care with Electronic Clinical Quality Measures (eCQMs) Summary Measurement is an essential component of quality improvement. HIT has an important role to play in any measurement strategy. eCQMs are designed specifically to maximally leverage HIT and have the potential to revolutionize quality measurement. This concludes Measuring Quality of Care with eCQMs. In summary: Measurement is an essential component of quality improvement. Understanding variation provides a context to interpret the data we collect by allowing us to determine if there has been a change introduced to the system or it is random variation we are detecting. HIT has an important role to play in any measurement strategy. eCQMs are designed specifically to maximally leverage HIT and have the potential to revolutionize quality measurement. Health IT Workforce Curriculum Version 4.0

Measuring Quality of Care with Electronic Clinical Quality Measures (eCQM) References — 1 Agency for Healthcare Research and Quality. (20xx). United States health information knowledgebase. Retrieved March 2012, from http://www.ushik.org/ Centers for Medicare & Medicaid Services (20xx). eCQM specifications human readable file. EHR Incentive Programs eCQM Library. Retrieved March 2012, from http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html Centers for Medicare & Medicaid Services (20xx). eCQM specifications machine readable file. EHR Incentive Programs eCQM Library. Retrieved March 2012, from http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html Centers for Medicare & Medicaid Services. (2016). Blueprint for the CMS measures management system. Version 12.0. Retrieved May 31, 2016, from https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/MMS/Downloads/Blueprint-120.pdf eCQI Resource Center. (20xx). Retrieved March 2012, from http://www.healthit.gov/ecqi-resource-center National Quality Forum. (2011). Electronic measures. Retrieved May 31, 2016, from http://www.qualityforum.org/Projects/e-g/eMeasures/Electronic_Quality_Measures_(eMeasures).aspx

Measuring Quality of Care with Electronic Clinical Quality Measures (eCQM) References — 2 Charts, Tables, Figures 10.01 Figure: The Promise of eCQMs. National Quality Forum. (2011). Electronic measures. Retrieved May 31, 2016, from http://www.qualityforum.org/Projects/e-g/eMeasures/Electronic_Quality_Measures_(eMeasures).aspx 10.02 Figure: eCQM Lifecycle. 10.03 Figure: eCQM Standards. 10.04 Figure: Quality Data Model (QDM). 10.05 Figure: Quality Data Elements. 10.06 Table: eCQM Data Capture Issues. 10.07 Figure: eCQM Infrastructure. Courtesy Zahid Butt, MD, FACG.

Measuring Quality of Care with Electronic Clinical Quality Measures (eCQM) References — 3 Images Slide 8: HQMF: Human-Readable [Screen shot]. Retrieved March 2012, from http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html Slide 9: HQMF Machine-Readable XML [Screen shot]. Retrieved March 2012, from http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/eCQM_Library.html Slide 15: eCQI Resource Center [Screen shot]. Retrieved March 2012, from http://www.healthit.gov/ecqi-resource-center Slide 16: United States Health Information Knowledgebase [Screen shot]. Retrieved March 2012, from http://www.ushik.org/

Quality Improvement Measuring Quality of Care with Electronic Clinical Quality Measures (eCQM) This material (Comp 12 Unit 10) was developed by Johns Hopkins University, funded by the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology under Award Number IU24OC000013. This material was updated in 2016 by Johns Hopkins University under Award Number 90WT0005. Health IT Workforce Curriculum Version 4.0