1. An Introduction to the Agency for Healthcare Research and Quality’s (AHRQ’s) National Quality Measures Clearinghouse (NQMC) Mary P. Nix, Program Lead National Guideline Clearinghouse™ (NGC) National Quality Measures Clearinghouse™ (NQMC) Melanie M. Swan, MPH, PMP Program Manager, NGC/NQMC Richard C. Hermann, MD, MS NGC/NQMC Editorial Board Co-chair April 16, 2015 GIN - NA Webinar April 16, 2015 AHRQ

2. Acknowledgements AHRQ Mary P. Nix NQMC Management Team Vivian H. Coates Melanie M. Swan Anne Wert Vanessa Lucas Rand/Harvard/Tufts Cheryl Damberg Richard C. Hermann Lauren Hunter Grant Martsolf Heather Palmer ECRI NQMC Team Kariann Hudson Christina Latzko Amber Moran Kelly Petersen Jill Thomas Ayo Uko Kayla Van Osten NGC/NQMC Editorial Board Richard C. Hermann – Co-Chair Paul G. Shekelle – Co-Chair 2 GIN - NA Webinar April 16, 2015 AHRQ

3. Presentation Objectives NQMC: AHRQ’s vision Value to stakeholders NQMC overview ▫Current inclusion criteria ▫Current domain framework ▫Perspectives on inclusion criteria revision/evidence ▫How to retrieve measures in NQMC ▫Unique tool: Measure Matrix ▫National Quality Forum (NQF) and NQMC  Measure steward ▫Relationship of NQMC to NGC How to submit measures to NQMC 3 GIN - NA Webinar April 16, 2015 AHRQ

4. 4 http://www.qualitymeasures.ahrq.gov

5. Value to Stakeholders: A Decade of Progress Contains substantial structured descriptive data about thousands of measures submitted to NQMC from all over the globe Supports empirical analysis of measures to promote measurement science, quality measurement programs, and quality improvement Facilitates measure dissemination and provides summary usage reports for participating measure developers Currently evaluating tightening evidence inclusion criteria 5 GIN - NA Webinar April 16, 2015 AHRQ

6. Value to Stakeholders: How Does NQMC Serve its Audience? Help users to find and compare quality measures  Template of attributes Search using multiple controlled taxonomies  UMLS vocabularies (MeSH, SNOMED, ICD) Search or Browse using filters  Measures endorsed by NQF Forum for debate: Expert Commentaries Regularly updated tutorials about quality measurement 6 GIN - NA Webinar April 16, 2015 AHRQ

7. Current NQMC Inclusion Criteria To be included in NQMC, a measure must meet all of the following requirements: 1.The measure must address some aspect(s) of health care delivery or population health that can be classified into one of the domains described in the Domain Definitions tab. 2.The measure must be in current use or have been pilot tested within the last three years and must be the most recent version if the measure has been revised. A measure is in current use if at least one health care organization has used the measure to evaluate or report on quality of care within the previous three years. 7 GIN - NA Webinar April 16, 2015 AHRQ

8. Current NQMC Inclusion Criteria 3.The submitter must provide English-language documentation that is available upon request in print or electronic format (for free or for a fee) that includes at least each of the four following items:  The rationale for the measure  A description of the denominator and numerator of the measure (including specific variables for inclusion or exclusion of cases/events from either the denominator or numerator)  The data source(s) for the measure  Documentation of evidence supporting the measure and the criterion of quality is required for Quality Measures, and for the quality component of Efficiency Measures. Note: Any Related Measures must be part of a set or collection that includes Quality Measures to be considered for inclusion. 8 GIN - NA Webinar April 16, 2015 AHRQ

9. Current NQMC Inclusion Criteria 4.At least one of the following criteria must be satisfied with specific information attached in each case (evidence from peer-reviewed literature is preferred): ▫The measure has been cited in one or more reports in a National Library of Medicine (NLM) indexed, peer-reviewed journal, applying or evaluating the measure’s properties. ▫The submitter provides documented evidence evaluating the reliability and validity of the measure. ▫The measure has been developed, adopted, adapted, or endorsed by an organization that promotes rigorous development and use of measurement in health care. Such an organization may be at the international, national, regional, state or local levels (e.g., a multi-state consortium, a state Medicaid agency, or a health organization or delivery system). 9 GIN - NA Webinar April 16, 2015 AHRQ

10. 10 Current NQMC Domain Framework GIN - NA Webinar April 16, 2015 AHRQ

11. AHRQ: Explore modifications to the NQMC inclusion criteria on evidence to align with revisions to NGC ▫ Institute of Medicine, National Guideline Clearinghouse Explore requiring evidence supporting: ▫ Underlying clinical concept (“referent”) ▫ Reliability/validity Example: % of pts post AMI discharged on beta blocker Stakeholder interviews, expert panel input Pilot testing Transition time for developers GIN - NA Webinar April 16, 2015 AHRQ 11 Inclusion Criteria Revision: Evidence-based Quality Measures

12. How to Retrieve Measures in NQMC Various Measure Browse Options ▫By Topic ▫By Organization ▫By Domain ▫By National Quality Strategy ▫NQF-Endorsed Measures ▫Measure Initiatives ▫Measures in Progress ▫Measure Index ▫Measure Archive ▫Measures Most Viewed ▫Related NGC Guidelines 12 GIN - NA Webinar April 16, 2015 AHRQ

13. NQF-Endorsed Measures Browse Facilitates retrieval of measures represented in NQMC endorsed by the National Quality Forum (NQF) ▫Search by NQF number ▫Search by NQF endorsement date ▫Use the faceted browse feature to further narrow measure retrieval by selecting categories of interest  Measure Domain (e.g., process, outcome)  Professionals Involved in Delivery of Health Services  Measurement Setting  Type of Care Coordination  Measure Initiative (e.g., Physician Quality Reporting System)  Organization (e.g., submitting organizations, measure developers) 13 GIN - NA Webinar April 16, 2015 AHRQ

14. How to Retrieve Measures in NQMC Search Feature ▫Main search feature accessed via search box ▫Advanced Search  Ability to include keyword search and search indexing keywords only (Disease/Condition, Treatment/Intervention, Health Services Administration)  Target searches by selecting topics from selected controlled vocabulary fields  Measure Domain  Measurement Setting  National Quality Strategy Aim, National Quality Strategy Priority  Current Use  Data Source  Professionals Involved in Delivery of Health Services 14 GIN - NA Webinar April 16, 2015 AHRQ

15. Unique Tool: Measure Matrix Tool used to filter, retrieve, and export relevant content of interest Facilitates identification of gaps in measurement Select a field (a row and a column) to filter NQMC content Generation of a table of all measures at the intersection of these two fields 15 GIN - NA Webinar April 16, 2015 AHRQ

16. Unique Tool: Measure Matrix 16 GIN - NA Webinar April 16, 2015 AHRQ

17. How is NQMC different from NQF? NQMC is an initiative of AHRQ ▫Public resource for evidence-based quality measures and measure sets ▫Database and Web site for information on specific evidence- based health care quality measures and measure sets ▫Sponsored by AHRQ to promote widespread access to quality measures by the health care community and other interested individuals ▫NQMC also hosts the HHS Measure Inventory ▫NQMC does not endorse measures and includes measures from other countries 17 GIN - NA Webinar April 16, 2015 AHRQ

18. How is NQMC different from NQF? NQMC Mission ▫To provide practitioners, health care providers, health plans, integrated delivery systems, purchasers and others an accessible mechanism for obtaining detailed information on quality measures, and to further their dissemination, implementation, and use in order to inform health care decisions. 18 GIN - NA Webinar April 16, 2015 AHRQ

19. How is NQMC different from NQF? NQF is a not-for-profit, nonpartisan, membership-based organization ▫Forum that convenes work groups to foster QI in public- and private-sectors ▫Endorses consensus standards for performance measurement ▫Ensures that consistent, high-quality performance information is publicly available ▫Seeks real time feedback to ensure measures are meaningful and accurate 19 GIN - NA Webinar April 16, 2015 AHRQ

20. How is NQMC different from NQF? NQF draws on “measure stewards” to work with NQF during and following the measure endorsement process: ▫Submits measure(s) for endorsement consideration ▫Makes the measure specifications generally available for “Permitted Uses” ▫Maintain the measure(s) throughout the endorsement period ▫Provide changes to endorsed measures Reference: NQF Measure Steward Agreement (MSA); Version 11.4.2013 20 GIN - NA Webinar April 16, 2015 AHRQ

21. How is NQMC different from NQF? NQMC works with measure submitting and/or developing organizations to represent their measures in NQMC. Measure developers: ▫Submit measure(s) to be considered for inclusion in NQMC (including current contact information)  Work with NQMC inclusion review team to provide all required measure documentation ▫Review completed NQMC measure summaries for accuracy, completeness, and responds to related questions ▫Notify NQMC when changes are made to the measure(s) ▫Confirms currency of their published measures and organizational contact information on an annual basis (annual verification), thereby maintaining the measure(s) ▫Provide necessary copyright (if applicable) 21 GIN - NA Webinar April 16, 2015 AHRQ

22. Relationship of NQMC to NGC NQMC is a sister site of the National Guideline Clearinghouse (NGC). Both are sponsored by AHRQ. Similarities ▫Missions  NQMC: provide practitioners, health care providers, health plans, integrated delivery systems, purchasers and others an accessible mechanism for obtaining detailed information on quality measures, and to further their dissemination, implementation, and use in order to inform health care decisions.  NGC: provide physicians and other health professionals, health care providers, health plans, integrated delivery systems, purchasers, and others an accessible mechanism for obtaining objective, detailed information on clinical practice guidelines and to further their dissemination, implementation, and use. 22 GIN - NA Webinar April 16, 2015 AHRQ

23. Relationship of NQMC to NGC Similarities ▫Integrated (NGC/NQMC) Editorial Board  Resource for feedback and guidance on developments in health care that inform future work ▫Inclusion Criteria  Measures and guidelines must meet the respective clearinghouse criteria for inclusion in NQMC and NGC ▫Structured, standardized summaries  NQMC and NGC each provide structured, standardized summaries containing information about measures and guidelines using the respective templates of attributes ▫Similar workflow/processes  Lead development, inclusion review, measure abstraction, Web publication 23 GIN - NA Webinar April 16, 2015 AHRQ

24. Relationship of NQMC to NGC Differences ▫Measures are submitted in collections, sets, or as individual measures ▫NQMC Domain Framework – N QMC’s overarching classification system ▫NGC produces Guideline Syntheses – systematic comparisons of selected guidelines addressing similar topic areas ▫NGC offers Guideline Resources 24 GIN - NA Webinar April 16, 2015 AHRQ

25. Relationship of NQMC to NGC NQMC/NGC Linkages ▫NGC Link field  Identifies link(s) to guideline summary(s) in the National Guideline Clearinghouse™ (NGC) where the measure was developed from an evidence-based guideline ▫Measure Browse: Related NGC Guidelines  Identifies measures with associated guidelines available through NGC NGC/NQMC Linkages ▫Related NQMC Measures  Identifies link(s) to related quality measures in the National Quality Measures Clearinghouse™ (NQMC) ▫Guideline Browse: Related NQMC Measures  Identifies guidelines with associated measures available through NQMC 25 GIN - NA Webinar April 16, 2015 AHRQ

26. How to Submit Measures to NQMC Go to the Submit Measures Web page:Submit Measures ▫Measure Submission Kit  Submission Guidance/Instructions to Submit  Domain Framework and Inclusion Criteria  Measure Submission Checklist  Template of Measure Attributes  Glossary of Terms  Copyright Agreement Form Other resources ▫Background information (history, mission, user base) ▫Benefits of participation (dissemination, summary usage reports) ▫Delineation of measure developer role in NQMC 26 GIN - NA Webinar April 16, 2015 AHRQ

27. 27 Questions?