1. 1 NQF 101: Developing Consensus March 25, 2009 NQF Spring Membership Meeting Renaissance Cleveland Hotel Cleveland, OH

2. 2 NQF’s Mission The National Quality Forum is a private, not-for- profit voluntary consensus standards-setting organization. The mission of NQF is: to improve the quality of American healthcare by setting national priorities and goals for performance improvement; endorsing national consensus standards for measuring and publicly reporting on performance; and promoting the attainment of national goals through education and outreach programs.

3. 3 Developing Consensus Formal consensus development process (CDP) Attention to overall strategy for measuring and reporting healthcare quality, including establishing national goals Multi-stakeholder membership including consumers, purchasers, plans, professionals, providers, quality and researchers, community and public health and supplier and industry organizations Public and private sector representation on governing board Focus is on the entire continuum of healthcare Formal endorsement as “ voluntary consensus standards ”

4. 4 Consensus Development Process Draft recommendations National Priorities and NQF program priorities Specific project topics Project Steering/Review Committee Draft consensus standards NQF-endorsed consensus standards Technical Advisors or Panels; Workgroups Review (Member and public) CSAC/Board action Appeals Update as warranted Member voting

5. 5 Roles of the Steering Committee Act as a proxy for the NQF multi-stakeholder membership for a specific project Work with NQF staff to achieve the goals of the project Evaluate candidate measures against the formal measure evaluation criteria Make recommendations to the NQF membership for endorsement Respond to comments submitted during the review period Co-chair(s) represent the Steering Committee when the CSAC meets Respond to any directions from the CSAC

6. 6 Role of NQF Staff NQF project staff works with Steering Committee to achieve the goals of the project and ensure adherence to the consensus development process: –Organize and staff SC meetings and conference calls –Guide the SC through the steps of the CDP and advises on NQF policy and procedures –Draft measure evaluations and reports for SC review –Ensure communication among all project participants (including SC and measure developers)

7. 7 Roles of NQF Staff –Respond to NQF member or public queries about the project –Maintain documentation of project activities –Post project information to NQF website –Work with measure developers to provide necessary information and communication for the SC to fairly and adequately evaluate measures for endorsement –NQF project staff works with communications department to publish final report

8. 8 Strategic Measurement Issues for NQF Driving toward high performance Shifting toward composite measures Moving toward outcomes measurement Measure disparities in all we do

9. 9 NQF Endorsement Criteria New NQF measure evaluation criteria were approved by Board of Directors in August 2008 Clarify, strengthen and recommend changes to endorsement criteria in order to achieve: –a stronger link to national priorities and higher-level performance measures; –greater measure harmonization; –greater emphasis on outcome measures; and –for process measures, a tighter outcomes-process

10. 10 NQF Endorsement Criteria Old criteriaNew criteria (August 2008) Conditions for consideration ImportanceImportance to measure and report (must pass criterion) Scientific acceptabilityScientific acceptability of measure properties FeasibilityFeasibility (greater emphasis on Health IT) UsabilityUsability (greater emphasis on harmonization)

11. 11 Conditions for Consideration 1.The measure is in the public domain or an intellectual property agreement is signed. 2. The measure owner verifies there is an identified responsible entity and process to maintain and update the measure. 3. The intended use of the measure includes both public reporting and quality improvement. 4.The requested measure submission information is complete. Generally, measures should be fully developed and tested so that all information needed to evaluate the measure is available. Measures that have not been tested are only potentially eligible for a time-limited endorsement Measure owners must verify that testing will be completed within 24 months of endorsement.

12. 12 Importance to Measure and Report The specific focus of what is measured should be considered important enough to expend resources for measurement and reporting, not only that it is related to an important broad topic area. In the modified criteria, these concepts are addressed in separate sub-criteria for –Relation to an NQF priority or high impact aspect of healthcare –Evidence to support the measure focus –Opportunity for improvement.

13. 13 Importance to Measure and Report The recommendation that the importance criterion should be considered “must pass.” If a measure is not judged to be sufficiently important, then regardless of how well it might be assessed on the other criteria the measure would not be recommended for consideration as a voluntary consensus standard.

14. 14 Scientific Acceptability of Measure Properties The label clearly indicates this criterion applies to measure properties: –Precise specifications –Reliability, validity, and discrimination (testing is expected to demonstrate reliability and validity) –Demonstration of comparability if more than one data source/method is allowed –Specifications should allow for identification of disparities. –Risk-adjustment – Exclusions

15. 15 Exclusions Exclusions increase complexity and measurement burden, limit the use of electronic sources, and often create a barrier to measure harmonization. Evaluation criteria requires that evidence is presented to demonstrate that measure results would be distorted without the specified exclusions. If patient preference is a consideration in numerator or denominator exclusions, the measure should be specified so that the effect of patient preference on the measure is transparent Exclusions should not require additional data sources beyond what is required for scoring the measure (e.g., numerator and denominator) unless justified as supporting measure validity.

16. 16 Usability Requires evidence that the measure results are meaningful and understandable to intended audiences and useful for both public reporting and informing quality improvement. –This is consistent with NQF policy of not endorsing measures solely for quality improvement. –New criteria requiring that measures are harmonized and provide a distinctive or additive value to existing endorsed measures.

17. 17 Feasibility Extent to which the required data are readily available, retrievable without undue burden, and can be implemented for performance measurement. –Required data are routinely generated concurrent with and as a byproduct of care processes during care delivery. –The required data elements are available in electronic sources. If the required data are not in existing electronic sources, a credible, near-term path to electronic collection by most providers is specified and the data elements are specified for transition to the electronic health record.

18. 18 Staff Helen Burstin, MD, MPH Sr. Vice President, Performance Measures hburstin@qualityforum.org Marybeth Farquhar, PhD, MSN, RN Managing Director, Performance Measures mfarquhar@qualityforum.org Del Conyers, MPH Assistant Managing Director, Performance Measures dconyers@qualityforum.org