© 2011 The MITRE Corporation. All rights Reserved. Lessons Learned Behavioral Health eMeasure (BHeM) Project Saul A. Kravitz, PhD July 30, 2012

© 2011 The MITRE Corporation. All rights Reserved. Background ■Behavioral Health eMeasure Project –Sponsors: ONC and SAMHSA –Objective: Develop Portfolio of Behavioral Health Measures –Output: 10 eSpecified CQMs, 2 included in NPRM ■Novel Development Approach –Collaborative eMeasures and shared value set development –Model eSpecfication development with shared value sets –Develop portfolio of value sets supporting all 10 eSpecs ■Project Organization –Measure steward collaboration –eSpecification: NCQA (4), TJC (2), and MITRE (4) –Value Set Development: Apelon, Inc 2

© 2011 The MITRE Corporation. All rights Reserved. Lessons Learned ■eMeasures Intent and Selection ■Concept Development Identification ■Collaborative eMeasure and Shared Value Set Development ■Value of Impartial Review ■Importance of Testing ■eMeasures Tools Utilization ■Project Management Oversight 3

© 2011 The MITRE Corporation. All rights Reserved. Measure Intent and Selection 4 ■NQF-endorsed CQMs include –Intent (e.g., diabetics’ feet should be examined) –Data Source (e.g., claims data) –Logic ■Retooling a CQM preserves intent, replaces data source, adjusts logic ■Some CQMs will only work with a specific data source Lesson: eSpecifications of CQMs should be considered as closely related implementations of the intent of CQMs with alternate data sources

© 2011 The MITRE Corporation. All rights Reserved. Single Intent, Multiple Implementations 5 NQF0580: Bipolar Manic Agent Claims DataProvider EHR DataACO EHR Data NQF endorsedBHeMFuture? All three implementations of NQF0580 share intent: bipolar patients should be medicated appropriately Data sources and logic differ Sensitivity and specificity will differ

© 2011 The MITRE Corporation. All rights Reserved. Collaborative eMeasure and Shared Value Set Development ■BHeM Project separated eMeasure logic specification from value set development –Value set developer captured measure developer intent –MITRE and measure developer review of value sets –Iterative process to reconcile ■Traditional expression of value sets is an enumeration of code values (“extensional”) –Difficult to review –Difficult to maintain –Doesn’t take advantage of hierarchy of vocabularies ■Expression of Value sets as “recipes” (“intensional) –Much easier to review and maintain –Leverages hierarchy of vocabularies 6

© 2011 The MITRE Corporation. All rights Reserved. Traditional Extensional Value Set 7 …and many more Severe mixed bipolar I disorder with psychotic features, mood-incongruent Severe mixed bipolar I disorder with psychotic features Mixed bipolar I disorder in full remission Chronic bipolar II disorder, most recent episode major depressive Severe bipolar II disorder, most recent episode major depressive, in full remission Mild bipolar disorder Severe bipolar I disorder, single manic episode with psychotic features, mood-congruent Bipolar disorder Severe bipolar I disorder, single manic episode without psychotic features Bipolar I disorder, single manic episode with postpartum onset Severe manic bipolar I disorder without psychotic features Bipolar II disorder, most recent episode major depressive Mixed bipolar I disorder Bipolar I disorder, most recent episode manic with catatonic features Bipolar affective disorder, current episode manic Bipolar affective disorder, currently manic, mild Bipolar affective disorder, currently manic, moderate Bipolar affective disorder, currently manic, severe, with psychosis Bipolar affective disorder, currently manic, in full remission Bipolar affective disorder, current episode depression Bipolar affective disorder, currently depressed, mild Bipolar affective disorder, currently depressed, moderate Bipolar affective disorder, currently depressed, severe, with psychosis Bipolar affective disorder, currently depressed, in full remission

© 2011 The MITRE Corporation. All rights Reserved. Intensional Value Set Description 8 Lessons: Communication between developers of value sets and measure stewards is essential Value set recipes for clinical concepts should be specified ‘intensionally ’ Include codes that are in active use Include codes that apply to Humans

© 2011 The MITRE Corporation. All rights Reserved. Value of Impartial Review ■Independent reviews resulted in improvements to measure design –Simplification of the proposed measures –Identification of errors and issues with data and logic –Elimination of data elements –Improved feasibility ■Improved accuracy and generality of value sets ■Simplified and clarified measure logic based on development and review of test cases 9 Lesson: eMeasures and Value Sets intended for inclusion in national quality programs should be reviewed by 3rd party with focus on feasibility, correctness, and computability.

© 2011 The MITRE Corporation. All rights Reserved. Importance of Testing ■BHeM eSpecification deliverables included test data and outcomes in a common format –Specified at concept, not code level –Identify logical errors in eMeasures –Identify scenarios not considered by eSpecifiers –Facilitate testing Lesson: Testing is essential –Test data and outcomes should be required in documentation of eMeasures –Automated testing for conformance with test data should be the rule 10