1. © 2011 The MITRE Corporation. All rights Reserved. Lessons Learned Behavioral Health eMeasure (BHeM) Project Saul A. Kravitz, PhD July 30, 2012

2. © 2011 The MITRE Corporation. All rights Reserved. Background ■Behavioral Health eMeasure Project –Sponsors: ONC and SAMHSA –Objective: Develop Portfolio of Behavioral Health Measures –Output: 10 eSpecified CQMs, 2 included in NPRM ■Novel Development Approach –Collaborative eMeasures and shared value set development –Model eSpecfication development with shared value sets –Develop portfolio of value sets supporting all 10 eSpecs ■Project Organization –Measure steward collaboration –eSpecification: NCQA (4), TJC (2), and MITRE (4) –Value Set Development: Apelon, Inc 2

3. © 2011 The MITRE Corporation. All rights Reserved. Lessons Learned ■eMeasures Intent and Selection ■Concept Development Identification ■Collaborative eMeasure and Shared Value Set Development ■Value of Impartial Review ■Importance of Testing ■eMeasures Tools Utilization ■Project Management Oversight 3

4. © 2011 The MITRE Corporation. All rights Reserved. Measure Intent and Selection 4 ■NQF-endorsed CQMs include –Intent (e.g., diabetics’ feet should be examined) –Data Source (e.g., claims data) –Logic ■Retooling a CQM preserves intent, replaces data source, adjusts logic ■Some CQMs will only work with a specific data source Lesson: eSpecifications of CQMs should be considered as closely related implementations of the intent of CQMs with alternate data sources

5. © 2011 The MITRE Corporation. All rights Reserved. Single Intent, Multiple Implementations 5 NQF0580: Bipolar Manic Agent Claims DataProvider EHR DataACO EHR Data NQF endorsedBHeMFuture? All three implementations of NQF0580 share intent: bipolar patients should be medicated appropriately Data sources and logic differ Sensitivity and specificity will differ

6. © 2011 The MITRE Corporation. All rights Reserved. Collaborative eMeasure and Shared Value Set Development ■BHeM Project separated eMeasure logic specification from value set development –Value set developer captured measure developer intent –MITRE and measure developer review of value sets –Iterative process to reconcile ■Traditional expression of value sets is an enumeration of code values (“extensional”) –Difficult to review –Difficult to maintain –Doesn’t take advantage of hierarchy of vocabularies ■Expression of Value sets as “recipes” (“intensional) –Much easier to review and maintain –Leverages hierarchy of vocabularies 6

7. © 2011 The MITRE Corporation. All rights Reserved. Traditional Extensional Value Set 7 …and many more 10875004Severe mixed bipolar I disorder with psychotic features, mood-incongruent 10981006Severe mixed bipolar I disorder with psychotic features 111485001Mixed bipolar I disorder in full remission 1196001Chronic bipolar II disorder, most recent episode major depressive 12969000Severe bipolar II disorder, most recent episode major depressive, in full remission 13313007Mild bipolar disorder 13581000Severe bipolar I disorder, single manic episode with psychotic features, mood-congruent 13746004Bipolar disorder 14495005Severe bipolar I disorder, single manic episode without psychotic features 1499003Bipolar I disorder, single manic episode with postpartum onset 162004Severe manic bipolar I disorder without psychotic features 16295005Bipolar II disorder, most recent episode major depressive 16506000Mixed bipolar I disorder 17782008Bipolar I disorder, most recent episode manic with catatonic features 191618007Bipolar affective disorder, current episode manic 191620005Bipolar affective disorder, currently manic, mild 191621009Bipolar affective disorder, currently manic, moderate 191623007Bipolar affective disorder, currently manic, severe, with psychosis 191625000Bipolar affective disorder, currently manic, in full remission 191627008Bipolar affective disorder, current episode depression 191629006Bipolar affective disorder, currently depressed, mild 191630001Bipolar affective disorder, currently depressed, moderate 191632009Bipolar affective disorder, currently depressed, severe, with psychosis 191634005Bipolar affective disorder, currently depressed, in full remission

8. © 2011 The MITRE Corporation. All rights Reserved. Intensional Value Set Description 8 Lessons: Communication between developers of value sets and measure stewards is essential Value set recipes for clinical concepts should be specified ‘intensionally ’ Include codes that are in active use Include codes that apply to Humans

9. © 2011 The MITRE Corporation. All rights Reserved. Value of Impartial Review ■Independent reviews resulted in improvements to measure design –Simplification of the proposed measures –Identification of errors and issues with data and logic –Elimination of data elements –Improved feasibility ■Improved accuracy and generality of value sets ■Simplified and clarified measure logic based on development and review of test cases 9 Lesson: eMeasures and Value Sets intended for inclusion in national quality programs should be reviewed by 3rd party with focus on feasibility, correctness, and computability.

10. © 2011 The MITRE Corporation. All rights Reserved. Importance of Testing ■BHeM eSpecification deliverables included test data and outcomes in a common format –Specified at concept, not code level –Identify logical errors in eMeasures –Identify scenarios not considered by eSpecifiers –Facilitate testing Lesson: Testing is essential –Test data and outcomes should be required in documentation of eMeasures –Automated testing for conformance with test data should be the rule 10