SIM Evaluation Approach Presentation to the SIM Steering Committee September 25, 2013

Goals Establish a common set of quality/performance metrics. Develop system for continuous feedback of performance data to providers and SIM stakeholders that supports data driven decision making, and continuous improvement. Design and implement a robust evaluation that describes and documents the implementation of model interventions and assesses the impact of the innovation model on improving the quality of care, patient experiences, and reducing costs. Establish a local research collaborative and research program to test the effectiveness of healthcare innovations and interventions and facilitates the spread of best practices.

Evaluation Components Participation in CMMI Cross-Site Evaluation Design and implement a State-Specific Self Evaluation

CMMI Cross Site Evaluation Strategy Three part strategy: Design and data collection Rapid cycle evaluation of state models Impact evaluation

Self Evaluation: Core Components Implementation/Process Evaluation Cost Effectiveness Evaluation Impact/Effectiveness Evaluation

Evaluation Questions Does the model implementation lead to changes in service utilization patterns and reduction in costs? Does the model lead to improved care coordination and less fragmentation of care and, if so, for what populations? Does the model lead to improved processes of care? To what extent does the model improve the level of integration of physical and behavioral health across Maine’s health care system?

Evaluation Questions (Continued) Does the model lead to improved beneficiary health, wellbeing, functioning and reduced health risk behaviors? Does the model lead to improved beneficiary experiences of care and perception of services ? What factors influence the adoption and spread of model enhancements? To what extent are model components implemented consistently and with fidelity? What system, practice, and beneficiary level factors are associated with the model outcomes?

Evaluation Work Plan Highlights Draft RFP for Evaluation Contractor (Sep. 2013) Select evaluator and execute a contract (Nov. 2013) Identify committee members and convene Evaluation Committee (Nov. 2013) Review and refine evaluation logic model with Innovation Model partners and Stakeholders (Dec. 2013) Identify core quality/performance metrics for use in evaluation and performance monitoring (Dec. 2013) Develop study design, data collection, and analytics plan for local implementation study, economic/cost study, and impact evaluation (Jan. – Mar. 2014) Work with CMMI to develop and test standardized data collection and rapid cycle evaluation protocols (Jan. – Apr. 2014). Implement data collection for local self-evaluation studies (Jul. 2014) Implement data collection for CMMI Cross-Site model implementation and impact evaluation components (Jul. 2014)