Program Performance Evaluation Edward Freeman March 25, 2011

Program Evaluation Systems National Outcome Measures (NOMs) NYS Integrated Program Monitoring and Evaluation System (IPMES) NYS Annual Program Performance Review (PPR)

NOMS – Reduced Morbidity (e.g., abstinence); – Increased Employment and Education; – Decreased Criminal Justice Involvement; – Stability in Housing; – Social Connectedness; – Access and Capacity; – Retention in Care; – Perception of Care; – Cost Effectiveness; – Use of Evidence-Based Practices.

IPMES/Workscope Performance Measurement Systems Compare current program to similar program Compare current program to program performance to prior years Compare to minimum standards

Program Performance Review Summary evaluation of process for all funded programs Approve State funding on a prospective basis Insures that resources are used to meets standards for accessibility, effectiveness and efficiency Establish program improvement requirements

From Data to Performance Measurement Admission Reports Discharge Reports Monthly Serv. Delivery Methadone Update Data Sources Indicators Performance Analysis  Compare to similar programs  Compare current program performance to that of prior years  Compare to minimum standards  Compare to program performance targets (Workscopes )  Scorecards Utilization Rate Units of Service Client-to-Direct Care Staff Ratio Discontinued Use Employment Retention Completion

Review Elements Program Performance Site visits Regulatory Compliance Fiscal Audit and Viability Cost standards

Databases Client Data System IPMES Certification Fiscal Viability Consolidated Fiscal Reports Site visit reports Prior Year Management Plans Proposed Workscopes

Client Discharge Report

12 IPMES Report

13 Six Month Retention

Review Criteria IPMES Flags Reporting delinquency Utilization Productivity Retention Completion rates Discontinued Use of primary and secondary substances

Review Criteria (cont’d) Employment status Site review findings Regulatory compliance below partial Status of any investigations Gross Cost standards Underspending Unusual costs

Year Regulatory Complianc e Productivit yReporting 2007/ / / / /