1. Los Angeles County Evaluation System: An Outcomes Reporting Program 2008 Site Report Presentation to Contract Program Auditors

2. Purpose of this Presentation To provide a brief overview of the Los Angeles County Evaluation System (LACES). To review the content and purpose of the Site Reports. Review those content areas that are the focus of the evaluation.

3. Overview of LACES LACES is a partnership between ADPA, Treatment Providers, and UCLA/ISAP to conduct an evaluation - LACES is not an external study. Treatment programs are contractually required to comply with the evaluation efforts. Evaluate the alcohol and other drug treatment system. On-going evaluation, not a temporary study.

4. First… Some Questions… How many programs are your responsible for monitoring? How often do you visit the programs?

5. Whats a Site Report? Have you heard of a site report? Have you seen a site report? Have you had the opportunity to compare the information in the site reports with ADAM?

6. A Site Report Is… Brief: six pages, double-sided. Information is gathered from the admission & discharge LACPRS. Information is included on your site and for all sites of the same program type.

7. What is the Purpose of the Site Report? To provide information concerning the key performance/outcome areas. To provide feedback to sites concerning how they are performing with regards to the focus areas of the evaluation. To provide feedback on how other similar programs are performing.

8. Contents of the Site Report Executive Summary Includes overall participant admission and discharge numbers. Includes information on key outcomes. This will be the focus of this presentation as it includes the outcome information that will be used to evaluate programs. Full Report Includes detailed information on all participants. Includes more details on changes in key outcomes.

9. Executive Summary Single page, stand alone document. Covers the admission to discharge changes for your site compared to all sites of the same program type. Key Areas include Participant Flow chart. Performance Measures.

10. Important Performance & Outcome Measures Completion of treatment/Positive Compliance. Abstinence from/Reduction in drug/alcohol use. Exit Interviews Length of Stay/Enrollment in Treatment (2 benchmarks) 30 Days = Engagement. 90 Days = Retention. All important aspects of the treatment plan.

11. Why Completion of Treatment? Most outside of this field are interested in completion figures. Positive Compliance Completed treatment. Left treatment prior to completion with satisfactory progress. Others have adopted this mindset, but many have not.

12. Why Reduction/Abstinence? Silly picture, for a silly question. That is our business. If we cannot demonstrate abstinence or reductions in use, funders will move the money to those who can.

13. Interviews Responses to all discharge LACPRS questions. One of the most important of the evaluation foci. Information from both admission and discharge is required to measure treatment effectiveness. No exit interviews = No measurement of changes that occurred during treatment.

14. Why Engagement & Retention? First 30 days are critical to the outcome of treatment. Research shows that without an adequate amount of time in treatment, few improvements are found. Funding organizations are beginning to take note of engagement and retention rates.

15. Note on Documentation Accuracy is extremely important. Length of stay/enrollment. Example – length of stay/enrollment. In 2004-2005, average length of stay was 105 days. 40% were in treatment for 30+ days. If providers are not inputting discharge date accurately – what other information is being entered into LACPRS inaccurately?

16. Max LOS/LOE by Funding Source All funding sources support 30- and 90-day length of stay/enrollment benchmarks. FUNDINGOutpatient LOEResidential LOSDay Care LOE CalWORKS365 Days Drug Court730 Days90 DaysN/A Female OffenderN/A180 DaysN/A General Relief180 Days PPN180 Days N/A Prop 36300 Days90 Days204 Days Drug Medi-CalNo max length - approval is need for 180+ days.

17. Average Provider Performance During the last 3 fiscal years, average performance in these areas was… Does this make sense to you? PERFORMANCE AREAOutpatientResidentialDay Care Reduction in Primary Drug Use75.7%97.7%77.9% 30-Day Retention71.8%64.1%68.9% 90-Day Retention48%38.1%40.6% Exit Interviews48.6%73.7%47%

18. Roles of the Contract Monitor Answer questions regarding site reports. Review reports for accurate information. Provide suggestions on how to improve outcomes – to come. Seek assistance from ADPA or UCLA for responses to questions, concerns, or additional assistance.

19. How to Improve Exit Interview Rates Some ways to increase rates of exit interviews include Informing the participant of the exit interview when admitted. Include completion of the exit interview as part of the individualized treatment plan. Remind the participant when they are nearing completion of their plan of the exit interview.

20. Improve Engagement & Retention? Engage them from first contact Make same day appointments if possible. Appointment cards To encourage forgetful clients. Phone calls To remind clients of appointments. To follow-up with clients who may have dropped out.

21. Other Suggestions?

22. Challenges LACPRS Provides a lot of good information. Revisions and monitoring of information will be needed to ensure quality & validity. Site Reports Also requires revisions to format & content. Not getting past Executive Directors eyes.

23. Your Views on the Site Reports What do you think about the site reports? What have you heard from your providers? Do you have suggestions or recommended changed to the reports?

24. Future of the Site Reports The Site Reports will be used to asses individual sites in the key areas described in this presentation. Over time peer performance standards will be developed. Programs that fall short of the standards will be offered information, training and other forms of assistance in order to bring their sites performance up to the standard.

25. The End