1. Tracking and Measuring Outcomes in King County, WA Genevieve Rowe Mental Health, Chemical Abuse and Dependency Services Division genevieve.rowe@kingcounty.gov

2. King County Mental Health, Chemical Abuse and Dependency Services Division Manages publically funded mental health & substance abuse services and services for co-occurring disorders. Situated within the Department of Community and Human Services. Coordinates with regional human services partners via King County Human Services Framework Policies. King County's priorities for human service investments will be programs and services that help to stabilize and improve people's lives, and prevent or reduce emergency medical and criminal justice system involvement and costs. Human Services Priority Areas 1.Effective intervention and prevention strategies 2.Job readiness and employment to increase self-sufficiency 3.Prevention and elimination of homelessness 4.Services that reduce the growth of emergency medical and criminal justice system involvement and costs

3. Building a routine data collection process for evaluating supported housing and treatment programs - Has taken 10 years Isn’t finished Still includes some manual entry and cleanup Has considerably improved evaluation efficiency

4. In the beginning we had data for MHCADSD funded services - MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospitals admit date only)

5. In the late 1990’s the county jails were reaching capacity. Many frequent users were enrolled in public MH/SA treatment services but were not really engaged in those services. MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospital admit date only) King County Jail Booking Data was provided daily to help locate clients for service engagement.

6. The 2003 Criminal Justice Initiative funded new services and added evaluation resources. A new Data Sharing Agreement allowed use of the daily jail booking data for evaluations and added - Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospital admit date only) Detailed KC Jail Booking Data – a quarterly file of all bookings since 2001. Includes charges, flag if booking involved psychiatric housing, detention vs. EHD. Note: this added PHI. Many data sources are available retroactively. There is time to work things out.

7. Additional data sources were added for a few small programs - Quarterly Detailed KC Jail Booking Data Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospital admit date only) Municipal jail booking data (manual collection and entry process) Emergency Department data from one major hospital (clients signed Release of Information forms) Monthly logs with data for clients not in MH system Substance Use assessment forms (faxed hard copy)

8. A 2007 grant funded the FACT program and a.6 FTE evaluator. IRB approval and Confidentiality Agreements allowed collection of additional data - Municipal Jail Booking Data Partial Emergency Department Data Quarterly Detailed KC Jail Booking Data Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospital admit date only) Detailed State Psychiatric Hospital Data (including forensic admissions) for FACT population only Detailed Prison Data for FACT population only

9. FACT demonstrated the importance of complete state hospital data for this population. An agreement was reached to provide - Detailed State Psychiatric Hospital Data Prison Data Municipal Jail Booking Data Partial Emergency Department Data Quarterly Detailed KC Jail Booking Data Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospital admit date only) Detailed state psychiatric hospital data for civil admissions and flips (early 2011). Detailed state psychiatric hospital data for forensic admissions (mid 2011).

10. In 2008 the Mental Illness and Drug Dependency Action Plan, funded by a one-tenth of one cent sales tax increase, was enacted. The MIDD brought additional evaluation resources and began the long process of negotiating for complete jail data - Complete Detailed State Psychiatric Hospital Data Prison Data Municipal Jail Booking Data Partial Emergency Department Data Quarterly Detailed KC Jail Booking Data Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data (included civil commitments to state hospital admit date only) Complete Municipal Jail Data from all five jails available in December 2011.

11. After 10 years - Complete Municipal Jail Data Complete Detailed State Psychiatric Hospital Data Prison Data Partial Emergency Department Data Quarterly Detailed KC Jail Booking Data Daily King County Jail Booking Data MH Service Data Community Psychiatric Hospital Data Crisis Data - and monthly logs, other data forms

12. Where we are now - Complete KC Jail and Municipal Jail data Complete Community and State Psychiatric Hospital data Mental Health Service and Crisis data No Prison data (discontinued) Partial ED data for some programs (expanding to larger programs)

13. How Data Is Transferred In the Beginning… Secure FTP Password protected disks Faxed forms and spreadsheets Now… Secure FTP Encrypted thumb drives Encrypted email Faxed forms

14. You will be asked ….. How much did we save? Can you do a cost benefit analysis? What are the cost offsets? A cost benefit analysis is not an extra step in a program evaluation.

15. Jail Days Are Not All Equal

16. What about a comparison group? Whenever the number eligibles exceeds program capacity there is an opportunity for a comparison group. In a non-research setting it is very difficult to execute a randomized experimental evaluation design; but it can happen. Randomization or any limits to access to public services probably requires IRB approval. In reality you have no control over a control group.

17. How to begin… Start small and simple: who, what, when, why Look for opportunities to build capacity Be aware of what others have the capacity for Be patient – electronic records usually don’t disappear Address data security, confidentiality, and HIPAA Have participants sign an ROI if there is any question about permission to access data

18. Sample Data Collection Sheet – Coding Instructions

19. Sample Data Collection Sheet – Code Values

20. Sample Data Collection Sheet – More Code Values

22. Sample Data Collection Sheet – Master List and Demographics

23. Sample Data Collection Sheet – Monthly Service Hours

24. One page does it all…