Kara Taguchi, Psy.D. Mental Health Clinical Program Head MHSA Implementation and Outcomes Division RAND/SRI Data Systems Workgroup

Select Measures and Process for data collection. 10/2010 Translate into business requirements for electronic data collection and reporting. 2/7/11- 4/12/11 CIOB develops application, presents content, testing, training, roll out. 2/7/11-7/1/11

* Built upon CiMH support already in place * General measure and focus of treatment specific measure administered at the beginning and at the end of treatment * Scale scores or total scores entered in CiMH data shell or PEI Outcome Measures Application (OMA) * Measures selection based on: * CiMH Palette of Measures * Developer requirements * Cost * Length * Language translations * In general, one specific measure for each foci of treatment * Review with providers of service at PEI Clinical Implementation meetings

* The depth and breadth of the information you want to collect * Using tools with a copyright vs. public domain instruments * Valid, reliable measures vs. creating your own tools * How many users of your system * Planning for potential growth * Workflow and evolving processes

* Secure Log on based on associations in our billing system * Clients must be selected from our billing system * Once Focus of Treatment and Practice are selected, questionnaires are shown based on age * Front end validation * Entering data for multiple practices in same focus * Ordering of the assessments (Pre, Update, Post) * Required fields

CiMHLACDMH Triple PAll other PEI Practices Trauma Focused CBT MAP

* Clinical and administrative utility of collecting outcomes * Levels of evaluation * Quality improvement * Reports * Creating provider-to-provider learning

* Almost 26,000 treatment cycles for more than 23,000 clients * Data for 31 different EBPs * Data from 111 Legal Entities and 286 different provider site locations. * 381 users * Approximately 103,000 questionnaires saved

* Demographic client data and provider data * Beginning of Treatment information * Date of First Session * Starting Diagnosis * Clinician * Questionnaires (Pre, Updates, Post) * Scaled scores and/or total scores * Questionnaire administration date * Unable to collect and reason * End of Treatment Information * Date of Last session * Completed EBP * Measure of success * Disposition of Client * Number of Sessions * Ending Diagnosis

* Data collected can be used in a variety of ways * Pre/Post percent improvement in scores * Clients above and below clinical cut off scores * Duration of treatment * Percent completed vs. Drop outs * Outcome differences based on gender, ethnicity, primary language, age, focus, or EBP * Unable to collect reasons * Reliable Change

* CiMH Community Development Team (CDT) Model for CiMH supported practices * PEI Technical Assistance Site Visits * Practice Learning Networks * Service Area/Regional Outcome Data Workgroups

* Varying degrees of data entry challenges or communication around outcomes * The more outcomes are incorporated into normal routines the greater the compliance * Helping clinicians understand how data is valuable in interventions and as an assessment tool is important * Discussing outcomes in supervision sets a tone * Great variability in outcomes protocol and expectations from site to site

* Child Parent Psychotherapy (CPP) * Incredible Years (IY) * Group CBT for Major Depression (Group CBT) * Seeking Safety (SS) * Aggression Replacement Training (ART) * Crisis Oriented Recovery Services (CORS) * Families OverComing Under Stress (FOCUS)

* Recent Topics * Strategies for increasing data collection * Feedback loop of data to clinicians * Incorporating Outcomes into Supervision * Sharing Outcomes with Families * Availability of Training * General Technical Assistance Questions * Understanding the aggregate reports compared to site specific data.

* Funded through PEI Training, Technical Assistance and Capacity Building funds through a contract with CalMHSA, RAND Corporation facilitates workgroups comprised of providers and DMH administrative staff interested in reviewing Service Area outcome data for purposes of improving service quality and for developing strategies to make decisions based on the outcome data. * Goals of the workgroups include: * Provide a forum for Service Area providers to learn from each other in terms of practices that improve data quality and improve the use of data to make systems as well as clinical decisions within a Service Area. * Provide a forum for DMH administrative Service Area staff to develop their skills in utilizing reports to inform decision-making at the local level. * Strengthen Service Area capacity to formulate and conduct evaluations and to use appropriate data and analytic techniques to inform planning and resource allocations. * To develop strategies to review outcome data across programs and to use the analysis to help inform Service Area level decision- making.

Direct Support: * Outcome Measure Application (OMA) Labs Offered bi-weekly: all OMA staff are available in a one- stop lab to address user issues and train on reports. Providers drive content and focus. * OMA Users Group (In person and via Webinar) Meets Bi-Monthly: all Outcomes staff as well as CIOB and others meet and update interested users. The Users Group is also broadcast via WebEx for those who want to participate from their sites.

Publications and Broadcasts: * Egroups (OMAChat, PEI Alert, FCCS Alert) Specialized announcement/ discussion groups for important and timely announcements, including system alerts. * OMA Newsletter Quarterly reports on the state of the OMA offering tips and features about new aspects of the Outcomes effort. * Canned Users Groups and Trainings Pre-recorded Users Group sessions and special presentations are available through WebEx, Adobe Captivate, or as Windows Movie files.

References and Aids: * DMHOMA Wiki Project Website, covering all aspects of Outcomes (FSP, FCCS, SFC, Wraparound, PEI, and MHIP). As a wiki it is editable by any staff member. * Quick Guides Reference handouts, featuring tips for completing PEI questionnaires. * Worksheets, Spreadsheets, and other aides Fillable PDF and Excel files that help users calculate needed values and record dates and scores correctly.

Accessing the Data and Reports (Next Steps): * Internet Reports DMH-developed application to deliver specialized outcome data reports for providers on demand. * Electronic File Transfer: Legal Entity Extracts A secure transfer of a Legal Entitys outcome data allowing providers to analyze and manipulate their own data.

Kara Taguchi, Psy.D. Los Angeles County Department of Mental Health MHSA Implementation & Outcomes Division 213) http ://dmhoma.pbworks.com