Human Services Research Institute Identifying and Developing Evidence Based Practices Presentation to: Implementing Evidence-Based Practices and Performance Measures for Massachusetts Mental Health Services: An Educational Forum Brandeis University September 28,2005 Steve Leff The Evaluation

Human Services Research Institute Who Wants to Identify EBPs? Evidence-based practices is a new metaphor to me. I thought evidence was the source of mental health practices. Are you saying it was not? May still not be? On what are practices then based? At no time have I ever heard that people in the mental health profession based their acts upon anything other than evidence. Was that a lie? Is this "new" metaphor also a lie? And how would I tell the difference? (H. A. Maio, personal communication, May 17, 2002) Consumers

Human Services Research Institute Who Wants to Identify EBPs? Oregon SB 267 requires that 25 percent of state fund be used to treat people with mental illness who use or have a propensity to use emergency mental health services. In , the percentage of funds to be spent on EBPs increases to 50 percent and in to 75 percent. Instructions for Filling out OMHAS Application for an EBP Review The Office of Mental Health and Addiction Services will consider any practice for inclusion on its list of approved evidence-based practices. To have a practice considered for this list the attached application must be filled out completely with all requested information. The Office will review the information provided in accordance to criteria outlined in the OMHAS Operational Definition for Evidence-based Practices...Practices that do not meet those criteria will be noted as reviewed. The office may request additional information if needed. States

Human Services Research Institute Who Wants to Identify EBPs? ValueOptions EVIDENCE-BASED PRACTICE GUIDELINES Use of evidence-based practice guidelines is the clinical foundation of disease management. One way in which this is accomplished is through the employment of consensus- and evidence-derived treatment algorithms, which incorporate current science, including strategies, and tactics for patient care. …In order to successfully implement an algorithm-driven treatment approach, ongoing training, technical assistance, and support for clinicians in the use of algorithms is necessary. Providers

Human Services Research Institute Identifying EBPs in My Crystal Ball EBPs will be identified by registries and venues for the reporting of meta-analysis There will be explicit criteria for recognizing practices Federal, state, and private payer listings of EBPs will converge Listings will become “Arbiters of value” for most, if not all consumers, payers and policy makers This will alter the nature of mental health services Favor some practices and not others Make favored practices more consistent and expensive

Human Services Research Institute Registries Registries employ scientists to review evidence for individual “brand name” interventions “Brand name” interventions package patented or copyrighted materials and training To use you have to purchase these products and services 35% of programs listed on SAMHSA’s National Registry of Evidence-based Programs and Practices were distributed by for-profit companies National Registry of Evidence-based programs and Practices (NREPP) - SAMHSA What Works Clearinghouse - DOE

Human Services Research Institute Meta-analyses Meta-analyses review generic practices which are sets of interventions with specified common characteristics They are usually not copyrighted or patented, often they haven’t been “packaged” Prepared and updated by scientists working independently or as members of organizations Published on the web, in journals, and in printed reports “Meta-analytically supported treatments” (MASTs) - Will Shadish (University of Memphis) Cochrane Collaboration - Campbell Collaboration Evidence-based Mental Health -

Human Services Research Institute Example of a Brand Name EBP: Multisystemic Therapy : Example of a Brand Name EBP: Multisystemic Therapy : MST Services is at the center of efforts to disseminate Multisystemic Therapy within the United States and internationally. MST Services provides assistance with all aspects of program development and staff training through licensing agreements with the Medical University of South Carolina and the Family Services Research Center. Core MST clinical training package and other components  Pre-training organizational assessment and assistance  Master's-level therapists or highly competent bachelor's-level professionals  initial 5-day training,  Weekly on-site supervision from doctoral level mental health professionals  quarterly booster training, and  monitoring of fidelity and adherence to the MST treatment model.  formal outcome tracking systems.  outcome-focused personnel evaluation methods.

Human Services Research Institute Example of a Meta-analysis Site The Cochrane Library The Cochrane Library (ISSN X). Published by John Wiley & Sons, Ltd. Vocational rehabilitation for people with severe mental illness Crowther R, Marshall M, Bond G, Huxley P Authors' conclusions Pre-vocational Training places persons in competitive jobs after a period of preparation. Supported Employment place clients in competitive employment immediately whilst providing on-the-job support. Supported employment is more effective than Pre- vocational Training in helping severely mentally ill people to obtain competitive employment. There is no clear evidence that Pre-vocational Training is effective.

Human Services Research Institute Feeding the Pipe Line Consumers will want new evidence based interventions proven to address recovery and other unmet needs Diverse populations will want interventions tested with their groups and in their geographic areas Providers and developers will want their interventions recognized as evidence based Registries and meta-analysis sources will want to identify additional evidence based practices

Human Services Research Institute Coming Soon To A Computer Near You Becoming Evidence Based: An Interactive Guide to Planning and Grading Effectiveness Studies The What’s Evidence Library: Selected Readings on the Nature of Good Evidence Meta 2 base: Summaries of Scientific Reviews of Mental Health Practices Visit The Evaluation at