Understanding Evidence-Based Practices Ventura County Behavioral Health April 29, 2009 Presented by Todd Sosna, Ph.D.

2 Topics MHSA Prevention and Early Intervention Research Informed Practice Levels of Effectiveness Rating Criteria Examples Implementation Considerations Intervention Categories Examples of EBPs

3 MHSA Spectrum of Services

4 Prevention Services provided to individuals who do not have any signs of a mental illness –Universal: Provided to the general public or a whole population group that has not been identified on the basis of individual risk. –Selective: Provided to individuals or subgroups whose risk of developing mental illness is significantly higher than average. Promotes and supports emotional well-being Prevents the development of mental illness No time limits imposed

5 Early Intervention Services for individuals with minimal signs of mental illness Short duration (less than 1 year*) and low intensity * except services for treatment of early signs of severe mental illness Prevents mental health condition from worsening Supports return to well-being Avoids need for more costly services

6 Research Informed Practice The effectiveness of a service is one important consideration in care planning, sometimes referred to as evidence-based practices--planning that integrates –Professional expertise and judgment –Consumer and family values and preferences –Best research evidence on the effectiveness of services –Based on the definition used in “Crossing the Quality Chasm: A New Health System for the 21st Century” (2001), by the Institute of Medicine

7 Why is Research Informed Practice? Mental health disorders can be complicated, severe, and difficult to prevent and treat The causes of these disorders are not fully understood Prevention and early intervention models are not effective for all individuals in all situations However, some interventions are more successful than others

8 Why Research Informed Practices? Evidence-based practices result in more individualized and hopeful treatment decisions, and –Reduce adverse consequences of imprecise care –Are more likely to be effective –Achieve outcomes sooner –Outcomes that last longer –Are ethical and cost effective, allowing limited resources to be used to serve more children and their families

9 Levels of Effectiveness The degree to which research indicates that a service is effective, or responsible for achievement of an outcome Levels of evidence are on a continuum Level of evidence is related to the quality of the research

10 Quality of Research There is a tendency to assume that if a treatment was provided and there is improvement then the treatment caused the improvement However, positive outcomes may be achieved as a result of a number of factors unrelated to treatment –Spontaneous recovery: Individuals naturally strive for health, try strategies and seek social support to reduce distress and achieve their goals; often this is successful! –Placebo effect: Improvement associated with non- specific aspects of treatment, for example, the expectation of improvement, that is independent of the unique characteristics of the specific practice; also can be successful!

11 Other Unrelated Factors Non-specific factors –Treatment leads to outcomes but for reasons unrelated to the theory or active components Independent factors –Employment –Friends Bias –Selection bias More hope and motivation –Consumers bias Want to be helpful –Providers bias Want to show success

12 Quality of Research Research is helpful in clarifying the effect of a practice independent of other factors that promote health, and independent of a proponent’s bias in favor of the practice Quality of research studies vary The higher the quality of research, the greater the confidence in the conclusions of the study

13 Quality of Research Qualitative studies –Anecdotal observations –Case studies Quasi- or partially controlled experimental studies –Within-subjects or longitudinal (pre and post comparison) studies –Between groups comparisons without assignment Controlled experimental studies –Random clinical trials (between groups) –Random clinical trial-longitudinal studies

14 Quality of Research Qualitative studies are especially helpful in developing theories and practice elements Quasi-experimental studies provide support for the effectiveness of a practice but do not control for the influence of important factors Controlled studies provide the strongest evidence that the practice, and not other factors, is responsible for achievement of specific outcomes Studies vary in their ability to answer questions about if the intervention works (internal validity) and with whom (external validity) the intervention works

15 Internal Validity Level of confidence that the practice is responsible for the outcome Answers the question “Does it work?” –Typically involves highly controlled research studies –Homogenous populations (for example, to diagnosis, gender, ethnicity) –Standard, verified application of the practice (for example, practitioner is highly trained and supervised by the developer)

16 External Validity Level of confidence that the practice will be effective across diverse groups of individuals Answers the question “Will it work in my community?” –Also, involves highly controlled studies –Heterogeneous populations (for example, dual disorders, both genders and diverse ethnicities) –Replication in diverse (public mental health) usual care settings with diverse clients and practitioners

17 Levels of Effectiveness Evidence-based practice –Clearly articulated model –Substantial and credible evidence of positive outcomes based upon experimental or equivalently strong research methods (replication) Promising practice –Clearly articulated model –Generally consistent evidence of positive outcomes based upon qualitative or quasi-experimental research methods (may have replications) Emerging –Clearly articulated model, sound theory, intention to evaluate

18 Reviews of Effectiveness SAMHSA National Registry of Evidence Based Practices pp.samhsa.gov/index.htm pp.samhsa.gov/index.htm SAMHSA--A Guide to EBPs on the Web

19 Confidence in a Practice No practice works with all individuals Confidence in achieving a positive outcome is increased when controlled research has demonstrated that the practice is effective, in real world settings and with individuals from diverse backgrounds

20 Fidelity or Model Adherence Degree to which there is adherence to the model (high quality) Model adherent programs are most likely to result in achievement of similar outcomes to those reported in the research

21 Model Adherence or Fidelity Degree to which there is adherence to the model (high quality) Model adherent programs are most likely to result in achievement of similar outcomes to those reported in the research Drift from the model (poor fidelity) can jeopardize achievement of outcomes Achieving fidelity or model adherence is an important consideration Requires the practice be ready to disseminate

22 Fidelity or Model Adherence Requires the community be ready to adopt –Consumer and family readiness –Staff readiness –Agency readiness –Service system readiness Implementing and sustaining an intervention with model adherence requires-- –Training –Coaching (ongoing) –Monitoring and evaluation (ongoing)

23 Causes of Program Drift Insufficient training or supervision Staff are not interested in or oppose the practice Practitioners with multiple or competing duties Failure to adhere to practice specific workload standards Insufficient intra- and inter-agency coordination around referrals, related services, and so forth

24 Causes of Program Drift No administrative level champion Little or not attention to fidelity monitoring Increased scrutiny and accountability Interest in adapting the practice before it is well-established Attrition of practice-proficient practitioners Delays between training and service provision Competing initiatives

25 Implementing Designate an administrator/manager lead to champion learning and using the model Develop a concrete intervention-specific implementation plan Select providers/staff based on a full understanding of the intervention requirements and commitment to achieving and maintaining fidelity Adhere to practice workloads and related intervention characteristics

26 Implementing Focus on fidelity from the outset Support fidelity thorough training, coaching, monitoring, and evaluation Maintain momentum Expect and plan for interrupted progression Expect and plan for staff turnover (replacement training)

27 Selecting Fit with target population including cultural relevance Fit with intended outcome(s) Level of demonstrated effectiveness –Level of research support –Internal and external validity Readiness to be implemented and sustained with model adherence –Tried and proven training protocols –Tools for monitoring model adherence and outcomes

28 Selecting Select a practice with a high level of demonstrated evidence Select a practice that is valued by consumers, families and community Select a practice with a history of successful implementation across diverse communities –Relevant to MHSA PEI priority populations and intended outcomes –Suitable for use in Ventura County –Culturally sensitive and responsive to the diverse communities that comprise Ventura County

29 Intervention Categories Education campaign --Universal Prevention –Triple P Parenting –Adolescent Transition Program Regular education curriculum --Universal Prevention –Incredible Years –Promoting Alternative Thinking Strategies Parenting program –Incredible Years –Triple P Parenting –Parent-Child Interaction Therapy –SafeCare

30 Intervention Categories Family therapy –Functional Family Therapy –Multisystemic Therapy –Multidimensional Family Therapy –Brief Strategic Family Therapy School-based parent/child program –Family and Schools Together –Strengthening Families Program –Adolescent Transition Program Comprehensive –Nurse Family Partnership

31 Intervention Categories Disorder specific early intervention –Trauma Trauma Focused Cognitive Behavior Therapy Cognitive Behavioral Intervention for Trauma in Schools –Depression Depression Treatment Quality Improvement –Psychosis Early Detection and Intervention for the Prevention of Psychosis Foster Care –Multidimensional Treatment Foster Care

32 Triple P Parenting Children 0-16 years of age Parenting program Five levels of intervention Universal prevention, early intervention, and treatment Individual and group modalities Numerous random clinical trials Real world (South Carolina) trial –Improves parenting skills –Decrease in parental stress and depression –Decrease in child behavior problems –Improves parent anger management skills –Decreases social isolation

33 Incredible Years Children 0-12 years of age Three sets of comprehensive developmentally based curriculums for parents, teachers and children to promote emotional and social competence Universal prevention, early intervention, and treatment Strengthens parents’ and teachers’ competence in communication, child directed play, clear limit setting, effective (nonviolent) discipline Numerous random clinical trials Use with diverse populations and settings –Less behavior problems –Increases in effective parenting –Less parental depression and increase in esteem

34 PATHS Children in elementary school Classroom curriculum to promote social-emotional competence and reduce behavior problems Self-control, feelings and relationships, interpersonal cognitive problem solving units Universal prevention Random clinical trials Use with diverse populations and settings –Improved self-control and ability to tolerate frustration –Use of more effective conflict resolution strategies –Decreased report of conduct problems and symptoms of depression and anxiety

35 Parent Child Interaction Therapy Children 2-8 years of age and their parents, at risk of or presenting conduct problems Parent guided (by therapist) in interacting with their child Early intervention and treatment Numerous random clinical trials Use with diverse populations and settings –Improved parenting skills –Improved child behavior –Improved quality of parent-child relationship –Improved parental affect and personal distress Developed by Eyberg at University of Florida UC Davis

36 Functional Family Therapy Youth years of age, and their families, showing family conflict or serious delinquency, violence and/or substance use Strength-based, phasic family therapy involving sessions Numerous random clinical trials Use with diverse populations and settings –Low treatment drop out rate –Reduction in violent behavior and criminal activity –Improved family interactions –Reduced younger siblings’ high risk behaviors

37 Strengthening Families Program Children 3-17 years of age Parent and family skills training program 14 weekly child and parent (individual and combined) skills building sessions Early intervention and treatment Random clinical trials Use with diverse populations and settings –Improved parenting skills –Improved child behavior –Improved family communication, and child problem solving and anger control

38 Nurse-Family Partnership First time, low-income mothers (any age) Selective prevention involving home visitation, by public health nurses, intensively supporting maternal-prenatal and early childhood health, and well-being, over a 2 year period Focus on parental roles, family and friend support, physical and mental health, home and neighborhood environment, and major life events (e.g. pregnancy planning, education, employment) Random clinical trials Use with diverse populations and settings –Improved prenatal health –Increased maternal employment –Reduced childhood injuries

39 TF-CBT Children 4-18 years of age with trauma symptoms Individual sessions (weekly) with the child, parent and joint child-parent (12-16 sessions) Therapeutic relationship, psycho-education, emotional regulation, stress management, connecting thoughts-feelings and behaviors, gradual in vivo exposure, cognitive and affective processing of trauma experiences, personal safety and skills training Numerous random clinical trials Use with diverse populations and settings –Decreases PTSD symptoms –Decreases negative attributes (self-blame) –Decreases externalizing problem behaviors –Improves parent-child relationship

40 DTQI Adolescents to young adults (ages 13-22) with depression Comprehensive approach to managing depression Screening and assessment, CBT psychosocial treatment, symptom monitoring and management, relapse prevention, individual and group formats Random clinical trials Use with diverse populations and settings –Improved mood –Decrease in depression symptoms –Decrease in suicide ideation and behaviors Developers Joan Asarnow (UCLA) & Maggie Rea, (UC Davis)

41 EDIPP Teenagers to adults at-risk of psychosis Universal prevention, early intervention, and treatment Educational campaign to reduce stigma and barriers to treatment, and increase identification of individuals showing signs of psychotic disorders by community members (e.g teachers, doctors, nurses, police officers, parents), and use of assertive case management model Promising practice Several community-based trials in process –Delayed onset of psychotic disorders –Reduced symptoms –Improved functioning

42 MTFC In lieu of group home care for children ages 3-5 (preschool), 6-11 (child), and (adolescent) Multi-level child and family-focused behavioral foster care program Numerous random clinical trials Use with diverse populations and settings –Increase foster parent competencies –Decrease in child behavior problems –Improved parenting –Decreases parental stress and depression –Increase in social support –Promotes reunification and reduces juvenile crime

43 EBP Common Features Clearly articulated models –Curriculum or phases or strategies –Specific intervention goals –Defined start and end –Can be replicated Emphasis on engagement as an early goal of intervention and responsibility of practitioner Specific target populations Specific target outcomes Grounded in research-based theory

44 Contact Information California Institute for Mental Health –web: –Todd Sosna – –Phone: (916)