Class 5: Evidence Based Interventions and logic models UTA SSW, SOCW 6371 Community & Administrative Practice UTA school of social work Dr. Dick Schoech Copyright 2009 (permission required before use) Suggest printing slides for class using: Print | Handouts | 3 slides per page | grayscale options

Overview of Sessions To Date 1. Generalist macro practice history, change process, roles, levels of intervention 2. Theories, perspectives, values 3. Problem definition, theories of need 4. Capacities and needs assessment 5. Evidence based interventions & logic models 6. Program descriptions, community linkages

Learning Objectives for Today  Understanding evidence & decision making  Understanding evidence based practice (EBP) & similar concepts (evidence informed practice is the term used at the UT Arlington SSW)  Understand difference between EBP & options  Understand traditional EBP perspective  Understand traditional EBP process  Understanding logic models and their use  Understand a performance mana EBP approach

Traditional Approaches to Decision Making  Common sense approach to decision making – what’s common to me makes sense – can be very limiting  Expert approach (but who is an expert)  Theory approach (too vague for detailed decisions)  Often if lacking evidence, use the bureaucratic structure (1:8 ratio staff to supervisors) to control for variance, but expensive (e.g., CPS)  Decision making too complex for management control so Street Level Bureaucrats (Lipsky)

More Scientific Approaches to Decision Making  Best practices = finding & using best possible way of doing something  established by history, utilization review, CQI, committees, etc.  Data mining = searching agency data for trends, predictions, performance on objectives and standards, etc.  Data visualization = display data in an easy-to-understand format so decisions are obvious from the data  Research = application of scientific principles to practice  Program evaluation = Review process, outcome, and impact of intervention, e.g., pre-post client measures, focus groups, key informant interviews, public opinion surveys  Clinical trials = experimental designs with randomly assigned control group & control for intervening variables. Look for significance & effect size  Fidelity = A measure of how well the intervention was replicated. For example, dosage is often important. If research shows that 15 training sessions are needed and clients on the average only attend 10 sessions, then fidelity is compromised.

Classic EBP Perspective Placing the client’s benefits first, evidence-based practitioners adopt a process of lifelong learning that involves continually posing specific questions of direct practical importance to clients, searching objectively and efficiently for the current best evidence relative to each question, and taking appropriate action guided by evidence. Evidence-based practice is "a total process beginning with knowing what clinical questions to ask, how to find the best practice, and how to critically appraise the evidence for validity and applicability to the particular care situation. The best evidence then must be applied by a clinician with expertise in considering the patient's unique values and needs. The final aspect of the process is evaluation of the effectiveness of care and the continual improvement of the process" (DePalma, 2000). DePalma, 2000DePalma, 2000

EBP Process 1. Ask answerable questions about the decision/problem/situation 2. Track down the best evidence to answer the questions 3. Appraise the validity, impact, and applicability of the evidence 4. Determine if something needs to or can be done 5. Inform those involved and consider their opinions and expectations 6. Integrate all forms of evidence into an action plan 7. Monitor and evaluate the processes and outcomes of the intervention

Understanding logic models and their limitations  Logic Model is tool for:  Explicating and communicating program theory, assumptions, rationale (Here is how we see it)  Assessing program feasibility (Is program needed)  Developing programs (How does program work)  Performance management (Tie outputs to mission)  Program evaluation (Assess if program works & why)  knowledge building (capture why things work)  Review logic models in template  Reading by Savaya gives examples also

Problems with Current EBP Approach  Originated in medical field, not typical SW practice  Evidence in academic journals, not easy to use  Evidence is old--need evidence in real time  Requires new generation of trained social workers  No time to track down evidence  Difficulty of interpreting complex research  Focuses on individual, not organization’s role  Evidence always lost, no “organizational learning”  Few practitioners evaluation practice as per EBP  Few consider evidence when select interventions  No evidence that EBP is better than other options  May be culturally destructive, e.g., Native Americans powwow to solve family problems vs. EBP

Key EBP Changes Needed  Web support e.g. help desk or travel agent example  FROM: individual practitioner searching down relevant research for each case  TO: organization with tools that make data readily available so that workers can monitor their performance on well-researched, outcome based, practice standards.  Source: Schoech, D., Basham, R., and Fluke, J. (2006). A Technology Enhanced EBP Model. Journal of Evidence-Based Social Work, 3(3/4),

CPS Example: Performance Mana EBP Indicator 1. Length of time to achieve reunification 2. Stability of foster care placement 3. Foster care re-entries 4. Length of time to achieve adoption 5. Recurrence of maltreatment 6. Incidence of child abuse and/or neglect in foster care Standard 1. At least 76.2% are reunified in less than twelve months from the time of the latest removal from home 2. At least 86.7% had no more than two placement settings 3. Less than 8.6% re-entered foster care within 12 months of a prior foster care episode 4. At least 32% exited care in less than 24 months from the time of the latest removal from home 5. Less than 6.1% had another substantiated or indicated report within six months 6. Less than.57% were the subject of substantiated or indicated maltreatment by a foster parent or facility staff

Illustration of DEMOS EBP Approach

Advantages: Performance Mana EBP  Quicker to implement  Staff correct data that impact their work  Responsibility shifts from the worker to the organization, consistent with Deming TQM theory  Questions to ask are in organizational performance standards tied to agency goals  Shift from bureaucracy to decision support  Instantly available  User customized models & interfaces (e.g., my.yahoo)

Disadvantages of Approach  Infrastructure is required  Agreement on performance standards is required  Costly in terms of technology – human services can more easily justify hiring people than IT  EBP requires a cultural change

Discussion & Questions