1. Continuous Quality Improvement for CBT Groups Jennifer Lux, M.A. Research Associate, Corrections Institute Center for Criminal Justice Research University of Cincinnati First Annual Symposium Quality Assurance Continuous Quality Improvement

2. Objectives Discuss the importance of fidelity Identify and review key measures of CQI Review and practice how to provide CQI for group interventions –Using group facilitator evaluation form

4. Without Observation No chance to… –Reinforce good work –Correct mistakes or inconsistencies –Identify gaps in skills –Assure that there is fidelity to model

5. Fidelity to an Evidence-Based Practice Fidelity: Implementing the intervention as closely as possible to the way it was designed and delivered during the research stage Evidence-Based Practice: A practice that has been shown to work through use of scientific research

6. Evidence Based Practice is: Easier to think of as Evidence Based Decision Making Involves several steps and encourages the use of validated tools and treatments. Not just about the tools you have but also how you use them

7. Principles of Effective Intervention RISK WHO Deliver more intense interventions to higher risk offenders NEED WHAT Target criminogenic needs to reduce risk for recidivism RESPONSIVITY HOW Use CBT approaches Match mode/style of service to offender FIDELITY HOW WELL Deliver treatment services as designed The most effective programs are based on the principles of effective intervention:

8. Fidelity Principle Making sure the program is delivered as designed and with integrity: Ensure staff are modeling appropriate behavior, are qualified, well trained, well supervised, etc. Make sure barriers are addressed but criminogenic needs are targeted Make sure appropriate dosage of treatment is provided Monitor delivery of programs & activities, etc. Reassess offenders in meeting target behaviors

9. What Happens When We Don’t Have Fidelity? DRIFT Accidental adaptation can pose significant problems Too much adaptation might decrease an intervention's effectiveness

10. What Happens When We Don’t Have Fidelity? Schoenwald et al. (2003) –Therapist adherence to the model predicted post-treatment reductions in problem behaviors of the clients Henggeler et al. (2002) –Supervisors’ expertise in the model predicted therapist adherence to the model Sexton (2001) –Direct linear relationship between staff competence and recidivism reductions

11. What Happens When We Don’t Have Fidelity? Landenberger and Lipsey (2005) –Compared different cognitive-behavioral curricula to determine which was most effective –“Brand” of CBT didn’t matter but quality of implementation did –Implementation defined as low dropout rate, close monitoring of quality and fidelity, and adequate training for providers –Programs with good implementation demonstrated better outcomes than programs with poor implementation

12. Evidence Supporting Supervision Conditions –M–Manual Only –1–14 hour Workshop –W–Workshop + Feedback –W–Workshop + Coaching –W–Workshop + Feedback + Coaching All had initial skill acquisition Miller, et. al., A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing. Journal of Consulting and Clinical Psychology (2004)

13. Four Months Later Conditions –M–Manual Only –1–14 hour Workshop –W–Workshop + Feedback –W–Workshop + Coaching –W–Workshop + Feedback + Coaching Only condition that maintained benefit after 4 months Could not detect who had training

14. Is this true with cognitive- behavioral programming? Conditions –Manual Only –Manual + Web-based Training (40 hours) –Manual + Training + Supervision (observation and feedback) Sholomskas, et. al., We don’t Train in Vain: Three Strategies of Training Clinicians in CBT. Journal of Consulting and Clinical Psychology (2005)

15. Cognitive-Behavioral Intervention Conditions –Manual Only –Manual + Web-based Training –Manual + Training + Supervision No Transfer Modest Transfer Proficient Sholomskas, et. al., We don’t Train in Vain: Three Strategies of Training Clinicians in CBT. Journal of Consulting and Clinical Psychology (2005)

16. THE IMPORTANCE OF FIDELITY

17. Every major study shows a strong relationship between program integrity and recidivism INTEGRITY RECIDIVISM

18. What Do We Know About Fidelity? Fidelity is related to successful outcomes (i.e., recidivism reductions) Poor fidelity can lead to null effects or even iatrogenic effects Fidelity can be measured and monitored Fidelity cannot be assumed

19. Some Lessons Learned from the Research Who you put in a program is important – pay attention to risk What you target is important – pay attention to criminogenic needs How you target offenders for change is important – use behavioral approaches Program Integrity makes a difference - Service delivery, training/supervision of staff, support for program, QA, evaluation, etc.

20. CQI – What Is It? A method of continuously examining processes and making them better Key principles: –Use of data and team approaches to improve decision making –Involvement of entire organization to improve quality –Strong focus on customers –Continuous improvement of all processes and outcomes

21. CQI versus QA QA: QUALITY ASSURANCE Retrospective review process Emphasis on regulatory and contract compliance Catching people being bad leads to hide and seek behavior 21

22. CQI versus QA CQI: CONTINUOUS QUALITY ASSURANCE A prospective process Holds quality as a central priority within the organization Focus on customer needs; relies on feedback from internal and external customers Emphasizes systematic use of data Not blame-seeking Trust, respect, and communication Move toward staff responsibility for quality, problem solving and ownership of services 22

23. Benefits of CQI Proof of effective services Maintain or secure funding Improve staff morale and retention Educate key stakeholders about services Highlights opportunities for improvement Data to inform quality improvement initiatives

24. Benefits of CQI Establish/enhance best practices Monitor/ensure treatment fidelity Evidence that programs utilizing CQI/evaluation processes demonstrate better outcomes Engaging in CQI helps organizations to achieve their mission

25. CQI PROCESS FOR GROUP INTERVENTIONS

26. Feedback and Coaching

27. Place emphasis on learning process – don’t try evaluation until a pre-planned learning curve period Place emphasis on improving, rather than criticizing

28. Fidelity to Model Not a discussion of RIGHT or WRONG Key is to identify what is CONSISTENT OR INCONSISTENT with the model

29. Process of Supervision 1.Direct observation of sessions 2.Structured feedback about adherence and competence 3.Coaching to improve implementation with proficiency 4.Continue evaluation with frequency until reach proficiency

30. Coaching Format Staff should be afforded the opportunity to share their impressions of the session Coach should provide direct and specific feedback - Show them the rating form - Review strengths - Areas for improvement, give specific examples Concrete plans should be developed to address areas needing improvement

31. Rating Form Should include facilitator ratings in the following general areas: –Knowledge of cognitive-behavioral interventions and ability to explain new skills and concepts –Adherence to curriculum –Group behavior management –Ability to effectively communicate and build rapport with participants

32. One example… Let’s review the Group Facilitator Evaluation Form

33. Practice Activity Watch the following group demonstration and use your Group Facilitator Evaluation Form to record observations

34. DESIGNING AN ONGOING CQI PROCESS

35. Develop Proficient Staff Define proficiency –Provide information to staff Once rated as proficient, decrease observation frequency

36. For Non-Proficient Staff Define proficiency –Provide information to staff Work on change planning until proficiency is reached

37. Change Planning Plan of correction Proactive approach to problem solving Empower staff Use objective data to inform decision making

38. Use of PDSA Cycles PLAN – analyze a process, determine what changes are needed, and establish a plan for making the changes. DO – implement the changes on small scale. STUDY – check to see whether the changes are working. ACT – if changes are working, implement changes on larger scale. If not, begin the cycle again.

39. Strategies for Improvement Role play Trial and error Written vignettes – if client says... you respond _______. Role play and make common mistakes – have them identify mistakes and how to improve upon those Coach develops a pictogram or flow chart of methods Project client responses Rewards and consequences

40. DEVELOPING YOUR CQI TEAM

41. Effective CQI Leadership Who are the leaders? What makes them good leaders? What is their role in the CQI process? Management vs. Leadership

42. What Can You Do? Know the mission of your organization and your role in helping to achieve that mission Know the evidence-based practices for your client population and try to apply them with fidelity every day

43. What Can You Do? Become an active member of the CQI process Communicate with others about the value of CQI Learn to look for available data prior to engaging in problem- solving Understand that acknowledging that the agency can improve its performance on a particular measure does not mean that the agency is saying that staff are not competent at their jobs. Adopt a personal philosophy that you want to be better at your job tomorrow than you are today.

44. Summary The relationship between strong fidelity and decreased recidivism is proven Group interventions can be supervised using CQI methods The goal is to guide facilitators to proficiency

45. Questions Jennifer Lux University of Cincinnati Corrections Institute Jennifer.lux@uc.edu