+ Evidence-Based Practice in Psychology and Behavior Analysis By William O’Donohue & Kyle E. Ferguson University of Neveda, Reno Summarized by Chloe Ruebeck High Incidence Grant University of Utah

+ History- EBP Movement Clinical practice guidelines (CPG) were created 20 years ago to help standardize the decision-making process for treatment planning by using valid and empirical treatments APA began developing CPGs, in 1991, to assist in the decision making processes in the psychiatric field Psychiatrists created a pharmacological bias, even though neither medication or alternative treatments were shown to be superior

+ History Cont. APA addressed concerns about bias with the Division 12 Task Force on Promotion and Dissemination of Psychological Procedures (Chambless et al.,1996) This task force came up with criteria for efficacious treatments Therapies can fall into two groups, well-established treatments or probably efficacious treatments

+ Criteria for EBP Well-Established Treatments I. At least two good between group design experiments demonstrating efficacy in one of two ways: A. The treatment is statistically significant to the placebo B. The treatment is = to already established treatment with a sufficent sample size OR

+ Criteria for EBP Cont. II. More than a series of 9 single subject experiments demonstrating efficacy. III. Experiments must have treatment manuals. IV. Participant criteria must be stated. V. Results must be replicated by at least two different research projects.

+ Criteria for EBP Cont. Probably Efficacious Treatments I. Two experiments that show the treatment is statistically significant to a waiting-list control. OR II. One or more experiments meeting all the criteria except criterion V. III. Three or fewer single subject designs otherwise meeting all previous requirements for well-established treatments.

+ Weaknesses of EBP 1. The studies on the list were chosen because of statistical significance and not clinical significance. 2. They were selected based on efficacy not effectiveness. 3. Many heterogeneous patient populations clinicians work with from day-to-day were excluded. 4. Bias for group design over signal subject and analytical methods.

+ Statistical significance vs. clinical significance Statistical significance merely means that the observed difference between means was not due to chance. Clinical significance refers to the extent to which the treatment is meaningful. If a treatment is clinically significant an individual can move from an undesirable category to a more desirable category. Chambless et al. did not take clinical significance into consideration.

+ Efficacy vs. Effectiveness EBP pertains to efficacy NOT effectiveness. Effectiveness refers to the extent to which the treatment can generalize to other settings.

+ Participant characteristics: Heterogeneity vs. homogeneity In many instances, co-morbid patients are excluded from research. This is a major problem because co-morbid patients are more the rule than the exception. It is very important to find treatments that serve co-morbid populations.

+ Group design and inferential statistcal bias. It only takes two between group designs experiments to be considered well-established, whereas it takes 9 or more single subject experiments. Signal subject rarely use inferential statistics and they usually compare treatment to baseline conditions and not a placebo. These factors exclude them from established treatment consideration.

+ Well-established and probably efficacious treatments with a behavior analytic focus See Article for full list.ee article for full list

+ Causes for Concern 1. Old School vs. New School Behavior Analysis 2. Certainly The Law of Effect Does Work 3. Developmental Disabilities are Conquered, What about all the other problems? 4. Leaving Quality Improvement behind 5. Beating the competition

+ Old School vs. New School Most of the contributions to this field were made by first generation behavioral analysis. There have been fewer recent contributions. 1. What is the rate of discovery of new behavioral principles? 2. What is the rate of using these new applications successfully in new ESTs?

+ The Law of Effect There is a constant repetition of studies in the Journal of Applied Behavioral Analysis. There needs to be new studies done on new behavioral analysis techniques.

+ What About Other Problems?  JABA has an overwhelming focus on developmental disabilities.  It is thought that behavior analysis could greatly benefit the healthcare sector by helping to manage the aging population and chronic disease.

+ Leaving Quality Improvement Behind The healthcare field is too stuck on the role of science. More focus needs to be put on implementing sound quality improvement systems. The “Theory of Bad Apples” is used in the healthcare world, but the “Theory of Continuous Improvement” would be more efficient.

+ Suggestions to Improve EBP 1. Basic education in quality improvement 2. Technologies to further understand consumer needs 3. Understanding service delivery and how it affects outcomes 4. Consistent use of quality indicators 5. Reliable, affordable, information systems that capture quality indicators.

+ Suggestions Cont. 6. Learning trials and EPB that is continually improved 7. Incentive systems for meeting or exceeding quality goals and making new suggestions that work 8. Benchmarking 9. Report cards so the purchasers can be educated on the service provided

+ Beating the Competition  What should be done about the practice that is not evidence based?  What is deemed bad behavioral health practice? These are the difficulties behind those ?’s 1. Lack of consensus of treatment standards 2. Little prior orientation to this sort of question in the field

+ Beating the Competition Cont. 3. Reliance on “Bad Apple” theory which looks down upon errors 4. A relativistic ethic, where all perspectives are seen as equal 5. An idiographic view of case formulation 6. Ignoring bad practice rather than fighting it

+ What Can Be Done About Unethical Treatments? 1. Behavior analysts need to educate the public 2. Filing unethical charges against practitioners using these interventions 3. Educating watchdog groups 4. Work with district attorneys and other law enforcement agencies 5. Working with government agencies to pass laws criminalizing these acts

+ Quality Improvement Is continuously “upgrading” products and services in keeping with consumers’ ever changing demands and expectations Needs Improvement 1. Improve productivity 2. Utilize interesting technologies 3. Improve value of services 4. Sensitive to unintended negative effects 5. Shift from short-term to long-term goals

+ Improving the System “You can’t improve a process or outcome without measuring it” and “don’t waste time and effort measuring it if you aren’t going to try and improve it” Take home message- the behavior analysis system needs improvement!