Dennis Embry, PhD Paxis Institute Kate Lyon, MA James Bell Associates, Inc. Aleta Meyer, PhD Office of Planning, Research, and Evaluation Administration for Children and Families April 2012 Single Case Design: An Option for Evaluating Programs in Communities with Small Sample Sizes

2 Our Ancestors were everyday scientists They were scientists of their physical, social, and spiritual environments…

3 Humans are born as Everyday scientists…

4 What makes a good everyday scientist of behavior? Holds labels and past explanations lightly.

5 What makes a good everyday scientist of behavior? Ability to notice variation and sources of variation in. Physical environment Social environment

6 What makes a good everyday scientist of behavior? Ability to count behavior reliably.

7 What makes a good everyday scientist of behavior? Experiments by hypothesis testing.

8 What makes a good everyday scientist of behavior? Notices cause and effect in time.

9 What makes a good everyday scientist of behavior? Notices how behavior is adaptive to context.

10 Single Case Design: Rigorous Tests impact on a single case Determines general impact using small sample size Visual analysis of data Each case serves as its own control

11 Requires replication of effect with multiple cases to demonstrate causal evidence that intervention caused change in behavior Single Case Design:

12

13

14 Precisely define target behavior to allow for measurement of the relative occurrence of the behavior before and after an intervention Dependent Variable = observable target behavior that should be changed by the intervention

15 Your definition of a dependent variable should be: Objective Clear Complete Two Examples: 1.Increase positive parenting practices OR 2.Increase parents’ rates of verbal praise for children and decrease rates of verbal criticism

16 Dennis ’ definitions as a prevention scientist… Behavior = Reliably measurable actions (frequency, duration and/or intensity) of a person (adult or child) in a specific environment.

17 Frequency = counting how many behaviors in a certain time (e.g., per minute, per hour) What behaviors on your list might you count by frequency when at a home visit?

18 Duration = how long a specific behavior lasts (e.g., seconds, minutes, hours) What behaviors on your list might you count by duration at a home visit?

19 Intensity = positive or negative strength (i.e., voice, gestures, expressions) What behaviors on your list might you count by intensity at a home visit?

20 Frequency means? Vote A) How loud or soft a behavior is. B) Number of occurrences per unit of time. C) How long a behavior lasts. D) None of the above

21 Duration means? Vote A) How long a behavior lasts after learning. B) Percentage of behavior decay. C) Total time from start to stop of a behavior. D) A and B

22 Intensity means? Vote A) Qualitative rating of force, threat, or loudness of a behavior. B) Lumens per second of the behavior. C) Duration times the frequency of a behavior. D) All of the above.

23 Ask yourself 3 questions: 1. Is your dependent variable a behavior? 2. Is your definition: Objective? Clear? Complete? 3. Does your definition use: Frequency? Intensity? Duration?

24 What are your tools (not program) to increase or decrease the behaviors on your list at a home visit while you are there?

25

26 Independent variable = the practice, treatment or intervention being tested that should produce observable changes in the dependent variable Is it time limited? Can it be started and stopped? Why will the IV change DV?

27 What is your lever? Will it move your rock?

28 Which lever moved the rock?

29 The application of the independent variable is observed and measured over time

30 How do you test whether the lever moves the rock? Withdrawal and reversal designs Multiple baseline design Selection of design depends on the research question and objectives

31 ABAB (Reversal) Design

32 ABAB (Reversal) Design ABA Example… Teaching parents to increase their developmental delayed child’s competencies Benji & Mom Muir, K. A., & Milan, M. A. (1982). Parent reinforcement for child achievement: the use of a lottery to maximize parent training effects. J Appl Behav Anal, 15(3), doi: /jaba

33 Step 1: Multiple Baseline Design

34 Multiple Baseline Design with 3 Participants

35 Multiple Baseline Example… Teaching parents how to reduce children’s dangerous or risky behavior Embry, D. D. (1984). The safe-playing program: A case study of putting research into practice. In S. Paine & B. Bellamy (Eds.), Human Services That Work: From Innovation to Standard Practice (pp. 624). Baltimore, MD: Brookes Co.

36 You decide… Reversal design Multiple baseline design When would you use each? Which is better for you?

37 Establish a Stable Baseline stable or instable ?

38 Measurement of dependent variable must be monitored for consistency by measuring inter-observer agreement Dependent variable is measured repeatedly to allow for identification of behavior patterns Measures need to be sensitive to change

39 Must document three demonstrations of the experimental effect at three different points in time with a single participant or across different participants

41 Expanding your toolkit for influencing behavior change

42 Embry, D. D., & Biglan, A. (2008). Evidence-Based Kernels: Fundamental Units of Behavioral Influence. Clinical Child & Family Psychology Review, 39. Download at: FREE DOWNLOAD

43 Four Types of Evidence- Based Kernels Antecedent Kernel Reinforcement Kernel Relational Frame Kernel Physiological Kernel Happens BEFORE the behavior Happens AFTER the behavior Creates verbal relations for the behavior Changes biochemistry of behavior

44 Definition and properties of kernels Irreductable unit of behavioral influence (e.g. simple, cannot be made smaller) Have scientifically proof and peer reviewed publications showing effect using valid experimental design Produce observable behavior change quickly Often described in folk wisdom (e.g., grandmother told you) Easy to explain Can be used alone or combined with other strategies Are trialable Low cost or no-cost Can be used at many levels Individual, family, classroom School or community Used in environmental policies

45

46

47 Training Family Therapists Training therapist to praise parent Change in parent & child behaviors Parent scolding, reprimanding, negative to child Parent praise for positive child behavior Negative child behavior Positive child behavior Isaacs, Embry & Baer (1982), Training Family Therapists, JABA,

48 Applied Behavior Analysis An evaluation using a single- subject design (ABA model) needs back up if the initial intervention strategy fails to leverage change. ABA’s are not about finding a failure; rather about a solution. Takeaway: Build it until it works Huybers et al. J Appl Behav Anal Winter;37(4):

49 And. you can be big with single subject studies

50 Everyday scientists then and now

51 For more information… Kate Lyon, James Bell Associates (TEI/JBA) Julie Morales, James Bell Associates (TEI/JBA) Aleta Meyer, Office of Planning Research and Evaluation Carol Gage, Administration for Children and Families (ACF) Bridget Shea Westfall, ACF Moushumi Beltangady, ACF

52 Questions and Comments

53 The Tribal Home Visiting Evaluation Institute (TEI) is funded by the Office of Planning, Research and Evaluation, Administration for Children and Families, Department of Health and Human Services under contract number HHSP WC. TEI is funded to provide technical assistance to Tribal Home Visiting grantees on rigorous evaluation, performance measurement, continuous quality improvement, data systems, and ethical dissemination and translation of evaluation findings. TEI1 was awarded to MDRC; James Bell Associates, Inc.; Johns Hopkins Bloomberg School of Public Health, Center for American Indian Health, and University of Colorado School of Public Health, Centers for American Indian and Alaska Native Health. For more information on TEI contact: Nicole DenmarkKate Lyon Federal Project OfficerProject Director Office of Planning Research and EvaluationJames Bell Associates, Inc.