Social Validity and Treatment Integrity Also, finishing up single subject research

Baseline Terminology and Comprehension What is it? Why is it important? What do we need to know? Baseline Predictive Function Trend Decreasing Baseline Increasing Baseline How do I compute baseline stability and what is it?

Explain the Design AB Design & Changing Criteria Reversal Design Alternating Treatments Design Multiple Baseline Define/Purpose Limitations and Strengths Graph and explain an example use based on a hypothetical experiment

Single Subject Research: Assessment Observable Behavior Only Typically does not Use indirect measures Identified a target behavior Considers all observable human activities a “behavior” Internal thoughts feelings exist, but you can not observe them.

Observable Events Antecedent (Stimulus) Events- events that happen or precede target behaviors Consequence Events-events that happen after target behaviors The antecedent events of hanging out a the pub this week seems to have affected the consequence event of a D on the test!

Decision Tree for Data Systems Academic or Social Behavior Related to numerical dimension Related to time dimension Prior to Response Time between initiation response and end of behavior Discrete Discrete or Continuous Event Recording Moderate frequency High Frequency Latency Recording Duration Recording Interval Recording Time Sampling

Interval/Time Sample Observations One good way to record behaviors in a time sample is to have a pre-set key where you mark or circle the behavior. Coded Symbols are good way to mark operational observable behaviors Pre-recorded signals that are precisely timed is one common method of using Interval observations

Duration &Latency Forms Duration recording forms record the length of time from when a student starts a behavior On the other hand, Latency is recorded from when you give a cue or instruction, and you are recording the time until a student starts a behavior. This duration form includes other important information to be coded.

Whole vs. Partial Interval recording is may not be practical for the teacher, but is desirable for experimental conditions- you may have to strike a balance However, a whole or partial time sample may work A whole is where the behavior occurs throughout the interval- probably not workable for most situations A partial time sample if a behavior occurs during the interval of the sample All time samples have limited accuracy In time sampling you could miss low frequency behaviors

Find the Social Validity Concepts. How are they related? Pre-intervention Post-intervention Why do we assess Whom do We assess

“The degree to which an intervention is implemented as planned” Treatment fidelity Intervention fidelity Treatment Integrity “The degree to which an intervention is implemented as planned” Procedural fidelity Treatment plan implementation Intervention integrity Hagermoser Sanetti, 2008

Why care about treatment integrity (Hagermoser Sanetti, 2008)? 1. Referral/ assessment data suggest intervention is warranted 2. Evidence-based intervention selected & implemented 3. Student outcomes (SO) assessed 3. Treatment integrity (TI) assessed Data reviewed 5a. Continue intervention + SO + TI Impacts the evaluation of services we provide to any and every student. Impacts the internal validity of experimental studies—if we want to know if X treatment caused Y effect, we have to know, not assume, that treatment X was in fact implemented. As the field moves to an RTI framework in which all students are provided with a high quality, empirically supported instruction and behavioral supports and their progress is monitored. In an RTI model, student’s reponse to the provision of EBIs is the primary factor thtat determines the level of intervention required—to make vaid decisions about a student’s need for additoinal services, practitioners will need to assess TI. The decision of providing more intervention to a student with small-group or individualized interventions could be crtical to the academic and potentially earning potential of a student—Danielson, 2007-- 4. Data-based decisions 5b. Implement strategies to promote treatment integrity - SO - TI - SO + TI 5c. Change intervention

Framework for assessing treatment integrity (Power et al., 2005) Hypothesized dimensions of treatment integrity; Hagermoser Sanetti, 2008 Framework for assessing treatment integrity (Power et al., 2005) How much of the intervention was implemented? How well was the intervention implemented? Adherence Extent to which specific program components delivered as planned Exposure Index of the number, length, frequency, of sessions, Program differentiation A manipulation check performed to safeguard against diffusion of treatments Power and colleagues (2005) combined the dimensions of treatment integrity proposed by Dane and Schneider (1998) with treatment integrity assessment guidelines proposed by Gresham (1989) into a framework for treatment integrity assessment. The framework suggests that two estimates of integrity and five dimensions of integrity should be considered in a thorough assessment of treatment integrity. The two estimates of integrity are component integrity (i.e., an average rating for a specific intervention component) and daily integrity (i.e., an average rating for all intervention components combined). Power et al. conceptualized each dimension as being either content or process related. According to this conceptualization, adherence, exposure, and program differentiation, address the content of the intervention and answer the question “How much of the intervention was implemented?” The remaining two dimensions of treatment integrity, quality of delivery and participant responsiveness, address the process of implementing the intervention and answer the question “How well was the intervention implemented?” Power and colleagues suggest that to execute a thorough assessment of treatment integrity that includes both estimates of and all of the dimensions of treatment integrity, it is essential to use a partnership model in which the responsibility for planning and executing treatment integrity assessment is shared by researchers and interventionists. This framework for assessing treatment integrity is the most comprehensive proposed to date. With regard to the estimates of integrity within the BC literature reviewed above, researchers who have provided quantitative data in their reports have focused solely on daily integrity. However, for many interventions, some intervention components are conceptualized as being essential for behavior change while implementation of other components is ideal but not essential (Waltz, Addis, Koerner, & Jacobson, 1993). Thus, assessing and reporting component integrity will allow data-based (as opposed to theory-driven) determinations as to which components of an intervention are truly essential to positive treatment outcomes. As mentioned above, research regarding the dimensions of treatment integrity is just emerging and additional studies are needed to confirm this conceptualization. Quality of delivery Qualitative aspects of the intervention— enthusiasm, effectiveness, preparedness Participant responsiveness The degree to which intervention participants were engaged in the program Daily estimate: average rating for all intervention components Component estimate: average rating for individual components

Quick Critical Analysis: Single Subject Study What are the questions and Purpose of my article? What kind of measurement approach/ instrumentation did they use? Did the establish reliability and how? What did they do establish validity of instruments? What was the design and what did they do to insure it was valid? Did they consider social validity or treatment acceptability? What were the findings and was the discussion consistent with the findings? What did the authors list as limitations? Do you think study is replicable and/ or generalizeable?