Defining and Measuring Behavior Kahng et al.(2011) Fisher et al. ch7

Introduction Behavior and the events that precede and follow it occur in a stream. One hallmark of ABA is precise measurement of observable behavior This involves measuring relevant dimensions of behavior from the stream. Measures are derived from relevant dimensions such as frequency, duration, Latency etc. Comments made here

Three Characteristics of a Measurement System Accuracy True values of the behavior dimension of interest are gained (measurement error) Validity : the system measures what it purports to measure ( higher with direct measures than derived) Reliability: The extent to which the system yields consistent outcomes over repeated measures. Same outcome over repeated measures, agreement between 2 observers These gauge its predictive value

Operationally Defining Behavior Behavioral definitions or observation codes are designed to control observer recording behavior Objectiveness: Observable events ( behaviors not concepts) that 2 or more observers agree occurred or did not occur. Clarity: Observers can read and paraphrase the behavior accurately. Completeness: The definition’s inclusion of all relevant instances and exclusion of irrelevant aspects

Functional VS Topographical Definitions Refers to some effect the behavior has on the environment ( E.G., door opening, where the outcome is important but not how it was achieved) TOPOGRAPHICAL: The form of the behavior is important (e.g., only the hands can be used)

Functional VS Topographical Definitions In ABA the form of behavior often is important. For example whether one asks for something as opposed to hitting other people to get it Sometimes the outcome is important and not the form For example property destruction may involve a variety of forms and we want to eliminate all of them

Humans & Machines as Observers In ABA we typically use human observers PRO: They are flexible & can record a variety of events as opposed to a few CON: They are subject to a variety of issues that render their recorded product invalid or unreliable

Human Observer Problems Errors and Biases Expectancy Bias: An observer’s expectation can bias what is recorded to agree with an hypothesis or desired outcome Give feedback on accuracy not behavior Have observers be “blind” as to the conditions or purpose of the experiment Periodically replace observers

Human Observer Problems Observer Bias. The presence of another observer may affect recording IOA is accurate but other times is not Surprise IOA; video record the events Reactivity. A participant’s performance is changed when being observed (e.g., employees in an OBM study) Observe without knowledge, allow for adaptation

Human Observer Problems Observer Drift. The accuracy of recording varies over the study Possibly due to boredom, fatigue, illness Periodically re-train, Introduce new observers Observer Training. The goal is accurate transduction of behavior to record (written or computer). Typically expose to a representative sample of the behavior and with a trained observer, providing review, practice and feedback until high agreement

Interobserver : Agreement IOA Collected on 25% to 30% of all conditions for all subjects Typically requires 80% -90% agreement NOTE: does not increase validity Both observers could be wrong Larry-compare to hard sciences “calibration” Low IOA indicates problems with the observation system

IOA Total Agreement Only compares the total amount or number of behavior events between 2 observers Smaller # divided by the larger and X100 for % Does not guarantee individual event agreement NOT recommended unless total behavior or product is of interest

IOA Interval Agreement Behavior recordings taken during frequent recording intervals of small duration (5-10s)are compared. Each interval or trial is compared and judged as agreement or disagreement. Agreements are divided by agreements & disagreements and multiplied by 100 for a percent agreement score Did behavior occur or not? A more conservative method is Exact Agreement of the frequency of behavior within each interval, or “block by block” get percent agreement for each interval and report an “average” Occurrence/nonoccurrence agreement. When behavior is at low rates blank intervals inflate agreement. # intervals with agreement divided by all intervals recorded by either observer or the opposite for high rates

Use of Technology Computerized data collection: very common Increases accuracy of analyses and graphing ability. Some systems are extremely accurate but take a longer time to analyze results Digital recordings (video): allows for greater accuracy and examples for observer training One drawback is the potential for loosing or erasing before analyzed. Mechanical Recordings. Advantages are great but cannot be followed blindly ( Larry, the calibration issue)

Direct vs Indirect Measurement Indirect Procedures. Permanent products: the tangible outcome vs the behavior Was the outcome produced by the behavior? Self Monitoring: Can be used with overt or covert behaviors Suffers from all the possible human biases Start with IOA on observable events and SR+ accuracy Interviews, ratings & surveys. Can be useful preliminary information to aid in direct observation to confirm relevant information

Direct Recording (continuous) Continuous. Record all occurrences Discontinuous or intermittent sample from all possible occurrences. Continuous. Recording on the basis of actual occurrence in an uninterrupted time flow. Frequency. If in different observation sessions convert to rate If in sessions or other opportunity based occurrence compare to opportunity or per cent etc. Best for similar discrete performance of equal duration each time

Direct Recording (continuous) Duration: amount of time the behavior occurs Need to define onset & offset accurately Need to observe continuously Latency: Time from signal to performance. Intensity. Magnitude, force, amplitude. Etc.

Direct Recording (discontinuous) Interval Recording. The observation session is divided into equal intervals (e.g., 10 sec) and behavior is recorded in each interval Whole interval. Scored as positive if the behavior occurred during the whole interval. Partial interval: if it occurred at all; ( predominative =>1/2) See Fig 7.1, 7.2, & 7.3 for differing outcome records using different parameters. ( e.g., PIR always over estimates over all duration of behavior; more accurate estimates of frequency require shorter duration intervals

Momentary Time Sampling (MTS) Behavior is judged to occur if it is occurring when a signal is supplied to the observer at the end of a fixed time period. Fig 7.1 MTS slightly underestimates behavior of varying duration and moderate rate. Fig 7.2 MTS grossly underestimates frequency & duration. Fig 7.3 MTS close estimate of duration and overestimate of frequency of long duration behavior

Conclusion See the decision flowcharts on Pages 126 & 127