Data Collection and Analysis

Baseline Data Whenever possible collect baseline data. This is most important for behavior programs. Also very useful for academic programs when student knows some of the skill but is not proficient. If student has never demonstrated skill it is not necessary to collect baseline - we know it is 0 Having baseline data gives greater evidence that intervention/instruction is responsible for the behavior change. Collect several days of data when possible.

Baseline Data

Frequency of Data Collection Once or twice per week data collection is sufficient for most academic skills. Don’t collect more data than you need or can use. Determine data collection schedule prior to start of day or week (e.g., collect data during workbox activities on Wednesday). Ensures that student behavior on a specific day does not influence data collection.

Frequency of Data Collection Data collected should be representative of overall student behavior not just when student is demonstrating challenging behavior or exceptionally positive behavior. If practicable, collect data on each significant incident of challenging behavior. Frequency data

Frequency of Data Collection Collect data on instructional trials as often as possible. Many students will make progress rapidly frequent data collection will show this progress and lead to a more responsive instructional program.

Frequency of Data Collection When teaching specific skills such as, using a schedule it is best practice to collect data on each trial. Generally these are task analyzed skills and taught at relatively low rates (2-5 times a day). Graph data on trials in which student was able to complete skill independently.

Event Recording or Frequency A tally or count of discrete behaviors as they occur. Examples: dropping to floor, physical aggression, words spoken, talk outs, directions followed. Behavior should be discrete, having a definite beginning and end. Behavior should not occur at very high rates. When expressed in a ratio with time it is a measure of rate.

Rate Rate - number of occurrences in a specific period of time. Rate is useful measure of fluency or how quickly a student can perform a task. Most common rate measure is words read per minute. Rate is also a useful measure in addition to accuracy for workbox activities. Fluency and rate are important for future employment of many of our students.

Rate Assembly or sorting tasks Collect data on the number of items assembled or sorted correctly out of items possible. Provides measure of accuracy. Collect data on the time it took to complete task. Provides a measure of rate. Many students can be taught to start timer at beginning of task and stop timer at end of task.

Accuracy Percentage of correct responses. Example: Number of trials in which the student correctly performed the behavior divided by number of opportunities. Very common and easy measure to use. Can also be used when collecting data on behaviors that can be broken into steps (task analyzed). Independent responding is considered a correct response.

Accuracy Be careful when comparing percentages if opportunities for behavior are not the same. Use percentages or accuracy only to compare same number of opportunities over time. 50% is really not the same if it is a comparison of 1 of 2 trials vs 10 of 20 trials.

Rate and Accuracy Flashlight assembly At end of task 5 of 6 flashlights assembled correctly and work properly. Task took student 5 minutes. Accuracy: 5 assembled correctly out of 6 5/6 = 83% Rate: 5 completed correctly in 5 minutes 5/5 = 1 per minute

Rate and Accuracy Workbox sorting task Sorting by color (red, blue, green, yellow, purple) 5 of each color, 25 possible correct responses Student places 20 of 25 correctly = 80% correct Student completes task in 5 minutes - 20/5 = rate of 4 per minute (20 done correctly). Rate could also be calculated as overall items in 5 minutes (25/5 = rate of 5 per minute.)

Rate and Accuracy Reading fluency - WRC and Errors is also a measure of rate and accuracy. One of the most common measures used in education.

Duration The extent of time a behavior occurs during an observation period. Examples: Length of time student could wait. Length of time student worked on independent task. This measure shows how long a student engages in a particular behavior.

Duration Per Occurrence The extent of time of each occurrence of the behavior. Examples: Length of time student was able to work at independent task. Length of time student could wait in lunch line. This measure provides data on the number of occurrences, duration of each occurrence, and total duration. Provides more information about the target behavior than total duration does.

Level of Assistance Most often used for students that need differing levels of prompts. Provides record of how much external assistance was needed. Examples: full physical prompt, partial physical prompt, verbal prompt. Useful descriptive information. Graph only behaviors/steps that student can perform independently.

Time Sampling Momentary Time Sampling - Behavior immediately following specified time period is recorded. Example: on-task behavior. It is easy to use and will not interfere with other ongoing activities of the observer. This method can be used easily for either individual or group behavior. Excellent for behaviors that are not discrete. Socializing/playing at recess See attached data sheets

Time Sampling Variety of time intervals can be used. 10 second; 5 - 15 minute 1 minute; 15 minutes 5 minute; 25 minutes Use a consistent measure for comparisons. Can be presented in a bar graph to demonstrate a discrepancy in comparison to peers.

Data Collection Considerations What type of data will be collected? Dictated by the goals and objectives or the ultimate purpose of the intervention. Reduction in challenging behavior, increase in independent responding etc. Be cognizant of data collection when writing goals. When writing the goal or objective ask: Is this something that can be reliably measured? Is it observable and quantifiable? Can I measure this in reasonably efficient manner?

Data Collection Examples Schedules are used to increase independent transitioning. Collect data on: Student’s progress in learning steps to using schedule independently. Frequency of student independently transitioning from one activity to another.

Data Collection Examples Schedules are used to decrease challenging behaviors at transition Collect data on: Frequency of challenging behaviors at transitions prior to and after schedule is implemented. Baseline data should be collected. Remember to collect data on student’s progress in using skill independently.

Data Collection Examples Workbox system is used to develop or increase ability to work independently. Collect data on: Student’s ability to complete tasks independently. Number of tasks completed. Duration of time student worked independently. Number of prompts needed.

Data Collection Examples Workbox activities are used to maintain skills. Collect data on: Rate and accuracy of task completion. Number of items completed correctly. Speed at which task was completed.

Data Collection Examples Social Skills Following instructions, accepting corrective feedback, etc. Common to use point sheet to collect data on behavior goals. Point sheets, at best, provide general indictor of skill use.

Data Collection Examples Social Skills Data can then be collected on the the number of times the student demonstrates the the individual skill steps vs the number of opportunities. Accepting corrective feedback ___will increase his ability to accept corrective feedback from refusing to accept feedback on 7 of 10 trials and raising his voice, arguing, screaming, hitting, kicking, or throwing things to calmly accepting feedback on 8 of 10 trials through the use of social skill instruction, visual supports, praise, positive and corrective feedback, logical consequences, and other behavior management techniques.

Data Collection Examples Count number of times student accepted feedback divided by opportunities (number of times feedback was given). Provides a concrete measure of actual skill use (obtain a measure of accuracy). Number of times correct = 2, Opportunities = 3 Student correctly responded on 2 of 3 trials during this observation. Next observation do the same and add up total trials and total correct (2/2) Total for two observations 4/5

Data Collection Examples The same procedure can be used to collect data on social skill objectives. Collecting data on scaffolded objectives is simple and easy. Data collection can be part of instruction and done at time of instruction. Demonstrates that student knows the steps to the skill and how to use skill even if he/she is not yet using the skill in the natural environment.

Data Collection Examples 1: When asked, ____ will verbally identify the steps to accepting feedback or a consequence (look in the direction of the person, say "okay," and don’t argue) with 100% accuracy for 3 consecutive trials as measured by teacher observation. 2: In a role-play, ____ will practice using the steps to accepting feedback or a consequence (look in the general direction of the person, say "okay," and don’t argue) with 100% accuracy for 3 consecutive trials as measured by teacher observation.

Data Collection Examples 3: When given corrective feedback or a consequence ____ will increase his behavior of accepting feedback by using the steps to accepting feedback 6 of 10 opportunities of the time as measured by periodic observation. 4: When given corrective feedback or a consequence ____ will increase his behavior of accepting feedback by by using the steps to accepting feedback 8 of 10 opportunities as measured by periodic observation.

Data Collection Examples ______ will increase positive response to directions from following 4 of 10 instructions and arguing, hitting, kicking, throwing objects or leaving the area to promptly following 8 of 10 directions through the use of social skill instruction, visual supports, praise, positive and corrective feedback, logical consequences, and other behavior management techniques.

Data Collection Examples Monitor number of directions given and number of directions student followed without exhibiting challenging behavior. Remember 1X per week is usually sufficient Collect data when student is given typical directions in resource room or mainstream classroom. If provided support in mainstream have support staff monitor directions given by teacher and number followed by student.

Data Collection Examples _____ will increase his ability to remain on-task from being on-task an average of 60% of the time to being on-task 85% of the time through the use of skills training, classroom modeling, teacher feedback and praise.

Data Collection Examples Momentary time sampling Remember 1X per week is usually sufficient Special Ed teacher or para can collect data Collect data when student is in resource room or mainstream classroom. If possible, during both teacher lead instruction and independent work time. If provided support in mainstream have support staff could collect data for 5-10 minutes. Remember to communicate with mainstream teacher that support staff is to collect data during this time.

Interpreting and Analyzing Data

Interpreting and Analyzing Data After data is collected, draw a graph and use graph to evaluate student progress and to formulate instructional decisions. There is no substitute for a graphic display of data. Graphic displays do not need to be complex or fancy.

Interpreting and Analyzing Data A simple hand drawn graph is sufficient.

Interpreting and Analyzing Data There are two basic methods to interpreting academic data: Decision rules based on the most recent 3-4 consecutive scores. Decision rules based on the trend-line.

Goal-Line Versus Student’s Current Rate of Progress Examine both level and rate of student progress to determine whether student is progressing adequately to reach end-of-year goals Compare student’s current rate of progress with projected rate of progress (i.e., goal-line): To judge whether the instructional program needs to be modified to better meet student needs or To determine whether the goal should be raised

General Decision Making Framework 4-Point Rule (supersedes the trend-line rule): If 3 weeks of instruction have occurred and at least 6 points have been collected, then examine the four most recent data points: If all four are above goal-line, then increase goal. If all four are below goal-line, then make a teaching change. If the four data points are both above and below the goal-line, then keep collecting data until trend-line rule or 4-point rule can be applied.

What Is the Data-Based Decision Rule? The 4-point rule may be applied: Data-based decision is to raise the goal.

General Decision Making Framework Trend-Line Rule: If 4 weeks of instruction have occurred and at least 8 data points have been collected, then figure trend of current performance and compare to goal-line: If trend of student progress is steeper than goal-line, then raise goal. If trend of student progress is less steep than goal-line, then make a teaching change.

What Is the Data-Based Decision Rule? The trend-line rule may be applied: Data-based decision is to make an instructional change.

Benchmark Report - Box & Whiskers Graphs (box plots) AIMSweb commonly uses box plots to report data. This chart will explain box plots: Consider bell-curve. Box plots are somewhat similar in shape and representation. outlier Above Average Range 90th percentile 75th percentile Median (50th percentile) 25th percentile Average range of population included in sample. Below Average Range 10th percentile

Strategic Monitor Individual Student Report

Progress Monitor Report

Progress Monitor Report

Percent vs Percentiles Percent refers to the proportion of the whole thing, as in He got 25% right on the test, or 90% of the kids passed the test. Essentially, it answers the question, How much? or What part of 100. A percent is absolute, referring to the exact part of the whole.

Percent vs Percentiles Percentile is often used as a shortening of the term percentile rank. In the world of assessment, a percentile rank describes how a score fits in to the distribution or spread of scores in the comparison group. If a student scored at the 40th percentile, he scored better than 40 percent of those in the comparison group. It answers the question How well compared to…? A percentile rank is a score relative to the comparison group, and is totally dependent on how everyone in that group performs. AIMSweb box plots are percentile ranks.

Behavior Decision Rules Some time after 5 days (minimum number for evaluating an intervention) but no longer than 15 days (maximum number of days for allowing any program to run before change in behavior is expected), analyze the data and if necessary develop a program change. Purpose is to compare data before the intervention to that following the intervention using analyses of (a) change in level, (b) slope, and (c) variability.

Variability Variability or bounce in data refers to the range of scores in the data collected. Can be an indicator of a number of issues such as low motivation, ineffective prompting, inconsistency in implementation, inconsistency in expectations across staff.

Indicating finished

Variability If the student’s data indicates considerable variability or the program is not effective, check for fidelity of implementation issues before making other changes.

Fidelity of Implementation Fidelity of implementation occurs when staff use the instructional strategies and deliver the content of the intervention in the same way that they were designed to be used and delivered. Frequency and consistency are critical to fidelity of implementation.

Data Collection Summary Keep the data collection simple and useable. Instill in your staff that data collection is as important as instruction. Always use the data to drive instruction. Use ready made tools to simplify the process.