2. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Overview of measurement, assessment, and evaluation July 31, 2003

3. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Todays agenda Welcome (5 minutes) –Please remember to click –Bonus for reminding me about Free/Writes Questions & answers (5 minutes) Student presentations (45 minutes) –Difficult to discriminate stimuli –Model concept analyses Overview of Assessment

4. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Some terms… Measurement – the gathering of quantitative information Assessment – the pacing of information within a context Evaluation – making a decision based on assessment information

5. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Two Main Types of Assessment: Criterion referenced –Compares the students performance to a specified mastery level –Instructional objectives Norm referenced –Compares the students performance on a task to a group of students –Age equivalent, grade equivalents, percentile ranks

6. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis 3 Levels of Assessment: Micro level Assessment conducted on an on-going basis related to specific instructional objectives Meta level Assessment conducted on a regular basis related to broad skill clusters within curriculum areas Macro level Large-scale assessment to determine overall impact of educational intervention

7. Intervals Corresponding to Levels of Assessment

8. Micro

9. Intervals Corresponding to Levels of Assessment Meta

10. Intervals Corresponding to Levels of Assessment Macro

11. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis The Micro Level of Assessment: Every skill being established or practiced is monitored Monitoring occurs each time a skill is taught Collects data along the same dimension as the instructional objective Is used to inform instructional decisions

12. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Quality Intervention requires: Continuous Measurement Student responding to intervention practices is continuously measured. Visual Displays - A relevant dimension of the behavior to be taught is graphically displayed. - Graphic displays (indicating the progress of the student) are always available for viewing. - Graphs are used to make sound decisions about tactics for individuals. Adapted from work by R.D. Greer

13. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Quality Intervention requires: Responsiveness Intervention driven by the moment to moment responses of each individual student and guided by existing research findings. Adapted from work by R.D. Greer

14. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis The Micro Level of assessment Effects of instructional decisions are tracked relative to each skill Monitoring continues until essential learning outcomes are demonstrated Skill retention Skill endurance Skill application Skill stability

15. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis The Meta Level of Assessment: Clusters of interrelated skills are assessed together Assesses students performance on a range of skills within a curriculum area –E.g., reading, mathematics, social skills, language Helps determine whether intervention is being effective Performance is compared to typical peers Also called Curriculum Based Assessment (CBA)

16. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Examples of CBA Measures: To assess effectiveness of: –Sounds instruction –Words instruction We might measure the number of words read correctly per minute from prose samples To assess effectiveness of: –Expressive labeling of objects instruction –Spoken sentence structure (semantics) instruction We might measure the number of words and sentences spoken spontaneously during a five minute language sample

17. An Example:

18. Another Example:

25. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis The Macro Level of assessment: Skill clusters are assessed together on complex tasks Uses norm-referenced, standardized, published tests –e.g., Woodcock-Johnson, Preschool Language Scale-3, Photo Articulation Test, Child Development Inventory, etc. Allows comparison of client progress relative to the norm group Emphasis is placed on rate of progress pre/post intervention compared to typical peers

26. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis An example of the importance of RATE of progress Emma has made significant progress since her last evaluation in her motor skills. She has gone from an age equivalent of 30 months to 54 months in her gross motor development. - School Psychologist, in a triennial evaluation report Did Emma progress as a result of educational intervention?

27. An example of why RATE of progress is so important... Emma - Gross Motor Development 0 10 20 30 40 50 60 70 80 1 4 7 10131619 222528 313437 40434649 525558 6164677073 Chronological Age Performance Age Equivalent Typical Child's Progress (1 month per month) Emma's Progress - Baseline (.83 months per month) Emma's Progress - DTT + preschool (.75 months per month) Batelle Developmental Inventory

28. Daniels Expressive Language Progress with Intervention Skill Age Equivalent (in Months) Chronological Age (in Months) 0 5 10 15 20 25 30 35 40 45 50 13579 11131517192123252729313335373941 Age in Months Daniel's Progress -.76 months of gain per month Typical Child's Progress - 1 month gain per month Therapy Progress - 3.2 months of gain per month Legend Preschool Language Scale - 3

29. Daniels Auditory Comprehension Progress with Intervention Skill Age Equivalent (in Months) 0 10 20 30 40 50 60 13579 11131517192123252729313335373941 Chronological Age (in Months) Age in Months Daniel's Baseline Progress -.63 months gain per month Typical Child's Progress - 1 month gain per month Daniel's Therapy Progress - 4 months of gain in one month Legend Preschool Language Scale - 3

30. 1.12 - 0.21 Standard Deviations from Peers Pre-Therapy (11/00)Current (05/00) Daniels Articulation Progress with Intervention Photo Articulation Test

31. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis You should begin working on… You will be responsible for completing a report related to assessment of children with autism. Your report is to follow APA style guidelines and should have the following features: 1. Describe at least three published assessment tools used with children with autism. In your description, be sure to include descriptions of the skills and repertoires assessed by the tool, whether the tool is normatively or criterion referenced, a description of the referencing process used by the tool developers, and critical features of children for whom the tool would be appropriate (age, pre-requisite skills required, cultural background, etc.) 2. For one of the assessment tools you describe, list the assessment items from the tool and identify the type of verbal operant tested (if the test items are, in fact, testing verbal behavior). For example, if a test item requires the student to answer the question, "Who lives in the Hundred Acre Wood?" by saying, "Winnie the Pooh", you would classify this as testing the intraverbal repertoire. NOTE: you will make me very happy if you also identify the type of intraverbal response tested (sequelic, codic, or duplic). As another example, if the test item asks the student to say "Winnie the Pooh" when shown a picture of Winnie the Pooh, you would classify this as testing the tact repertoire. If the test item is not testing verbal behavior, then categorize the item according to the taxonomy system you learn in class.

32. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Tonights readings… White, O.R. (1984). Selected issues in program evaluation: Arguments for the individual. In B.K. Keogh (Ed.) Advances in Special Education: Documenting Program Impact. Greenwich, CT: JAI Press.Selected issues in program evaluation: Arguments for the individual Vargas, E.A. (1988). Verbally-governed and event-governed behavior. The Analysis of Verbal Behavior, 6, 11-22.Verbally-governed and event-governed behavior Tiemann, P.W. & Markle, S.M. (1990). Principles (chapter 6). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company Tiemann, P.W. & Markle, S.M. (1990). Strategies (chapter 7). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company Tiemann, P.W. & Markle, S.M. (1990). Sets of related principles--some issue to consider (appendix 10). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company Tiemann, P.W. & Markle, S.M. (1990). Guidelines for the analysis of a set of related principles (appendix 11). Analyzing Instructional Content. Champaign, IL: Stipes Publishing Company

33. Department of Behavior Analysis BEHV 5250: Advanced Topics in Intervention Summer 2003Department of Behavior Analysis Tonights homework… complete the in-text questions for Tiemann & Markle, chapters 6 and 7 (due 7/25/03) identify four examples of instances of rule-governed behavior (due 7/28/03) that children with autism often need to learn from each of the following areas: academic skills social skills self-help skills leisure skills of the examples you identified above, analyze one from each of the four areas using the example formats Tiemann & Makle provide in appendix 11 ;please be prepared to present at least one of these in class tomorrow