1. Alternate Assessments on Alternate Achievement Standards Student Population Jacqueline F. Kearns, Ed.D. Elizabeth Towles-Reeves, MS

2. OBSERVATION INTERPRETATION COGNITION  Student Population  Academic content  Theory of Learning  Assessment System  Test Development  Administration  Scoring  Reporting  Alignment  Item Analysis & DIF/Bias  Measurement error  Scaling and Equating  Standard Setting VALIDITY EVALUATION  Empirical evidence  Theory & logic (argument)  Consequential features The Assessment Triangle & Validity Evaluation Marion & Pellegrino (2006)

3. Cognition Vertex Validity Questions 1) Is the assessment appropriate for the students for whom it was intended? 2) Is the assessment being administered to the appropriate students? Both are important for the validity evaluation

4. More Different Than Alike SOURCE: Education Week analysis of data from the U.S. Department of Education, Office of Special Education Programs, Data Analysis System, 2002-03

5. Issues in Teaching/Assessing Students in Alternate Assessments Varied levels of symbolic communication Attention to salient features of stimuli Memory Limited motor response repertoire Generalization Self-Regulation Meta-cognition Skill Synthesis Sensory Deficits Special Health Care Needs Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students with significant cognitive disabilities: Models of student cognition. National Alternate Assessment Center, Human Development Institute, University of Kentucky, Lexington. (PDF File) Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students with significant cognitive disabilities: Models of student cognition. National Alternate Assessment Center, Human Development Institute, University of Kentucky, Lexington.

6. Previous Data 165 Students across 7 states 165 Students across 7 states Extensive documentation through 111 item inventory Extensive documentation through 111 item inventory Findings suggest: Findings suggest: 64% routinely use verbal language 64% routinely use verbal language 46% routinely understand pictures used to represent objects 46% routinely understand pictures used to represent objects 11% don’t understand pictures used to represent objects. 11% don’t understand pictures used to represent objects. Almond & Bechard (2005) An In Depth Look at students who take alternate assessments: What do we know. Colorado EAG. Almond & Bechard (2005) An In Depth Look at students who take alternate assessments: What do we know. Colorado EAG.

7. Learner Characteristics Demographic Variables Learner Characteristics (all on a continuum of skills): Expressive Language Receptive Language Vision Hearing Motor Engagement Health Issues/Attendance Reading Mathematics Use of an Augmentative Communication System (dichotomous variable)

8. Methodology Four partner states chose to participate States 1, 2, and 3: bubble-sheet) gathered data in the administration process for their AA- AAS via scannable document (i.e., bubble-sheet) State 4: gathered data using Zoomerang, an online survey package. N= 7,075

9. States & LCI Response Rates StateGeographyDemographicSampleNResponseRate State1EasternRuralSuburban 3595 359575% State2 North Eastern Urban-Suburban2793100% State3EasternUrban46891% State4WesternRural 219 21947%

10. Alternate Assessment Participation Rates : % Total population State 1. 959% State 2 1.14% State 3.766% State 4.55%

12. “Most significant cognitive disabilities”

13. Expressive Language

14. Receptive Language

16. Use of Augmented Communication

17. Reading

18. Mathematics

22. Who are the Kids? Represent ~1% or less of the total assessed population All disability categories were represented but primarily 3 emerge, Mental Retardation Multiple Disabilities Autism Highly varied levels of expressive/receptive language use Most students in the population use symbolic communication Level of symbolic language distribution is similar across grade-bands Only about 50% of the pre and emerging symbolic language users use ACS Pre-symbolic expressive language users are more likely to have additional complex characteristics. Most of the population read basic sight words and solve simple math problems with a calculator. Lack of skill progression in reading across grade bands (elementary, middle & high) Skill progression apparent in mathematics across grade bands but still small

23. Limitations Only four state participants Small sample size Global items in reading and math Participation rates at 1% or less

24. Cognition Vertex: Validity Evaluation Essential Questions Who is the population being assessed? Who is the population being assessed? How do we document and monitor the population? How do we document and monitor the population? What do we know about how they learn (theory of learning) academic content? What do we know about how they learn (theory of learning) academic content? What do our assessment results tell us about how the population is learning academic content? What do our assessment results tell us about how the population is learning academic content? Are our data about the population and theory of learning consistent with student performances on the assessment? Are our data about the population and theory of learning consistent with student performances on the assessment? If not, what assumptions are challenged? If not, what assumptions are challenged? What adjustments should be made? What adjustments should be made? Participation Participation Theory of Learning Theory of Learning Student Performance Student Performance

25. References Agran, M., Fodor-Davis, Moore, & Martella, (1992). Effects of peer-delivered self-instructional training on a lunch-making task for students with severe disabilities. Education and Training in Mental Retardation, 27, 230-240. Billingsley, F., Gallucci, C., Peck, C., Schwartz, I., & Staub, D. (1996). "But those kids can't even do math: An alternative conceptualization of outcomes in special education. Special Education Leadership Review, 3 (1), 43-55. Brown, L., Nisbet, J., Ford, A., Sweet, M., Shiraga, B., York, J., Loomis, R. (1983). The critical need for non-school instruction in educational programs for severely handicapped students. Journal of the Association of the Severely Handicapped. 8, 71-77. CAST (2002). Fox, (1989). Stimulus Generalization of skills and persons with profound mental handicaps. Education and Training in Mental Retardation, 24,219-299. Haring, N. (1988). Generalization for students with severe handicaps: Strategies and solutions. Seattle, WA: University of Washington Press. Hughes, C. & Agran, M. (1993). Teaching persons with severe disabilities to use self-instruction in community settings: An analysis of the applications. Journal of the Association for Persons with severe Handicaps, 18, 261-274. Hughes, C., Hugo, K., & Blatt, J. (1996). Self-instructional intervention for teaching generalized problem-solving with a functional task sequence. American Journal of Mental Retardation, 100 565-579. Westling, D., & Fox, L. (2004). Teaching Students with Severe Disabilities. Columbus: Pearson (Merrell). Whitman, T. L. (1990). Self-regulation and mental retardation. American Journal on Mental Retardation, 94, 347-362.

26. Contact Information Jacqueline Kearns, Ed.D. 1 Quality Street, Suite 722 Lexington, Kentucky 40507 859-257-7672 X 80243 859-323-1838 Jacqueline.kearns@uky.edu Elizabeth Towles-Reeves, MS 1 Quality Street, Suite 722 Lexington, Kentucky 40507 859-257-7672 X 80255 859-323-1838 Liztowles-reeves@uky.edu www.naacpartners.org