1. Alternate Assessments on Alternate Achievement Standards Student Population
Jacqueline F. Kearns, Ed.D.
Elizabeth Towles-Reeves, MS

2. The Assessment Triangle & Validity Evaluation
Marion & Pellegrino (2006)
OBSERVATION
INTERPRETATION
COGNITION
Assessment System
Test Development
Administration
Scoring
Reporting
Alignment
Item Analysis & DIF/Bias
Measurement error
Scaling and Equating
Standard Setting
VALIDITY EVALUATION
Empirical evidence
Theory & logic (argument)
Consequential features
Student Population
Academic content
Theory of Learning

3. Cognition Vertex Validity Questions
Is the assessment appropriate for the students for whom it was intended?
Is the assessment being administered to the appropriate students?
Both are important for the validity evaluation

4. More Different Than Alike
2% Mulitple disabilities (Student example: Leslie)
SOURCE: Education Week analysis of data from the U.S. Department of Education, Office of Special Education Programs, Data Analysis System,

5. Issues in Teaching/Assessing Students in Alternate Assessments
Varied levels of symbolic communication
Attention to salient features of stimuli
Memory
Limited motor response repertoire
Generalization
Self-Regulation
Meta-cognition
Skill Synthesis
Sensory Deficits
Special Health Care Needs
Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students with significant cognitive disabilities: Models of student cognition. National Alternate Assessment Center, Human Development Institute, University of Kentucky, Lexington. (PDF File)

6. Previous Data 165 Students across 7 states
Extensive documentation through 111 item inventory
Findings suggest:
64% routinely use verbal language
46% routinely understand pictures used to represent objects
11% don’t understand pictures used to represent objects.
Almond & Bechard (2005) An In Depth Look at students who take alternate assessments: What do we know. Colorado EAG.

7. Learner Characteristics Demographic Variables
Learner Characteristics (all on a continuum of skills):
Expressive Language
Receptive Language
Vision
Hearing
Motor
Engagement
Health Issues/Attendance
Reading
Mathematics
Use of an Augmentative Communication System (dichotomous variable)

8. Methodology Four partner states chose to participate
States 1, 2, and 3:
gathered data in the administration process for their AA-AAS via scannable document (i.e., bubble-sheet)
State 4:
gathered data using Zoomerang, an online survey package.
N= 7,075

9. States & LCI Response Rates
Geography
Demographic
Sample N
Response
Rate 1
Eastern
Rural
Suburban
3595
75% 2
North Eastern
Urban-
2793
100% 3
Urban
468
91% 4
Western
219
47%

10. Alternate Assessment Participation Rates : % Total population
State 1
.959%
State 2
1.14%
State 3
.766%
State 4
.55%

12. “Most significant cognitive disabilities”
Those kids that are PreSymbolic and also have low vision, low motor, and low engagement characteristics.
Same for Emerging and Symbolic!
“Most significant cognitive disabilities”

13. Expressive Language
To communicate expressively, most students in each state used verbal or written words, signs, Braille, or language-based augmentative systems to request, initiate, and respond to questions, describe things or events, and express refusal (79%, 70%, 78%, and 75% respectively in States 1, 2, 3 and 4). A smaller group of the population in each state used understandable communication through such modes as gestures, pictures, objects/textures, points, etc., to clearly express a variety of intentions (13%, 18%, 13%, 17% respectively). An even smaller group of students primarily used cries, facial expressions, change in muscle tone, etc., to communicate, but these students had no clear use of objects/textures, regularized gestures, pictures, signs, etc., to communicate (8%, 11%, 9%, and 8% respectively).

14. Receptive Language
Receptively, students in each state fell into two primary groups: those students who independently followed 1-2 step directions presented through words (e.g. words could be spoken, signed, printed, or any combination) while not requiring additional cues (53%, 45%, 55%, and 56% respectively in state 1, 2, 3, and 4); or those students who required additional cues (e.g., gestures, pictures, objects, or demonstrations/models) to follow 1-2 step directions (37%, 43%, 36%, and 34%). A smaller group (8%, 10%, 8%, and 7%) alerted to sensory input from another person (auditory, visual, touch, movement) but required actual physical assistance to follow simple directions. Finally, less than three percent of the population in each state displayed an uncertain response to sensory stimuli (e.g., sound/voice; sight/gesture; touch; movement; smell).

16. Use of Augmented Communication
In State 1, results suggest that 47.2% of the students identified as communicating “primarily through cries, facial expressions, change in muscle tone, etc., but with no clear use of objects/textures, regularized gestures, pictures, signs, etc.” do not use a communication system. In addition, 54.6% of the students identified as “using intentional communication but not at the symbolic level” do not use a communication system.
In State 2, results suggest that 51.2% of the students identified as communicating “primarily through cries, facial expressions, change in muscle tone, etc., but with no clear use of objects/textures, regularized gestures, pictures, signs, etc.” do not use a communication system. In addition, 49.7% of the students identified as “using intentional communication but not at the symbolic level” do not use a communication system.
In State 3, results suggest that 23.3% of the students identified as communicating “primarily through cries, facial expressions, change in muscle tone, etc., but with no clear use of objects/textures, regularized gestures, pictures, signs, etc.” do not use a communication system. In addition, 53.2% of the students identified as “using intentional communication but not at the symbolic level” do not use a communication system.
In State 4, results suggest that 77% of the students identified as communicating “primarily through cries, facial expressions, change in muscle tone, etc., but with no clear use of objects/textures, regularized gestures, pictures, signs, etc.” do not use a communication system. In addition, 57% of the students identified as “using intentional communication but not at the symbolic level” do not use a communication system.

17. Reading
In States 1, 2, and State 3, teachers noted that 2-4% of the population read fluently with critical understanding in print or Braille. State 2 did not provide this option on the inventory. 15% of the students in State 1, 17% in State 2, and 15% in State 3, and 34% in State 4 were rated as being able to read fluently with basic (literal) understanding from paragraphs/short passages with narrative/ informational texts in print or Braille. The majority of students (50%, 46%, 39%, and 33%) were rated as being able to read basic sight words, simple sentences, directions, bullets, and/or lists in print or Braille, but not fluently from text with understanding. A smaller percentage of students (21%, 18%, 25%, and 18%) were rated as not yet having a sight word vocabulary, but being aware of text/Braille, following directionality, making letter distinctions, or telling a story from pictures. Finally, teachers noted 13% of students in State 1, 20% of students in State 2, and 17% of students in State 3, and 13% of students in State 4 had no observable awareness of print or Braille.

18. Mathematics
Across the states, 3-6% of students applied computational procedures to solve real-life or routine word problems from a variety of contexts. The largest category of students within each state (57%, 44%, 45%, and 51% respectively) was able to complete computational procedures with or without a calculator. Nearly 21% of students in State 1, 25% of students in State 2, and 23% of students in State 3, and 27% of students in State 4 were described as performing at the more basic level of counting with one-to-one correspondence to at least 10, and/or make numbered sets of items. A smaller percentage still (6%, 10%, 13%, and 6%) were described as being able to count by rote to 5, but without the higher skill sequences of one-to-one correspondence or computation. Finally, teachers noted that nearly 12% of students in State 1, 18% of students in State 2, and 14% of students in State 3, and 12% of students in State 4 had no observable awareness or use of numbers.

19. Skill level 1: Students with 3 for Expressive, 4 for Receptive, 3-5 for Reading and 3-5 for Math

22. Who are the Kids?
Represent ~1% or less of the total assessed population
All disability categories were represented but primarily 3 emerge,
Mental Retardation
Multiple Disabilities
Autism
Highly varied levels of expressive/receptive language use
Most students in the population use symbolic communication
Level of symbolic language distribution is similar across grade-bands
Only about 50% of the pre and emerging symbolic language users use ACS
Pre-symbolic expressive language users are more likely to have additional complex characteristics.
Most of the population read basic sight words and solve simple math problems with a calculator.
Lack of skill progression in reading across grade bands (elementary, middle & high)
Skill progression apparent in mathematics across grade bands but still small

23. Limitations Only four state participants Small sample size
Global items in reading and math
Participation rates at 1% or less

24. Cognition Vertex: Validity Evaluation Essential Questions
Who is the population being assessed?
How do we document and monitor the population?
What do we know about how they learn (theory of learning) academic content?
What do our assessment results tell us about how the population is learning academic content?
Are our data about the population and theory of learning consistent with student performances on the assessment?
If not, what assumptions are challenged?
What adjustments should be made?
Participation
Theory of Learning
Student Performance

25. References
Agran, M., Fodor-Davis, Moore, & Martella, (1992). Effects of peer-delivered self-instructional training on a lunch-making task for students with severe disabilities. Education and Training in Mental Retardation, 27,
Billingsley, F., Gallucci, C., Peck, C., Schwartz, I., & Staub, D. (1996).  "But those kids can't even do math:  An alternative conceptualization of outcomes in special education.  Special Education Leadership Review, 3 (1),
Brown, L., Nisbet, J., Ford, A., Sweet, M., Shiraga, B., York, J., Loomis, R. (1983). The critical need for non-school instruction in educational programs for severely handicapped students. Journal of the Association of the Severely Handicapped. 8,
CAST (2002).
Fox, (1989). Stimulus Generalization of skills and persons with profound mental handicaps. Education and Training in Mental Retardation, 24,
Haring, N. (1988). Generalization for students with severe handicaps: Strategies and solutions. Seattle, WA: University of Washington Press.
Hughes, C. & Agran, M. (1993). Teaching persons with severe disabilities to use self-instruction in community settings: An analysis of the applications. Journal of the Association for Persons with severe Handicaps, 18,
Hughes, C., Hugo, K., & Blatt, J. (1996). Self-instructional intervention for teaching generalized problem-solving with a functional task sequence. American Journal of Mental Retardation,
Westling, D., & Fox, L. (2004). Teaching Students with Severe Disabilities. Columbus: Pearson (Merrell).
Whitman, T. L. (1990). Self-regulation and mental retardation. American Journal on Mental Retardation, 94,

26. Contact Information
Jacqueline Kearns, Ed.D.
Elizabeth Towles-Reeves, MS
1 Quality Street, Suite 722
Lexington, Kentucky 40507
X 80243
1 Quality Street, Suite 722
Lexington, Kentucky 40507
X 80255