1. UDL Institute CAST Tracey E. Hall 2012

2. The gathering of information about a learner from his or her performance in a variety of tasks, subjects, and learning contexts to determine abilities and knowledge for the purpose of making educational decisions (Salvia & Ysseldyke, 2009 ).

3.  Summative Evaluation: Measurement to determine subject performance at the end of a specific grade level, unit, year, or instructional episode. Mastery Measurement  Formative Evaluation: Assessment of progress toward a long-term goal or major objective, used primarily to diagnose what students have learned in order to plan further instruction (i.e., the on-going assessment of progress toward an objective).

4.  Summative Evaluation: Measurement to determine subject performance at the end of a specific grade level, unit, year, or instructional episode. Mastery Measurement  Formative Evaluation: Assessment of progress toward a long-term goal or major objective, used primarily to diagnose what students have learned in order to plan further instruction (i.e., the on-going assessment of progress toward an objective).

5. Educational tests are:  Indirect measure of a construct or a set of closely related constructs.  Constructs are not observable  A test provides a sample of observable behaviors believed to be the product of the intended construct  Observable behaviors are used to make an inference about the construct

6. 2 perspectives 1.Access to the test 2.Access to the construct

8.  Test construct refers to the concept or the characteristic that a test is designed to measure. e.g., In a mathematics assessment, an item designed to evaluate students' ability to show equivalence has a test construct of numerical equivalency. American Educational Research Association, American Psychological Association, & Natio nal Council on Measurement in Education. (1999).

9. Tests are designed to:  Present information to stimulate the construct.  Interact with the information as the construct is operating  Produce a response which becomes a physical product

10. Tests are designed to:  Present information to stimulate the construct.  Interact with the information as the construct is operating  Produce a response which becomes a physical product

11. How Items Function Interact with Apply View Product Measure of Stimuli Construct of Construct Construct If a challenge or barrier exists at any of these levels, inference about the construct will fail. Present Information Produce Response Interact with Construct Inference to Construct A Quantita- tive Score Michael Russell 2011

13.  When there are construct irrelevant impediments, distractions, or barriers in the assessment methods it is essential to provide scaffolds, supports or accommodations in order to improve accuracy and validity, especially for individuals with disabilities.

14.  Keep in mind, using those same scaffolds, supports or accommodations inappropriately – where they affect the construct relevant demands of the assessment – is likely to invalidate the measure.

15.  Construct relevant refers to the factors (e.g., mode of presentation or response) that are relevant (related) to the construct that the test is intended to measure.  Identifying the symbols for a math calculation problem is a construct relevant change to the assessment item.

16.  Construct irrelevance is the extent to which test scores are influenced by factors (e.g., mode of presentation or response) that are irrelevant (not related) to the construct that the test is intended to measure.  For Elisa, a student with severe cognitive disabilities, changing the font size and increasing sizing of images helped her to see the assessment item for counting objects and doing so was construct irrelevant to assessing ability to count.

17.  Accommodations, scaffolds, and supports should not be exclusive to the instructional environment but should be a part of the whole instructional episode including assessment.  The preservation of the distinction between what is construct relevant and what is construct irrelevant is essential when making decisions about accommodations, scaffolds, and supports in the assessment environment. 

20. Ability to concentrate on the problem is one of the constituent parts of the successful mental work

21. Many students in the United States struggle in the areas of reading and writing, with difficulties emerging early and continuing into the secondary school years and beyond (National Center for Education Statistics, 2002, 2003, 2005, 2007). As student performance levels diverge, it becomes increasingly necessary to provide greater differentiation of instruction. Two approaches are particularly suited for helping teachers and practitioners to differentiate instruction.

23.  Flexible font size (magnification)  Color contrasts  Format, White space Easy in a digital environment, no item interference. No impact on measuring the construct. Only change is how the item is being displayed. “These are no brainers in a computer environment” Mike Russell 09. On paper, more difficult to make changes such as these.., for administration not student – lots of different papers

24. water agua H2OH2O

27. Example: Options for to read and respond. Students are (a)engaged by choice (b) often select based on interest or prior knowledge (c) perform better

28.  Marginal efficacy Consider: ◦ Extended time ◦ Read-aloud accommodation vs. independent reading and strategic test-taking  May invalidate measurements Consider: ◦ Use of read-aloud for decoding task ◦ Use of calculator for arithmetic task Thus it is often difficult to level the playing field with accommodations, especially if the playing fields was initially un-level for all students.

29.  During Test Development ◦ Ensure narrowly-defined constructs ◦ Conduct bias review early ◦ Ensure items are suitable for appropriate delivery to all students  During Test Delivery Provide multiple means of recognition, expression & engagement Ensure matching of classroom and assessment supports

30. The cartoon above depicts United States frustration with A) the Good Neighbor policyB) Dollar Diplomacy C) the Spanish-American WarD) the Bay of Pigs invasion 12 th Grade NAEP U.S. History Item Broadly-Define Constructs / Bias Review: Assumptions of Prior Knowledge

31. Circle the picture that starts with “B” Circle the picture that starts with “T” 1 st Grade Phonological Awareness Tasks

32. “Sarah must determine the diameter of a circle that she has drawn, but doesn’t know the value of П. Can she still do it?” 8 th Grade Geometry: Word Problem vs. Symbol Recognition Consider effect of read-aloud accommodation on the following test question:

33. MCA-II Grade 10 Reading Sample Item

35. UDL Considerations for Assessment Development Structure:  Based on the UDL Principles  Research on working memory  Research on eye movement *handout

36.  Linguistic Complexity  Information Density  Self-Regulation and Visio-Spatial information

37. Linguistic Complexity (LC) The relationship between the sentence processing mechanism and the available computational resources (Gibson, 1998)  LC-1 Syntax –  LC-2 Simplifying vocabulary  LC-3 Reduced sentence length  LC-4 Language translation  LC-5 Clarify anaphoric references

38. Information Density (ID) The two critical limitations on handling information in working memory are the (1) small number of pieces of information an individual’s memory can handle and (2) the short duration of time during which information can remain in memory (Sylwester & Choo, 1992). The following modification structures help to reduce working memory load and maximize use of information read when used carefully.  ID-1 Question placement  ID-2 Emphasize key information  ID-3 Chunking or combining tasks (questions or response options)  ID-4 Guide information processing—  ID-5 Contextualizing skills –skill icons  ID-6 Hint  ID-7 Line numbering  ID-8 Passage primer

39.  Self-regulation is seen by many cognitive researchers as a pivot upon which students’ achievement turns. The structures noted here when used in a non-construct relevant application help to support self-regulation and attention to tasks.  SR-1 Progress map   SR-2 Self check   SR-3 Optional Workspace   SR-4 Skill Icon Preview  SR-5 Reduce Reflexive Eye Movement

40. 40

41. 41

42. 42

43.  Discussion –UDL considerations  Small groups ◦ Start with the task directions ◦ Think UDL access to executive functioning