1. Warlick, Quenemoen, Rigney 2004 Options for including students with learning disabilities in assessments for school accountability Ken Warlick, University of Kentucky Rachel Quenemoen, University of Minnesota Sue Rigney, U.S. Department of Education

2. Warlick, Quenemoen, Rigney 2004 Objective Increase awareness of implications of federal assessment and accountability requirements on students with learning disabilities Increase awareness regarding accommodations Increase awareness regarding alternate assessments Increase awareness on emerging trends in assessment

3. Warlick, Quenemoen, Rigney 2004 Purpose of No Child Left Behind “…to ensure that all children have a fair, equal, and significant opportunity to obtain a high- quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments”

4. Warlick, Quenemoen, Rigney 2004 Purpose of Assessment Requirements of IDEA Improve results for student with disabilities through improved teaching and learning Raise expectations for students with disabilities Increase access to the general curriculum Provide parents information about their child’s achievement in relationship to the performance of other children in their school

5. Warlick, Quenemoen, Rigney 2004 Issues in both NCLB and IDEA Students with disabilities previously exempted from assessment and accountability system Students with disabilities previously received instruction in separate curriculum Change from low to high expectations for students with disabilities State leadership in fostering school and district accountability

6. Warlick, Quenemoen, Rigney 2004 Assessment Options General assessment General assessment with accommodations (or modifications) Alternate assessment on grade level achievement standards Alternate assessment on alternate achievement standards Computer-based assessments

7. Warlick, Quenemoen, Rigney 2004 Accommodations: The National Picture Accommodations use is on the rise. About 50% of students with learning disabilities receive an accommodation during testing. Evidence from experimental studies indicates that some accommodations boost performance. Most common accommodations are –Small group administration –Read-aloud –Extended time

8. Warlick, Quenemoen, Rigney 2004 Definitional Issues Some accommodations are considered to change the construct assessed, and others are viewed as “ok” OkayNot Okay Accommodation Adaptation Standard administration Modification Non-approved accommodation Non-allowed accommodation Non-standard administration Know what terms mean in your state and district!

9. Warlick, Quenemoen, Rigney 2004 What Makes An Assessment Accommodation OK? It is aligned with instructional accommodations, but is not an excuse not to teach Student needs it to demonstrate knowledge and skills – or to participate in assessment The consequences of using the accommodation have been carefully considered OK is not defined by what the test publisher says (unless has research evidence)

10. Warlick, Quenemoen, Rigney 2004 Closing the Achievement Gap Looks Like This:

11. Warlick, Quenemoen, Rigney 2004 Can states close achievement gap without paying attention to LD? Keep high incidence populations in school and in special education (LD 3 X as likely to drop out of school as non-disabled) LD comprises 50% of special education population LD is a life-long disability. Student’s needs change over time. They need learning and coping strategies but special education does not cure them

12. Warlick, Quenemoen, Rigney 2004 Research on Accommodations for LD Students – What Does It Tell Us? Accommodations most frequently studied are oral presentation (13) and extended time (10) Various research designs are used Many limitations in conducted research are identified

13. Warlick, Quenemoen, Rigney 2004 Issues: Tendency to allow too many accommodations, possibly reducing expectations for student learning. Poor decision making about accommodations, reflecting lack of knowledge about instructional accommodations. Use of accommodations as an excuse to exclude students scores on reports or accountability.

14. Warlick, Quenemoen, Rigney 2004 Promising Practices: States keeping track of how many students use accommodations, which ones, and their impact on achievement. Out-of-the-box thinking about the use of accommodated test scores. Clear decision-making criteria (e.g., alignment to instructional accommodations) and training on how to make decisions. Teaching high school students (at the latest) about the tests they take and about the accommodations that they need.

15. Warlick, Quenemoen, Rigney 2004 Implications for IEP teams Must be familiar with the state or district content/learning standards Must be familiar with the curriculum Must be familiar with typical instructional activities in general education

16. Warlick, Quenemoen, Rigney 2004 Adequate Yearly Progress

17. Warlick, Quenemoen, Rigney 2004 Alternate Assessments  Aligned with the State’s content standards.  Yield results separately in reading/language arts and math.  Designed and implemented to support use of the results to determine AYP.

18. Warlick, Quenemoen, Rigney 2004 Alternate Assessments should have…  Clearly defined structure  Guidelines for which students may participate  Clearly defined scoring criteria and procedures  Report format that clearly communicates student performance in terms of the academic achievement standards defined by the State

19. Warlick, Quenemoen, Rigney 2004 Alternate Assessments Must meet the same requirements for high technical quality that apply to regular assessments under NCLB:  Validity  Reliability  Accessibility  Objectivity  Consistent with nationally-recognized professional and technical standards.

20. Warlick, Quenemoen, Rigney 2004 States may use more than one alternate assessment  Alternate assessment scored against grade-level standards  Alternate assessment scored against alternate achievement standards

21. Warlick, Quenemoen, Rigney 2004 Out-of-Level Assessments Could only be considered an alternate assessment based on alternate achievement standards if  Alternate achievement standards defined through a documented and validated standards-setting process  Proficient results included in the 1% cap

22. Warlick, Quenemoen, Rigney 2004 Researching Out-of-Level Testing 10 Recommendations: See Myth- Busters Handout

23. Warlick, Quenemoen, Rigney 2004 Universally Designed Assesmsents: A Quick Definition Universally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students!

24. Warlick, Quenemoen, Rigney 2004 But, what does that really mean? Do we want to change the standard of performance?NO Can we forget about accommodations if we do this?NO Is this all figured out – for now and forever?NO Is this something that will benefit only students with disabilities?NO

25. Warlick, Quenemoen, Rigney 2004 Universally designed assessments reflect good measurement qualities:  Minimize skills required beyond those being measured  Reflects the intended content standard (reviewers have information about what is being measured) Measures What it Intends to Measure

26. Warlick, Quenemoen, Rigney 2004 An Example of Universal Design: Mathematics Tests The reading requirements of a math test often prevent students with marginal reading ability from demonstrating competency in math.

27. Warlick, Quenemoen, Rigney 2004 Ordering Pizza (Original Item) The cafeteria manager surveyed the students in a middle school to find out if they would buy Brand X pizza on Friday if the manager sold it. She made a circle graph to display the results of her survey. NO YES Based on the results of the survey, answer the following questions: 1.What fraction of students would buy Brand X pizza on Friday? 2.What percent of students would buy Brand X pizza on Friday? 3.There are 1200 students in this school. How many students will buy Brand X pizza on Friday if the manager’s survey is accurate?

28. Warlick, Quenemoen, Rigney 2004 Ordering Pizza (Revised Item) Maria surveyed the students in her school to find out if they liked pizza on Friday. She made a circle graph to display the results of her survey. NO YES 1.What fraction of students said “yes”? 2.What percent of students said “yes”? 3.There are 1200 students in Maria’s school. How many students said “yes”?

29. Warlick, Quenemoen, Rigney 2004 Suppose a test item requires a student to read an Aesop’s fable What could this item measure? –Decode text? –Comprehend extended passages? –Extent of vocabulary? –Understand the moral or point of the fable? –Discuss the common elements of any fable? –Compare and contrast fables with news reports? –Articulate the relationship between the fable and the overall culture? –Anything else? (National Center on Accessing the Curriculum, 2003)

30. Warlick, Quenemoen, Rigney 2004 Something to think about

31. Warlick, Quenemoen, Rigney 2004 The “door” we need to go through: What is meant by the construct of ‘reading’ What about ‘literacy?’ The answer(s) to these questions will determine the universal design or accommodations that can be used when certain content standards are taught and assessed!

32. Warlick, Quenemoen, Rigney 2004 Modes of print interaction: with examples of accommodations Visual Tactile (feeling print) Auditory (listening to printed messages) Multi-modal (using any combination of the above modalities) Printed text; ASL Text in Braille and Nemeth Codes Listen to taped text Computer-based “assistive” reading/ viewing programs

33. Warlick, Quenemoen, Rigney 2004 Individualized accommodation decisions should be linked to the standard, the construct assessed, the nature of instruction or the assessment, and the student’s characteristics Purposeful reading – reading to select and apply relevant information for a given task Does this allow different modes of print interaction? And, what are the implications of these different modes for accommodations?

34. Warlick, Quenemoen, Rigney 2004 Issues: Universal design does not eliminate the need for accommodations. Universal design requires the systematic application of good test development – more attention needs to be given at the same time that we need to develop more assessments. Little research exists on universally designed assessments.

35. Warlick, Quenemoen, Rigney 2004 Promising Practices: Item developers who know disability issues and their interaction with test formats and procedures. “Bias” or “sensitivity” review committee with at least one member who knows disability issues. Field testing that includes students with disabilities who use accommodations during the field test.

36. Warlick, Quenemoen, Rigney 2004 What is computer-based assessment Does NOT mean adaptive testing DOES mean making use of technology for accommodations DOES mean grappling with very, very important issues

37. Warlick, Quenemoen, Rigney 2004 Opportunities Efficient administration Preferred by students Improved writing performance Built-in accommodations Immediate results Efficient item development Increased authenticity

38. Warlick, Quenemoen, Rigney 2004 Challenges Use of technology cannot take the place of content mastery Issues of equity and skill in computer use Added challenges for some students Technological challenges Security of online data Lack of expertise in designing accessible Web pages Prohibitive development cost

39. Warlick, Quenemoen, Rigney 2004 Example: Kentucky Online Assessment Web-based, individualized assessment : –Students with IEP or 504 Plan that specifies need for "reader" as an instructional and assessment accommodation; –Students who require and routinely use text-reader or screen-reader technologies to access printed material in classroom instruction and assessment; –Students who have accessed and used the CATS Online Practice Area. Based upon success of pilot studies, 16 districts, 31 Schools, & 204 students participated in “live” CATS Online in the spring of 2003

40. Warlick, Quenemoen, Rigney 2004 CATS Online

41. Warlick, Quenemoen, Rigney 2004 Visit: education.umn.edu/nceo or Search for NCEO Check us out!