1. Martha Thurlow and Laurene Christensen National Center on Educational Outcomes CEC Preconvention Workshop #4 April 21, 2010

2.  Martha Thurlow  Laurene Christensen  Introduce yourself –  Name  State  One more thing about you 2

3.  Review five-step process for monitoring accommodations decision making and implementation  Identify strategies for improving the use of accommodations in instruction and assessment  Share tools and techniques for checking on decision making and use  Discuss how to address validity issues related to accommodations 3

4. Accommodations – Changes in materials or procedures that enable students to meaningfully access instruction and assessment. Assessment accommodations do not change the construct that is being measured. Accommodations mediate the effects of a student’s disability and do not reduce learning expectations. 4

5. Modifications – Changes in materials or procedures that enable students to access instruction and assessment. Assessment modifications do change the construct that is being measured. Modifications create challenges for assessment validityAccommodations mediate the effects of a student’s disability and do not reduce learning expectations. 5

6. 6 Another way to talk about this ------- Accommodations remove construct irrelevant variance – that is, variability that may be systematic (e.g., the effect of the disability), but that is not what the test is intended to measure. Modifications may also remove construct relevant variance – that is, variability that may be systematic (e.g., the effect of the disability) – but in this case, it is also related to what the test is intended to measure.

7. 7 Accommodations are intended to produce valid measures of what a student knows and is able to do. To determine whether each is valid requires relating the accommodation to the standards and content that is being assessed.

8. Standard accommodations ? Nonstandard accommodations ? Conditional accommodations ? Adaptations ? Other terms ? Share your terminology – and what might be confusing about the terms. 8

9.  Binder with agenda, powerpoints, resources, and exercises  Accommodations Monitoring Tool, with examples, checklists, additional resources, and glossary  Accommodations Manual, with tools  Hints and Tips  Accommodations Video  Flash Drive with materials 9

10.  Analyses of accommodation policies (1992- 2009) and effects (1999-2008)  Accommodations for computer-based tests (2002)  Accommodations manuals projects (2005, 2009)  Principled Approach to Accountability Assessments  Peer review project ◦ Hints and Tips Tool ◦ Technical Report on Peer Review of Accommodations 10

11. 1.Selection of accommodations 2.Linking of instructional and assessment accommodations 3.Monitoring accommodations 4.Accommodations use allows for valid and meaningful scores Big Issue that Emerged: Monitoring accommodations 11

12.  Developed in collaboration with CCSSO  Reviewed extensively by ASES SCASS group  Based on a review of publicly available materials  5 Step process  Each step includes examples, a checklist, additional resources  Appendices include additional examples from states  Extensive glossary at the end 12

13. Federal Laws that Pertain to Monitoring Accommodations ◦ IDEA ◦ ESEA What is monitoring? 13

14.  IDEA (Individuals with Disabilities Education Act) ◦ Requires students with disabilities who need accommodations to have them ◦ Document accommodations on an IEP ◦ Monitor the educational results for students ◦ Ensure States follow through with the requirements of IDEA Nothing specific about monitoring accommodations. 14

15.  ESEA (Elementary and Secondary Education Act) ◦ Requires accommodations as necessary in order to measure student achievement ◦ Requires accommodations use to result in a valid score Monitoring is identified in the standards and assessment peer review materials. 15

16. So, what is meant by: “monitoring accommodations”? 16

17. Monitoring accommodations is a process of evaluating policies, procedures, and practices in order to ensure that a state’s assessment system is inclusive of students with disabilities and English language learners. 17

18.  The State has analyzed the use of specific accommodations for different groups of students with disabilities and has provided training to support sound decisions by IEP teams.  The State routinely monitors the extent to which test accommodations are consistent with those provided during instruction. But, it is about more than compliance... 18

19.  Systematic attention to the provision of accommodations can ensure that students are able to show what they know and can do 19

20.  Training for decision-making teams  Evaluating IEP team decisions about accommodations  Tracking accommodations students use for instruction and assessment  Tracking whether the use of accommodations results in a valid score for the student  Reporting the use of accommodations 20

21. 21

22. Break! 22

23.  Keeping track of training  Keeping track of IEP decisions on accommodations  Are accommodations used for instruction consistent with those used for assessments?  Keeping track of requests for unique accommodations 23

24. Questions to ask:  What kinds of training on accommodations are offered by the state or district?  Who attends the trainings?  How are materials made available to participants and others who might need them?  Are materials available online? 24

25. IEP Team Making Decisions about Accommodations 25

26.  Your ideas?  Summarize on Flip Chart 26

27.  Are there model IEP forms for schools to use?  Does the IEP form reference current state policy in some way so that teams are aware of the consequences of their decisions? 27

28.  Do IEP forms indicate instructional and assessment accommodations separately?  Does the electronic data system (if one exists) allow for matching instructional and assessment accommodations? 28

29.  Documentation of accommodations on IEPs?  Connection between instructional and assessment accommodations?  Ease of access to accommodations information? 29

30. 30

31. 1. Do decision-making teams know how to make special requests for accommodations?  Do they know who to contact? Do they know what information should be provided to help the state make a determination? 2. Is there a clear process in place so that teams know how and when decisions on special requests will be made? 3. How are special requests tracked? 31

32. 32

33. 33

34.  What we have done so far: Background Step 1 – Know rules and regulations Step 2 – Document decisions about accommodations  What is still to come:  Step 3 – Document use of accommodations  Step 4 – Review accommodations decisions and use  Step 5 – Evaluate and report on accommodations 34

35.  Documenting accommodations on student demographic sheets  Documenting inappropriate use of accommodations on test day  Documenting that students received their IEP accommodations on test day 35

36.  What information is asked for?  How are accommodations listed?  Are invalidating accommodations (modifications) noted on the bubble sheet?  What about “Other” accommodations? 36

37. Information about the Student 37

38. Information on accommodations 38

39. Accommodations – Provided or Used? 39

40. Bubble Sheets: 40  Strengths  Needed Improvements

41.  Testing irregularity forms  Testing certification forms  How are these forms used?  How are they reviewed? What is done with the information? 41

42. 42

43. 43

44.  Direct observation on test day  Record reviews  On-site visits on a day other than test day  Interviews with students, teachers, and administrators about the use of accommodations 44

45.  Is there an established process?  How are schools chosen?  Are visits announced or unannounced? ◦ Are there requirements to prepare in advance? Example: Arkansas 45

46.  How will records be reviewed?  How will records be selected?  How will information on accommodations be tracked? 46

47. 1. Are the accommodations/modifications appropriate for the skill area affected by the disability (no oral testing for math disability)? Yes No If no, example: 2. Are the accommodations identified in the IEP for state and district- wide assessment provided in their instructional program? (Do they match?) Yes No If no, example: 3. Were the accommodations identified in the IEP for state and district- wide assessment “USED” during assessment administration? (compare the coding on the assessment data sheet with the assessment accommodations listed in the IEP) Yes No If no, example: 4. If the student is identified as taking an alternative assessment, does he or she meet the criterion and has it been documented on the IEP? Yes No If no, example: Monitoring Questions from South Dakota 47

48.  Can be for compliance, or for technical assistance and professional development  May serve as a needs assessment  May or may not include interviews with stakeholders, such as administrators, teachers, and students 48

49. Utah Example 49

50. South Carolina Example 50

51. 1. What accommodations do you use for instruction? How do they help you learn? 2. What accommodations do you use for assessments? How do they help you do your best? 3. Do you have any questions about the accommodations you use for instruction or assessments? 4. How do your teachers make sure you have the accommodations you need on test day? 5. Is there anything else about the accommodations you use that you want to share? 51

52. 1. How do you ensure that students receive the necessary accommodations as indicated on the IEP? 2. How do IEP teams make decisions about what assessment a student will be given (i.e., what data are used to make the determination)? 3. How do IEP teams make decisions about what assessment accommodations a student needs? How do you ensure that accommodations happen? 4. How are families/parents involved in assessment and accommodations selection? 5. How is the provision of accommodations monitored by the school? 52

53. 1. How do you ensure that students receive the necessary accommodations as indicated on the IEP? 2. How do IEP teams make decisions about what assessment a student will be given (i.e., what data are used to make the determination)? 3. How do IEP teams make decisions about what assessment accommodations a student needs? How do you ensure that accommodations happen? 4. How are families/parents involved in assessment and accommodations selection? 5. How is the provision of accommodations monitored by the school? 6. As an administrator, how do you ensure that decision- making teams receive appropriate training on accommodations? 53

54. 54 Arkansas Case Study: Discussion

55. 55

56. 56 Break!

57.  Analyze accommodations  Report on accommodations  Revise accommodations policies 57

58. Alternatives to a formal research study: 1.Document process of collecting information on accommodations 2.Look at existing research on accommodations 3.Conduct extant data analyses 4.Triangulation: formal lit review, expert judgment, and empirical evidence 58

59. 59

60. Hawai’i Example Discrepancy Analysis 60

61. Accommodations Report Delaware Example 61

62. Resource: NCEO Dataviewer http://data.nceo.info 62

63. 63

64. 64

65. Next Steps! 65

66. 66 Contact Information Martha Thurlow: THURL001@umn.eduTHURL001@umn.edu Laurene Christensen: chri1010@umn.educhri1010@umn.edu National Center on Educational Outcomes (NCEO): www.nceo.info