1. Opting-Out of Accountability: Background and Implications
Presentation at the National Conference on Student Assessment
Gregory J. Cizek
University of North Carolina at Chapel Hill
21 June 2016

2. Plan
I. Some Context II. Some Data Collection Concerns III. Non-participation Patterns IV. What Can’t Be Done to Address Non- participation; What Can V. $even Strategies for Reducing Testing Time VI. Conclusions

3. Some Context

4. Reality vs. Perceptions of Testing Time
Some Context
Reality vs. Perceptions of Testing Time

5. Federal/State Mandated Summative Testing Time in hours (grade)
Unit
Elementary
Middle
H.S. NC
9.0 (3)
13.0 (8)
15.0 (10)
Ohio
17.2 (3)
19.3 (8)
21.7 (11)
*CGCS
20.6 (3)
25.3 (8)
22.5 (11) US
16.6 (3)
17.1 (7)
---
* Includes district mandated tests

6. Federal/State Mandated Summative Testing Time
Total Testing Time/Instructional Time * 1.7% (Teoh, et al, 2014) * 1.6% (Guindon, et al, 2014) 180 x 6 = 1080 x = 18.7

7. Reality vs. Perceptions of Testing Time

8. Reality vs. Perceptions of Testing Time
n > 300 K, 3, 7 teacher estimates of testing time Kindergarten: x 3.04 Third Grade: x 1.95 Seventh Grade: x 1.10

9. Reality vs. Perceptions of Testing Time
n > 300 K, 3, 7 teacher estimates of testing time Kindergarten: x 3.04 Third Grade: x 1.95 Seventh Grade: x but likely underestimates

10. Reality vs. Perceptions of Testing Time
Additional Factors:

11. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests

12. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests * non-mandated tests

13. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests * non-mandated tests * test preparation activities (tests, prep)

14. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests * non-mandated tests * test preparation activities (tests, prep) * test administration, scoring, reporting activities

15. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests * non-mandated tests * test preparation activities (tests, prep) * test administration, scoring, reporting activities * test administration “windows”

16. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests * non-mandated tests * test preparation activities (tests, prep) * test administration, scoring, reporting activities * test administration “windows” * “non-interference” actions

17. Reality vs. Perceptions of Testing Time
Additional Factors: * district, other mandated tests * non-mandated tests * test preparation activities (tests, prep) * test administration, scoring, reporting activities * test administration “windows” * “non-interference” actions * facilities scheduling disruptions

18. Five Conclusions

19. Five Conclusions
1) Little state/federal mandated testing

20. Five Conclusions
1) Little state/federal mandated testing 2) Modest total testing

21. Five Conclusions
1) Little state/federal mandated testing 2) Modest total testing 3) Substantial perceived amount of testing

22. Five Conclusions
1) Little state/federal mandated testing 2) Modest total testing 3) Substantial perceived amount of testing 4) Many perceived negative consequences

23. Five Conclusions
1) Little state/federal mandated testing 2) Modest total testing 3) Substantial perceived amount of testing 4) Many perceived negative consequences 5) Few (negligible?) perceived benefit(s)

24. Some Data Collection Concerns

25. Some Data Collection Concerns
1) Actual amount of opting-out * opting-out vs. not tested * “non participation”

26. Some Data Collection Concerns
2) Characteristics of opt-out examinees * demographic vs. psychometric

27. Some Data Collection Concerns
3) Actual reasons for opting-out

28. Patterns of Non-participation

29. Patterns of Non-participation
* Increase across grades

30. Patterns of Non-participation
* Increase across grades
* Greater for Math than ELA

31. Patterns of Non-participation
* Increase across grades
* Greater for Math than ELA
* Lower for larger districts

32. Patterns of Non-participation
* Increase across grades
* Greater for Math than ELA
* Lower for larger districts
* Lower for more disadvantaged districts

33. Patterns of Non-participation
* Increase across grades
* Greater for Math than ELA
* Lower for larger districts
* Lower for more disadvantaged districts
* Greater for districts with lower scores

34. Patterns of Non-participation
* Increase across grades
* Greater for Math than ELA
* Lower for larger districts
* Lower for more disadvantaged districts
* Greater for districts with lower scores
* Greater for accountability tests

35. Patterns of Non-participation
* Increase across grades
* Greater for Math than ELA
* Lower for larger districts
* Lower for more disadvantaged districts
* Greater for districts with lower scores
* Greater for accountability tests
* Association with educator evaluation

36. Questar Puts Pearson Exec. in Charge of Assessment
“Guess what - NYS parents will be sure there are NO TESTS AT ALL.... So they need to just give up because we aren't going to .... OPT OUT FOREVER!!”
C. Solnik (2016, May 6). Long Island Business News.com

37. What Can’t Be Done to Address Non-participation; What Can

38. What Can’t Be Done to Address Non-participation; What Can
1) Repeat information on actual amount of testing

39. What Can’t Be Done to Address Non-participation; What Can
1) Repeat information on actual amount of testing 2) Appeals to civic duty

40. What Can’t Be Done to Address Non-participation; What Can
1) Repeat information on actual amount of testing 2) Appeals to civic duty 3) Statistical adjustments without accurate information on ability distribution

41. What Can’t Be Done to Address Non-participation; What Can

42. What Can’t Be Done to Address Non-participation; What Can
1) Nothing

43. What Can’t Be Done to Address Non-participation; What Can
1) Nothing 2) More complete data collection

44. What Can’t Be Done to Address Non-participation; What Can
1) Nothing 2) More complete data collection 3) Reassure that all scores, categories are still valid

45. What Can’t Be Done to Address Non-participation; What Can
1) Nothing 2) More complete data collection 3) Reassure that all scores, categories are still valid 4) Incentivize participation by providing something of value or consequence

46. What Can’t Be Done to Address Non-participation; What Can
1) Nothing 2) More complete data collection 3) Reassure that all scores, categories are still valid 4) Incentivize participation by providing something of value or consequence 5) Actually reduce testing time

47. Seven Strategies for Reducing Testing Time

48. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone)

49. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone) 2) Operational field testing

50. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone) 2) Operational field testing 3) Narrow testing windows

51. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone) 2) Operational field testing 3) Narrow testing windows 4) Eliminate untimed tests

52. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone) 2) Operational field testing 3) Narrow testing windows 4) Eliminate untimed tests 5) Shorter tests

53. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone) 2) Operational field testing 3) Narrow testing windows 4) Eliminate untimed tests 5) Shorter tests 6) Reduce/eliminate other state/district mandated and optional testing

54. Seven Strategies for Reducing Testing Time
1) Embedded field tests (not stand alone) 2) Operational field testing 3) Narrow testing windows 4) Eliminate untimed tests 5) Shorter tests 6) Reduce/eliminate other state/district mandated and optional testing 7) Out-of-school testing

55. Thank you.