Beth Fultz Kansas Department of Education Test Delivery Beth Fultz Kansas Department of Education National Conference on Student Assessment Saturday, June 22, 2013

Technology Readiness

Data My school was prepared with the technology required to administer the SBAC computer- based Pilot test: Strongly Agree Agree Neither Agree or Disagree Disagree Strongly Disagree 26.4% 48.8% 6.8% 13.6% 4.3%

Lack of understanding around length of test - Volume Control Common Issues Lack of understanding around length of test - Volume Control Headphones Ability to move to the next question Programming issues Skipped questions Split screens

Computer Delivery States Assumption - SBAC computer test technology would work just like the state assessment Required closer coordination with district IT staff Multiple test takers using the same computer on the same day (access point/reboot)

Band-width and wireless connectivity Age of school computers Paper-Pencil States Band-width and wireless connectivity Age of school computers Unexpected “internal system updates” Students not as familiar with online matching, drag-and-drop and calculator tools Lack of experience in dealing with small problems Student getting logged out Browser getting stuck

Browser/Delivery Systems

Neither Easy or Difficult Data How easy or difficult did you find the SBAC delivery system to use: Very Easy Easy Neither Easy or Difficult Difficult Very Difficult 8.7% 39.8% 28.9% 18.8% 3.7%

Data SBAC Kansas Windows 81.29% 74.89% iPad 1.89% 10.15% Mac 14.74% 14.92% Other 2.08% .04%

Drag & drop didn’t always work Common Issues Audio Film clips - often audio would not play Re-listen to entire section – not a selected section Drag & drop didn’t always work Issue – test would not allow the student to go on to next question

Difficulty seeing the entire question/scrolling iPad and Tablets Keyboards Difficulty seeing the entire question/scrolling Software would only work if all other programs were closed Overall, very successful pilot Fewer technology issues

Technical Difficulties

Data Did you, or any of the test takers, experience any technical difficulties during the administration of the test: Yes 78.6% No 21.4%

What technical difficulties did you, or any of the test takers, experience? Reasons/Responses Problems with school computer equipment 36.0% Issues with the testing platform/ the test itself 51.5% Lack of resources to conduct testing 10.1% Overall the system was difficult to use 14.2% Training materials were inadequate 15.0% Insufficient time allowed for introducing students to the test environment 21.3% Other 22.1%

Conclusions

Comment from a Kansas Testing Coordinator: “Most problems are of the sort that will be resolved before the final test is operational. In other words, the pilot is serving its purpose.”