Race to the Top Technology and Innovation in Assessments Boston, MA Tony Alpert Oregon Department of Education

Online Adaptive Student Engagement: Provide tests that are more consistent with students’ abilities rather than too easy or too hard Content Coverage: Can cover the same grade level content but with different difficulty based on student ability Precision of Results: Provide more precise estimates of student ability for lowest and highest performing students including SWD and ELL Reporting of Results: Provide immediate feedback to students and teachers

Secure Connection to Online test that restricts access to only allowable resources (e.g. Secure Browser) Robust Systems: OS independent and OS resilient (platform and version) Efficient: Small Footprint with minimum of overhead Flexible: Function on a variety of network conditions (e.g. server, clients, thin clients, etc) User-Friendly: Minimum of IT staff support Pragmatic: Work on the oldest computers with the least amount of memory used among the districts with the least amount of resources

Project Management Project Manager System Life Cycle Development Process Planning: Specific Tasks with Roles and Responsibilities Weekly project summaries with Risks and Mitigation Strategies Requirements Documents Mock-ups

Application test plans documents Documentation of unit and systems tests User acceptance testing in development and pre-production environments Change management with Specific planned enhancement cycles Project Management (cont)

Universal Design Computer Interfaces: Netbooks, notebooks, desktops, Handhelds and tablets. Goal should be 1:1 computer to student ratio Input: Speech-to-Text, Stylus, Mouse, Mouse-Pad, Keyboard, Touch Screen Output: Text-to-Braille, Text-to-Speech, Text-to-ASL; Dynamic presentation and/or item selection based on students’ disabilities

IT Support IT Diagnostic tools to pinpoint desktop vs server vs network vs ISP problems. Added value if it can provide pragmatic solutions in plain language. Bandwidth as a Resource: Traffic shapers Cell phone Wi-Fi Routers Reports of Bandwidth Capacity with Longitudinal Comparisons Professional development for LEA IT staff

Planning and Administration Support Provide tools to plan and to help ensure the right students take the right test at the right time with the right accommodations Length of testing window needs to consider test security, volume of kids, anticipated test duration, # of computers and bandwidth availability Teacher, parent and student portals with robust multi-tiered security Real-time individual and aggregate results

Efficiency

Computer Based Testing Online Field Testing without spiraling Can deliver multiple opportunities No shipping and sorting Distinguish cost from expenditures Oregon spends more on its Online assessment, but adds more value than paper. Estimated cost for paper is about $15 per test as compared to Online assessment which is about $4 per test

Online Test Development Distributed Item Writing Online Collaborative Content Review Online Collaborative Bias Review Online Alignment Review Online Test Development System

Online Writing/Constructed Response Artificial, Human and hybrid options Combine Paper and Online Random Allocation of validity and qualification papers Random back reads Real time statistics regarding reliability

Support Capacity Provide incentives for LEAs to manage resources well Provide incentives for Assessment Design to be more efficient Bandwidth needs are not as great as one might believe Good planning and flexible windows can compensate for capacity

Support Practitioners

Professional Development Assessment Literacy Training and Certification on the use of the System IT Literacy Multi-tiered help desk support (e.g. Regional Help Desks for Tier 1 and local issues. Central help desk for more complex or application issues)

Formative, Interim and Summative Training serves initial purpose of interim assessments. Best item writers “graduate” into summative assessment Allows interim assessments to potentially benefit from information derived from the summative assessments Allows instruction to be delivered consistent with students real-world application and the manner in which students will be assessed