Tablet Computers and Standards of Learning Testing: Insights from the Virginia Department of Education Monday, August 12, 2013

August 12, 2013 Virginia’s Standards of Learning Assessments: A Current Snapshot Virginia Demographics – Approximately 1.2 million students in grades K - 12 – Approximately 88,000 public school teachers – 132 public school districts Standards of Learning (SOL) Assessments – 34 assessments in English (Reading & Writing), Mathematics, Science, and History at Grades 3 – 8 and End- of-Course – Use of Multiple-choice and Technology Enhanced Items English Writing tests include a short-paper component

August 12, 2013 Virginia’s Standards of Learning Assessments: From Paper/Pencil to Online Tests 2000: Web-based SOL Technology Initiative begins – Emphasized technology, instruction, and assessment – Included a Demonstration or Proof of Concept Phase for online assessments – Existing funding targeted to emphasize school readiness 2001: First operational online SOL tests administered – High schools administered 1,700 online tests in fall 2001 – 3 of 12 EOC tests available in the online format – Optional participation at the division and school levels

August 12, 2013 Virginia’s Standards of Learning Assessments: From Paper/Pencil to Online Tests 2006: All tests for all grade levels available online except the SOL Writing tests (gr 5, 8, and EOC) 2010: Legislation requires all SOL tests to be administered online (except for students with a documented need for a paper test) by: – for all high schools – for all middle schools – for all elementary schools

Virginia’s Online SOL Assessments : Number of Online Non-Writing Tests Completed by Administration

One of the Next Steps in Virginia Assessments: Expanding From Computers to Tablets

Increased availability of tablets Increases in classroom use of tablets – For instructional purposes – For diagnostic and formative assessments – For administrative use Advancements in tablet technologies and mobile device management technologies

One of the Next Steps in Virginia Assessments: Expanding From Computers to Tablets Various implementation models in schools – Tablets for teachers and administrators – Tablets for special projects or programs – Classroom sets of tablets – One-to-one tablet initiatives A growing body of research about tablets in education

August 12, 2013 One of the Next Steps in Virginia Assessments: Expanding From Computers to Tablets Initial Considerations – User Interface – Size and orientation of display – Tablet operating systems – Increased mobility – Network demands – Limited use in high stakes assessments – Compatibility

August 12, 2013 Tablet Research in Virginia Cognitive Lab - Spring 2012 Online Writing Field Test Onboard and external keyboards Item functionality of technology-enhanced items Report available online: Fall 2012 Tablet Usability Research Specific Technology-Enhanced Item Types Different Tablets and Screen Sizes Report available online: 10

August 12, 2013 Tablet Research in Virginia: Findings Impact of touch screens on technology- enhanced items – The “dragger” or “hot spot” may be smaller than a student’s finger. CCSSO

Example of “Dragger” Smaller than “Finger Size”

Tablet Research in Virginia: Findings When typing a longer response, students found that the imprecision of their fingers to indicate location interfered with their ability to select text when trying to replace words or to fix spelling, punctuation, or capitalization.

August 12, 2013 Tablet Research in Virginia: Findings Students with typing/keyboarding skills were slower and less accurate with the on-screen keyboard and become frustrated with writing tasks. Younger students without typing skills often preferred the on-screen keyboard. Text selection and editing on a tablet was difficult even with the external keyboards; not all students used the keyboard’s arrow keys to position the text cursor. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

August 12, 2013 Tablet Research in Virginia: Findings Tools currently available in TestNav, the online assessment software currently used in Virginia, may not be available on tablets (e.g., hover feature) or may be more difficult to use (e.g., highlighter, pencil tool). Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

On a computer with a mouse or track pad, the icon representing the cursor on the screen will vary depending on the tool that was selected. No cursor appears Cursor is represented by the highlighter icon On a tablet, no cursor appears on the screen after a tool is selected. Users cannot rely on the appearance of different cursor icons.

On a computer with the pointer tool selected, the cursor appears as the hand/finger icon when hovering over text that can be selected. No cursor appears Cursor is represented by the hand/finger icon when hovering over an answer option indicating the option can be selected by clicking. On a tablet, no cursor appears on the screen after a tool is selected. Users cannot rely on the appearance of different cursor icons.

August 12, 2013 Tablet Research in Virginia: Findings Tools such as the calculator or the tablet’s onboard keyboard “cover up” the test item when accessed. 18

Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Tablet Research in Virginia Spring 2013 Written Composition Study Conducted with grade 5 and grade 11 students Five study conditions included: – iPad only – iPad plus external keyboard – iPad plus stylus – iPad plus external keyboard and stylus – Standard laptop Results not yet available. Will be posted at: http: // researchnetwork.pearson.com

August 12, 2013 Considerations for Expansion to Tablet Use Logistical considerations in “locking down” tablet desktop for secure testing Ensuring that items are developed to display on both computers and tablets Examining tools available within online assessment software (TestNav) and making adjustments for tablet delivery Continuing to evaluate onboard keyboard versus external keyboard

August 12, 2013 Considerations for Expansion to Tablet Use Flexibility in tablet positioning versus security concerns Impact on test administration procedures – Desktop lock down for school-owned versus student owned tablets – Additional procedures to discourage students from “sharing” tablets during testing

August 12, 2013 Timeline for Inclusion of Tablets in SOL Test Administration Additional research/pilots in Depending on results, move to administration of some tests on tablets in Current plan is to introduce tablet use in a phased approach

Questions? the Division of Student Assessment and School Improvement at