San Mateo County Results from the 2014 SBAC Field Test Survey Deann Walsh Manager, Learning Analytics & Program Evaluation

Agenda Field Test Overview Field Test Survey Results Student Survey Teacher/Proctor Survey Administrator Survey Next Steps

“A field test is not designed to be a valid and reliable measure of student achievement; rather, it is designed to help the test developers evaluate whether the test, individual items, and the technology platform work as intended before the first operational administration.” Deborah S. Delisle, U.S. Department of Education

Field Test Overview Timeline: o March 25 th through June 6 th o Schools assigned a five- or six- week window Participants: o Grades 3-8: All students o Grades 9 & 10: Only students selected for the scientific sample o Grade 11: Students selected for the scientific sample; all others encouraged Components: o Performance Task o Non-performance Task Format: Computer-based; not computer adaptive

SBAC Field Test Surveys Survey Number of Responses Number of Districts Responding Student Teacher/Proctor33617 Administrator5613

Student Survey Results Methodology Used o Weighted student results based on % of 2013 STAR testing population Approximately 28% of students encountered problems with the technology or testing system Student Preparation 50% Completed a practice test together as a class 53% Completed a practice test individually 54% Teacher reviewed technology needed for testing (i.e. computer, mouse)

Student Survey Results, Continued Testing Conditions 33% Computer Lab 56% Classroom with Laptops 4% Classroom with Tablets 7% Other Content Approximately 56% of students polled felt comfortable or very comfortable with their understanding of the questions asked

Teacher/Proctor Survey Results * Highest ranked item of the PD options Teacher Preparation 93% Overview of the SBAC Field Test 66% Instruction on the selection and administration of universal tools, designated supports and accommodations 78% Demonstration of the test administrator interface 70% Physical copies of required manuals

Teacher/Proctor Survey Results Impact on Instruction 53% indicated that the use of computer devices for testing purposes impacted their ability to use technology for instructional purposes Professional Development 85% of teachers ranked “training on preparing students for the content” as important or very important* *Highest ranked item of the PD options

Administrator Survey Results Technology & Systems 66% Experienced significant glitches in the system (e.g. device failure, missing log-in information) 89% Indicated that the network and bandwidth were reliable during testing Impact on Instruction 67% indicated that the use of computer devices for testing purposes impacted the school’s ability to use technology for instructional purposes

Next Steps SBAC Digital Library o Preview began on June 3 rd o Digital Library SMCOE August 5 th o Operational version to begin on October 1 st Interim Assessments – Available Fall 2014 o Interim Comprehensive Assessment (ICA) o Interim Assessment Blocks (IAB)

Next Steps Summative Assessment – Spring 2015 o Administered in the last 12 weeks of the school year o Operational test will differ from field test in terms of duration & length

SMCOE Support CIS Support o Teacher Professional Development o Administrator Professional Development ESS Support o District Assessment Leader Network o Regional Learning Analytics Institutes o Target Supported on Universal Tools, Designated Supports and Accommodations with Special Education & Assessment Staff

Questions?