1. DATAG The Importance of Good CBT Data
December 7, 2018
DRAFT

2. Glossary of Terms Acronym /Initialism Definition BEDS Code
Code: Basic Education Data System Code (SED Education Organizational Code)
BOCES
Boards of Cooperative Educational Services
CBT
Computer-Based Testing
ELA
English Language Arts
LEA
Local Educational Agency L2
Level 2 State Data Warehouse (WNYRIC)
NYSSIS ID
New York State Student Identification System Identifier
OODP
Out of District Placement Students
OSA
Office of State Assessment
SAFT
Stand-Alone Field Testing
DRAFT

3. Agenda CBT Overview CBT Data Requirements CBT Systems
CBT Data Problems & Solutions
Ordering CBT
Organizing CBT
NYSSIS IDs
Not Tested Codes
CBT and OODP
DRAFT

4. Computer-Based Testing, NY
Year
All schools given the option to administer stand-alone field tests on computers
902 schools administered field tests on computer
More than 60,000 students took the field tests on computer
Year
All schools given the option to administer operational tests with CBT in any subject, any grade level
184 schools administered operational tests with CBT
More than 28,000 students took the operational tests on computer
819 schools administered field tests on computer
Year
More than 650 schools administered operational tests with CBT
More than 145,000 students took the operational tests on computer
More than 1000 schools administered field tests on computer
DRAFT

5. CBT Schools by Region
RIC/Region
2017
CBT Schools
2018
NERIC 31
144
CNYRIC 53
110
WNYRIC 27 85
MHRIC 5 58
EduTech/Wayne Finger Lakes 12 47
Monroe/MAARS 6 43
SCRIC 19 41
Eastern Suffolk 15 32
Big 5
NYC, Rochester, Syracuse, Yonkers 4 49
Nassau 20
LHRIC 17
MORIC 1 9
Greater Southern Tier (GST) 2 7
We work closely with our RIC colleagues about CBT data issues:
RIC Working Group established in 2017
Our RICS are our first line of CBT Support in the field

6. CBT Data Timeline
DRAFT

7. CBT Data Requirements
Data requirements for the 2019 CBT Administration:
DRAFT

8. CBT Data Issues
We’ve already tackled a lot of CBT data issues and know the “importance of being good data”; bad data creates issues such as the following:
Student data cannot be loaded correctly to Nextera Admin for CBT administration
Student scores data does not flow to the district of accountability
We’ve developed solutions for myriad CBT data issues, but there is always more that we can learn and do.
“I have the simplest tastes. I am always satisfied with the best.”
- Oscar Wilde
DRAFT

9. CBT Systems Nextera Admin Nextera Test Delivery System
Online test administration system
Multi-tiered, role-based system
Manage students and student-test taking
Nextera Test Delivery System
Secure Browser
Students access and complete tests
Downloaded to students’ devices
Additional practice with previously released state question
New York Question Sampler:
Nextera Admin Video Tutorial:
DRAFT

10. Ordering CBT: Problem
Scenario: school orders tests under the wrong BEDS Code:
High School orders all tests for the district with multiple schools in the district
High School orders Grade 3 tests for the Elementary School
We identify students to populate Nextera Admin for CBT administration by the BEDS Code + Grade Level Enrollment
Outcome: L2 cannot locate any Grade 3 student enrollment records associated to the High School BEDS Code
Students cannot be loaded to Nextera Admin for CBT Administration
DRAFT

11. Ordering CBT: Solution
We added validation checks in two places:
CBT Team reviews CBT order reports daily to identify schools that have ordered outside of the grade level
We contact principals to advise them to adjust their test orders
In December, L2 sends reports to the CBT Team to distribute to the RICs that identify schools with missing student enrollment records for a grade level
The RICs contact the districts and schools to upload missing student records
DRAFT

12. Organizing CBT: Problem
In the first year of operational CBT, we loaded students to Nextera Admin according to their instructional classes:
SIRS Staff_Student_Course template
Loaded students into Nextera Admin according to ELA and Math class enrollment
Loaded all ELA and Math teachers:
Could only load one teacher per class
If a class had more than one teacher, the teacher with the “lowest” TEACH ID number was loaded
Primary Teachers were often missing from the data load
Teachers who were not involved with CBT were loaded to Nextera Admin
Related Article on CBT Support:
DRAFT

13. Organizing CBT: Solution
In the second year of operational CBT, our RICs and Districts suggested we load students into Grade-Level Enrollment “containers” in Nextera Admin:
Created phony classes
Created phony teachers
No teachers were loaded
Nextera Admin teacher accounts must be created by the district or school
DRAFT

14. NYSSIS IDs: Problem(s)
The NYSSIS ID is the student’s User ID for logging into the CBT exam and is created for a student when a student is first enrolled in a New York State (NYS) district/school:
How can a student who moves into NYS a few days or during the CBT administration window log into the CBT exam without a NYSSIS ID?
Students can be manually entered into Nextera Admin after the Pre-ID student data load, which allows for the potential of human error when typing a NYSSIS ID to create a student record.
DRAFT

15. NYSSIS IDs: Solution Fast-Track NYSSIS ID
Ahead of the first CBT administration (2017), we developed a process for districts to immediately create a placeholder NYSSIS ID for allowing a student to log into the CBT exam:
Can be created within minutes
Only available to CBT schools
Only available one week before and during the operational CBT windows
All Fast-Track NYSSIS IDs begin with “9999”
The district must still follow the standard process to have a NYSSIS ID created for the new-to-NYS student
The district must replace the Fast-Track NYSSIS ID with the student’s permanent NYSSIS ID in Nextera Admin
Fast-Track NYSSIS ID
DRAFT

16. NYSSIS IDs: Solution
DRAFT

17. Not Tested Codes: Problem
It is the responsibility of the school to mark students in Nextera Admin with Not Tested Codes when testing on computer:
Can be completed at a “later date” than at the moment the students are testing (potentially different workflow within the school from print answer sheet workflow)
A missing Not Tested Code, such as Medically excused, can result in a student’s score not being counted properly
DRAFT

18. Not Tested Codes: Solution
We remind our schools to verify student Not Tested Codes are set before the data is pulled for scores, and we ask the RICs to help with these reminders.
Summer Cleanup remains an option
Schools should NEVER use a print answer sheet when a student is testing on computer or refusing a CBT test session.
DRAFT

19. OODP: Problem Solution
Working through issues related to Out of District Placement (OODP) student populations and data flow is an example where NYSED, L2, and the RICs worked very closely together to develop a working solution:
Added new data fields to the Nextera Admin student data population file layout
Developed new workflows for ordering CBT
Developed new workflows for scoring CBT
Flexibility for BOCES with Virtual Locations
DRAFT

20. OODP Student Data
We load students based on enrollment + demographic data, identifying students with the following three data elements:
Home District (“district of accountability”)
Testing District
Testing Location (School)
BOCES program schools roll up to the primary BOCES:
L2 data warehouse
Nextera Admin
DRAFT

21. OODP Student Data Flow
L2 is already deriving OODP status for your students.
Identification: When the first 8 digits of the LEA District BEDS Code does NOT match the first 8 Digits of the Building BEDS Code (and building BEDS does not end in“0888” or “0777”)
THEN
L2 updates the last 4 digits of the Building’s BEDS Code to “OODP.” This is internal to L2.
OODP
= Student not identified as OODP
DRAFT

22. OODP Student Data Flow
OODP Students load to Nextera Admin under the testing district and testing location so that the OODP school has the ability to administer the computer-based tests to the student.
DRAFT

23. OODP & CBT
Ordering:
The testing location of the student is the location responsible for ordering the test.
Scoring:
The testing location of the student is the location responsible for scoring the test.
DRAFT

24. OODP & Scores Scores flow back to the district of responsibility.
Current processes in place for paper OODP student scores should be followed for CBT student scores.
As a result of the student data elements, district of accountability, etc., the scores flow back to the district and not to the BOCES or “testing location.”
DRAFT

25. District Moves
We had a lot of district moves last spring during the CBT administration. The best way to handle student district moves remains an area to improve for CBT with the following known issues that can result in complications for scores data flow :
Not editing a student Local ID once a student is manually moved in Nextera Admin
No data fields in Nextera Admin for the additional OODP home district information
Not exiting a student at the previous district
And more ….
DRAFT

26. CBT Listerv
Subscribe to the CBT Listserv for weekly messages about what’s new with CBT!
Send an to
First Name / Last Name
Address
Name of Educational Organization
BEDS Code
DRAFT

27. Questions about CBT Transition? Contact CBT Customer Support