Data Collections and Assessment: Demographic Data Meets Achievement Data September 1, 2010 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Agenda Summary of Assessment Division Assessment Preparation – Technology Files Used Facilities list FTE Enrollment (Collection 1) Pre-ID File Assessment Post Administration – Technology Files Used FTE Enrollment (Collection 1) Pre-ID File Student Record 2 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Summary of Assessment Division Assessment and Accountability Division Under Office of Standards, Instruction and Assessment Assessment Division responsible for the Creation and administration of tests, Dissemination of Reports Accountability Division responsible for Federal reporting (AYP, Report Card, Title III AMAOs) Assessment’s contact at district level is Assessment Director; aka System Test Coordinator 3 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Summary of Assessment Division Most of the tests mandated by State Law are criterion-referenced Tests given: GHSWT*, EOCT, Grade 8 Writing, Grade 5 Writing, Grade 3 Writing, ACCESS for ELLs, GHSGT*, GAA, CRCT* * program also has retest opportunities – Created solely for Georgia students (except ACCESS for ELLs) – Aligned to Georgia’s curriculum (GPS) ITBS is only Norm-Referenced Test 4 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Summary of Assessment Division Testing contracts are bid on via RFP process – Generally 5 year contract for each test program Assessment works with test vendors to create robust, reliable, and valid summative assessments customized for Georgia students Assessment is split into 2 Teams – Administration: manage program and support districts – ARD: QA for vendor and support Admin 5 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Facilities List ARD provides the Facilities List to test vendor in the Fall of each school year This file is a conglomeration of all districts’ information in one Statewide spreadsheet. Test vendor uses the list to update the school databases with correct school/system codes 6 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Facilities List Only 4 Fields are used by Assessment: System code, System name, School code, School name – Addresses are not used since many schools have a separate SHIPPING address where bulk materials (like testing materials) are received and processed. Assessment Division provides these shipping addresses to the vendor along with contact information of the System Test Coordinator 7 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Facilities List Facilities list becomes the Master List that all other school/system data are checked against and merged with The Facilities list makes it possible for: - Schools to receive correctly coded pre-printed materials (i.e., header sheets) - The vendor to begin organizing the shipping and receiving structure for the year 8 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: FTE Enrollment File FTE collection 1 yields data pertaining to the numbers of students per grade that are enrolled in a school by demographic groups (SWD, Ethnicity, GNETS participation, ELL status) ARD provides the FTE Enrollment File to test vendor in the Fall of each school year The FTE Enrollment File is a conglomeration of all districts’ FTE uploads 9 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: FTE Enrollment File Test vendor merges the enrollment Numbers per grade with the Facilities List At this preparation stage, the enrollment file provides information on – how many total test booklets will need to be created per grade (statewide) – How booklets should be bundled – How many boxes per district/school 10 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File The pre-ID file upload captures all students who will participate in testing event Names, ID numbers, demographic characteristics, and other information are included in label upload Purpose of labels is to alleviate the burden of gridding some parts of the answer document 11 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Benefits of using labels include 1.More time efficient than gridding 2.Improved data accuracy compared to gridding (esp. for GTIDs and FTEs) 3.Use label information as a confirmation of what information is housed in district SIS If label is incorrect, districts can check their SIS to be sure data are entered correctly for students 12 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Approx 4-6 weeks from the start of the testing window, ARD sends the pre-ID file to the vendor, includes all districts in one file Vendor prints labels for every student in the file (except those testing EOCT online) Hardcopy labels are shipped to districts: Sorted by school, grade/content area, teacher name, student name 13 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Fields that are printed on the label vary slightly by testing program, but these are common elements 14 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.” Student nameSystem name, code School name, codeGrade GenderBirth date GTIDEthnicity RacePrimary SRC code

Assessment Preparation: Pre-ID File Note that every field that is uploaded is not included on the hardcopy label (FTE, ELL, Teacher name) The label gives enough information to identify the student; all other fields are retained in the pre-ID file for later use 15 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Sample Label 16 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.” Sample CRCT

Assessment Preparation: Pre-ID File When schools receive labels, they are instructed to check that the information is accurate If the label contains wrong information (PSRC, GTID, Grade) it is not to be used—the school must grid the answer document No one enjoys gridding 17 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File 18 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Errors and Warnings upon upload were created to ensure proper testing data are uploaded This and other edit features are the prime benefits of uploading labels to the GaDOE instead of straight to testing vendor Not all errors can be prevented, but the edits and sign-off reports can assist in averting MAJOR snafus 19 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Sample Sign-off Report 20 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Still, there are some errors that can’t be caught… Examples of past ‘issues’ where the errors and warnings did not catch the problem 1.School received labels of all grades, but wrong students in each grade (e.g., 7 th grade students showing up as 6 th graders) – This is likely a case where the prior year’s extracted file was upload instead of current year’s file 21 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Examples of past ‘issues’ where the errors and warnings did not catch the problem 2.School received labels for most students, but a handful of schools, grades/content areas (for EOCT) are missing – Maybe one upload copied over another? 22 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Examples of past ‘issues’ where the errors and warnings did not catch the problem 3.School received labels, but they were not sorted – Likely that teacher name was not present in SIS for all students, or was not uploaded for all students 23 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Preparation: Pre-ID File Technology staff may not know what looks ‘reasonable’ for pre-ID data System Test Coordinators have access to the pre-ID information that the Technology Coordinators have uploaded Share the upload with the System Test Coordinators….Let them help ensure that pre-ID data are accurate! 24 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Post Administration: FTE Enrollment File Before scoring and reporting of test data begins, accuracy of scoring procedures must be checked One step is obtaining a representative sample with which to check equating A sample of districts is selected that looks like Georgia (statewide) when combined – Demographic characteristics (FTE Enrollment File) – Performance characteristics (Prior year test results) 25 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Post Administration: Pre-ID File After testing is complete and the data are scanned, district-level test data files and test reports are delivered to the district With test data files, districts can: load data into their SIS or data warehouse compute how district’s performance looks before official hardcopy reports are produced generate customized analyses as needed by the district 26 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Post Administration: Pre-ID File With Test Summary Reports, districts can see how groups of students in the district and school performed on the test Demographic data in test data files and on test summary reports come from pre-ID upload Goal: For these test sources to be similar to reports that use Student Record as source (e.g., AYP) 27 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Assessment Post Administration: Pre-ID File 28 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.” Districts can use these reports to respond to local board and media requests

Assessment Post Administration: Student Record After testing is complete and scores have been verified, test score to Student Record matching and AYP analysis begins Pre-id labels are integral to the matching of Assessment data to Student Record The resulting file is used for AYP determinations, Title III AMAO determinations, Report Card, other reporting 29 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Data Collections and Assessment: Demographic Data Meets Achievement Data Data received by districts are crucial for smooth assessment planning, administration, scoring, and reporting. Assessment Division appreciates your fastidious diligence in preparing clean, accurate data for each upload. 30 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Questions? 31 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”

Contact Information Adrienne Walker-Assessment Specialist Assessment and Accountability Assessment Research and Development Office of Standards, Instruction and Assessment Georgia Department of Education 32 Brad Bryant, State Superintendent of Schools “We will lead the nation in improving student achievement.”