Tennessee Data and Attendance Supervisors’ Conference Friday, April 22, 2016 Deborah Sauberer, Executive Director Assessment Logistics

TNReady Structure: What has changed and what is still the same

Change to Paper-Based TNReady Over the past few months, the department and districts worked together to strengthen our technology infrastructure and prep systems and devices across the state. On Feb. 8, a new issue we had not experienced before caused a network outage at the test vendor’s site that interrupted testing for some students. In order to protect our students’ instructional time and better ensure they had a positive test experience, we decided to move all students to a paper-and-pencil version of TNReady for the rest of this school year. This change applies to both Part I and Part II of TNReady, as well as social studies.

The Work to Prepare Will Still Pay Off TNReady on paper is still TNReady. The questions students practiced on a computer are the same types of questions they will see on paper. Developing keyboarding skills are part of our digital literacy expectations (which apply to grades K-12) and match the real-world demands we are preparing our children to meet. Many schools upgraded or added technology capabilities and tools that can used in everyday teaching and learning, so work to improve technology infrastructure and device access will still benefit students’ learning.

Future Online Transition We are committed to a computer-based assessment in the long-term. Technology is part of the real world for which we need to prepare our students. The preparation that the state, districts, and schools have taken over this past year to get ready will both support this transition and, most importantly, our students’ education. The department is currently exploring all options regarding the 2016-17 school year. However, due to contractual considerations and continued exploration, no decisions will be made until after May 2016. We will communicate as soon as possible (summer 2016) regarding plans for a future transition to online assessment.

Second Grade Assessment Update Questar Assessment, Inc. was the sole respondent and earned an average score of 89 of 100 points. Contract being negotiated and should be fully executed by April 15. Operational assessment available in spring 2017 for the 2016-17 school year. Evaluating possibility of pre-test for fall 2017 to potentially use for the generation of individual TVAAS for second grade teachers in 2017-18

Second Grade Assessment Update The second grade assessment will be paper/pencil. No current plans for online format. This is an OPTIONAL assessment. It is an LEA decision to participate (or not) in this criterion-referenced, Tennessee- standards specific assessment. K-2 (SAT-10) discontinued after this spring 2016 administration. No cost to districts who opt into 2nd grade assessment. Questar Contract Awarded Test Development & Design Distribution of Blueprints and Practice Items Operational Assessment Spring 2016 Summer 2016 Fall 2016 Spring 2017

Standards Setting

Standards Setting Committee Committee members will be selected from multiple stakeholder groups (educators, higher education partners, parents, community members) ELA Mathematics Social Studies Grades 3-5 Grades 6-8 Grades 6-7 High School High School: Alg. 1 and 2 High School: Geometry High School: Integrated Math 1, 2, and 3 Grade 8 and US History Virginia will edit the dates when she has all of the information from Standard Setting application.

Standards Setting Committee Knowledgeable in tested content Familiar with tested students Diverse backgrounds, demographics, and experience Gender Years of experience Region Ethnicity Student population

Standards Setting: Purpose To make recommendations to the Tennessee State Board of Education regarding cut scores for all grade 3-11 state assessments. To ensure recommendations are consistent with expectations stated in the Tennessee Academic Standards.

Standards Setting: What is a Cut Score? The minimum scale score a student must earn to be considered at a certain performance level Three cut scores result in four levels of performance. Level 1 Level 2 Level 3 Level 4 Cut Score A Cut Score B Cut Score C

Standards Setting: Ordered Item Booklets (OIB) The difficulties of these items are based on actual student performance. The actual responses of students who have taken the tests are used to compute the item difficulty Then these items are put in order from least difficult to most difficult.

Standards Setting: Bookmarking Process Level 1 Level 2 Level 3 Level 4 Based on the threshold PLD, think of target student for each performance level, from lowest to highest. The committee will go through the OIB and consider, item by item, whether the target student would be able to answer the item correctly. If not, the cut score marker is placed before that item.

Standards Setting Committee Applications are being accepted until April 25 at 5pm CDT Application link: http://tinyurl.com/Standards-Setting-Application

Reactions What are you excited about? What are your concerns?

Spring 2016 Testing Timelines

Data Review Phases YOU are Here Phase 1 Phase 3 Phase 4 Phase 2 EIS Download Demo Stats/ Review Demo Stats/ Roster Review Testing Discrepancy Review Instructional Roster Linkage Roster Quick Scores Pre-Id Upload Phase 2

New Assessments – New Terms Key Terms Part: the TNReady assessments (ELA and Math) and Social Studies will be given in two parts. For Spring 2016 Grades 3- 8, this will be February 22 – March 18 (Part I) and April 25 – May 6 (Part II). Subtest: a division of items within a test administration. For example, the ELA Part II assessments have two subtests of 85 minutes each. Session: a scheduled testing time. Students are scheduled for a testing session.

Spring Part II 2016 Scheduling Parameters   Grades 3-8 EOC Day 1 Mathematics Algebra I/Integrated Math I, English III Day 2 English Language Arts Algebra II/Integrated Math II, English I Day 3 Social Studies English II, U.S. History Day 4 Science (Part II only) Geometry/Integrated Math III, Biology/Chemistry (Part II only) Day 5 Completion of all SPED, English Learners and make-up testing NOTE: Make-up testing can begin any time, starting with Day 2.

Update to Scheduling Part II All testing may be scheduled through May 10, rather than limiting grade 3-8 testing to an end date of May 6. Districts may modify their testing schedules as needed, without any prior approval or notice to the department. This flexibility includes scheduling grade 3-8 or high school testing over more than five (5) days and skipping a day if need be such that testing days are not necessarily consecutive. EL and SPED students may not be tested outside the district/state testing window.

Administration Guidelines Confirm each student has their own Pre-ID answer document. Students may have clean, blank scratch paper during all subtests of the assessments. Scratch paper may be lined, unlined, or graph paper and should be collected and securely destroyed after each test session. Students will need a straight edge for math assessments; rulers, index cards or other straight edge may be used. There are no items that require rulers for measurement or protractors on the math assessments.

Best Practices/Recommendations Whenever possible, there should be a test administrator and a proctor in the testing environment. Best practice is to schedule students with administrators and proctors who are not their regular classroom teacher. Test administrators must be employees of the school and/or district. Proctors may be school volunteers. Proctors may not be left alone with students, proctor family members or handle materials. If a student finishes testing prior to the scheduled time, they may read or do other quiet academic activities that are not related to the content area being tested.

Schedule Guidance Related to Inclement Weather Districts may include a maximum of 13 stock pile days within their calendars by scheduling the school day up to an extra 30 minutes per day. Stock pile days that are used for events such as snow or ice count as available school days because the equivalent time has been embedded in the schedule.

Schedule Guidance Related to Inclement Weather District A has 13 stock pile days and has missed 15 days due to inclement weather; District A will need to make calendar adjustments to make up the 2 days missed in excess of their 13 stock pile days in order to meet the requirement for 180 days of instruction. District B has 13 stock pile days and has missed 10 days due to inclement weather; District B does not need to make any calendar adjustments at this time.

Schedule Adjustments Districts, like District A above, who need to adjust their school calendar should send the request to Paul.Rainwater@tn.gov and make any changes to the school calendar, i.e. report card date or graduation date, in EdTools. Districts that need to adjust their testing schedule should complete a contact support ticket in EdTools, once approval is received the EdTools calendar must be updated.

Statute Related to Test Administration TCA 49-6-6002 (b) stipulates that TCAP test should not be administered prior to completion of 150 days of instruction. Stock pile days count towards instructional days. District B above will not need to make any calendar adjustments. District A above will need to consider the following: When did school start When is testing currently scheduled to begin How will the 2 missed days be made up The answers to these questions will determine how the school calendar and testing schedule should be adjusted.

Statute Related to Claiming TCA 49-1-606 (a) states: A student must have been present for one hundred fifty (150) days of classroom instruction per year or seventy-five (75) days of classroom instruction per semester before that student's record is attributable to a specific teacher. Used stock pile days are not counted as absent or unavailable for instruction. Student instructional availability is determined using the combination of enrollment and attendance. Participation in school activities or events is not considered unavailable for instruction. The 150 (75) days of instruction is out of a possible 180 days, such that a student who has missed 20 days at the time of testing but is expected to be in attendance through the remainder of the school year would be considered available 150 days.

Data Collection

Business Rules for the EIS Pull For a Student record to be pulled – the student must have: An Enrollment in a school An Instructional Grade record for that enrollment A Student Class Assignment (course code) record for that enrollment A Class Section (local class number) for the Student Class Assignment A Teacher identified as the teacher of record for the class section (Master Teacher record) A Valid Teacher License number for the teacher of record (above)

File Uses Students will receive pre-ID paper response documents for testing. Establishes initial class rosters in EdTools for Teacher Student Connection (TSC)

EIS File Layout - Location

EIS File Layout The EIS File Layout is an excel workbook that contains separate worksheets for each of the following: Record layout: details for each field in a student record, the valid values for that field, the rules used by our EIS programmer to pull that information, and the EIS extract record source. Rules: additional business rules that would not fit in the file layout 3-8 Course and 3-8 Priority: the 3-8 course codes and the order in which they appear in the course table for the EIS pull Secondary and Secondary Priority: the secondary course codes and the order in which they appear in the course table for the EIS pull Schedule: the specific fields used to determine the student’s class, a student may have up to 9 courses in the record – this worksheet also contains more detail about the rules for Membership Demo Stats rules: Special rules used to populate the demo stats page in EdTools

Fields 1-4 Fields 1-4 contain general information to identify the test, district, school. These fields are automatically populated in the file and do not require anything from the school or district.

Fields 5 -10 Fields 5-10 contain student identification information that should be included in the student enrollment. A middle name or initial is not required.

Fields 11-18 Fields 11-18 contain ethnic origin, race and gender, these fields should be included in the student enrollment. Only student gender (field 18) will be uploaded into MICA. Ethnic origin and race will be available for review in Demo Stats in EdTools.

Fields 19 – Code A/B Fields 19 is a required field and populates information for school nutrition. In 2015-2016 a new student classification “J” has been added to identify all students who are eligible for free or reduced lunch through direct certification. A “directly certified” student is any student eligible to receive free lunch without an application, regardless of the district or school CEP status.  Code "J" will identify students that are direct certified via SNAP, TANF, foster or by state or local officials. This will impact all schools. Resources for regarding code “J” are provided on the next slide. In addition to Code J, Codes H, I, and U will be included in the Code A eligible classification.

Fields 19 – Code “J” Resources Information regarding Free and Reduced-Price Meals is available on the website: http://tn.gov/education/article/free-and-reduced-price-meals The memo defining the economically disadvantaged subgroup for accountability: https://gallery.mailchimp.com/b28b453ee164f9a2e2b5057e1/files/ED_Definition_for_Accountability_Memo.pdf Memo regarding classification code “J”: https://gallery.mailchimp.com/b28b453ee164f9a2e2b5057e1/files/New_Student_Classification_Code_J.pdf The FAQ regarding ED Definition for Accountability: https://gallery.mailchimp.com/b28b453ee164f9a2e2b5057e1/files/ED_Definition_for_Accountability_FAQ_.pdf

Fields 20-26 Fields 20-26 contain specific special program demographic information. The rules related to Fields 21, 22, and 26 are located on the rules tab of the workbook.

Fields 27-33 continued 34 - 89 Fields 27-89 contain the class schedule for the student. These fields will determine the class rosters in MICA and the Class Rosters in EdTools. Content areas require the course codes provided on the 3-8 and secondary tabs of the workbook. Testing Cycle will help to establish the test administration for the student.

3-8 Course Codes The course codes must be used in order for a student to receive a pre-id label or document. Applies to Fields 29, 36, 43, 50, 57, 64, 71, 78, 85

Secondary Course Codes

Rules for Testing Cycle – Test Schedule Determination of testing cycle will impact when a student is scheduled to test. This will be used regardless of class schedule in Field 93 (traditional/modified): Fall Block (F): Class end date < February 1 Spring Block (S): Class Start Date > December 1 Class End Date <= Attendance End Date or Null (EIS prefers null) Year Long Classes (Y): Class Start Date = Attendance Start Date Class End Date = Attendance End Date > May 1 or Null Applies to Fields 33, 40, 47, 54, 61, 68, 75, 82, 89

New Field for 2016-2017 EIS is adding a field in the 030 Class Section table. Test Admin Window: The class assessment schedule will be used for pre-id documents. Valid Values: Fall Block Schedule = “F” Spring Block Schedule = “S” Traditional (Year-long) Schedule = “T” Blank

Fields 90-93 Fields 90-91 are filler fields and are automatically populated. Field 92 indicates if a student is a CTE concentrator and is pulled from eTiger Field 93 indicates the school schedule and will be used to help establish block schedule test administrations

Fields 94 - 95 Fields 94-95 will generate Instructional Availability and Membership data. Previously these had to be hand entered. For Instructional Availability, which is used for teacher-effect, we will calculate the days based on date of enrollment and number of absences. Membership will be calculated based on date of current enrollment and any previous enrollments for this academic year. Membership is used as part of federal reporting only.

Fields 96 - 98 Fields 96-97 are new fields for 2015-2016. District 60% enrollment will be used to determine a student’s eligibility for inclusion in district accountability. It will be based on the date of enrollment. EL Excluded is determined on the first date of enrollment in a US School. Students who meet this criterion do not take the ELA assessments. They are required to take math, science and social studies assessments, however the scores for their math assessments do not count in accountability. Field 98 is the final field of the student record and is automatically populated.

Additional Rules The business rules for populating the following fields are provided on the rules tab: Instructional grade Title I status Service enrollments Teacher of record Alternative Schools Special Education hours Gifted Functionally Delayed Course codes Testing Cycle – this is very important for block schedule rosters Schedule – also very important for block schedule rosters Instructional availability – used for teacher effect Membership – used for federal reporting >60% enrollment – used for accountability

Reactions What are you excited about? What are your concerns?

Spring 2016 Administration

Block – Part I Pre-ID Student Labels Pre-ID Student Labels will be provided for all Block Part I. Additional blank answer documents will be provided for students who are not included in the Pre-ID file from EIS, enrolled late or change schools during testing. Accommodations, Modified format & Absent must be hand- bubbled. NOTE: The barcode information provided on a Pre-ID label wins over information bubbled on the answer document.

Block Part I - Answer Document Bubbles do not need to be filled in for students with a label. Place label in space provided, with barcode parallel to the document timing marks. Barcode will contain all necessary information to identify the student, school and district.

Part II Pre-ID Student Answer Documents Pre-ID answer documents will be provided for all Traditional and Block Part II. A human-readable student barcode will be printed on each form to identify each answer document. Additional blank answer documents will be provided for students who are not included in the Pre-ID file from EIS, enrolled late or change schools during testing. Accommodations, Modified format & Absent must be hand-bubbled. NOTE: The barcode information provided on a Pre-ID answer document wins over information hand-bubbled on the answer document.

3-8 Pre-ID Answer Document Bubbles will not be filled in, nor do they need to be. Barcode will contain all necessary information to identify the student, school and district.

High School Pre-ID Answer Document V R B D 3 2 6 4 9 Feb 24 2004

Accommodations Students with 504 Service Plan Special Accommodations A: Adult Transcription B: Assistive Technology C: Extended Time D: Read Aloud/Human Signer for ELA & Social Studies E: Visual Representations for Math F: Rest Breaks I: Unique Accommodation Request EL Accommodations R: Extended Time S: Word-to-Word Dictionary T: Rest/Breaks U: EL Exempt (ELA/SS)

Modified Format & Absentee Bubble the appropriate bubble – students may take the test in the test book or using a Braille writer but answers MUST be transcribed into a regular answer document. Absentee Status: Bubble if the student has been absent for ALL of the tested content area. Students who are unable to finish the test will be processed normally.

Test Administration Group Sheet (TAGS) TAGS for Part II will be used to establish class rosters for reporting. Test Administrator should be the teacher of record, up to 49 student documents may be placed under 1 TAG. Public Schools should place 1 grade and/or content area under a TAG. Transcribed answer documents for Braille, Large Print, or other students who take the test in the test booklet should be placed in the group with their teacher of record. Home School students may have all content areas under 1 TAG. In EdTools, use ELA for SGL in grades 3-8. For high school, use ELA or Math, if no ELA course taken.

SGLs REVISED New Audit – Combines SCAP and SGL data Part II SGLs will require TLNs.

Ancillaries Updated TAMs and Scripts for Block Part I and Part II will be uploaded into EdTools before start of day Monday, April 11.

Data Review

Processing Management Demographic Statistics Class Roster Assessment forms – includes SGLs & RIs SDDV Teacher-Student Connection

Student Data …. It’s all about the Demographics… SSN econ sped ell ethnicity Name gender Membership econ sped attendance ell accommodations

Demographic Statistics: Demographic Data Review NEW Demo stats will be refreshed with each bi-weekly pull through-out the year. Districts are able to download data at a school summary level and at a student specific level. This data is pulled by a TDOE EIS programmer and cannot be changed or researched by RANDA or Measurement Inc..

Delta Reports Delta reports indicate the percent of change between two different data pulls.

Individual Demographic Field At the School level click on the number in a specific demographic field to see the list of students currently reported.

RI’s The Report of Irregularity (RI) is used to report a serious irregularity during testing. The RI should be used only for the irregularities listed on the RI form. Sample Circumstances Action Recommendation Incomplete test section due to disciplinary reasons, illness, tardiness, etc. Complete RI (Option C) or Local Documentation Process Normally Student refused to test (no answers or random answering) Complete RI (Option J) Refusal/Did Not Attempt Cannot test due to serious long-term illness Complete RI (Option I) Medical Exemption Medical Exemption Time limits not followed (less than 30 minutes) Complete RI (Option D) or Local Documentation Time limits not followed (more than 30 minutes ) Complete RI (Option D) Nullify Student took wrong test (wrong grade level, wrong content area, etc.)* (Option F) Void Disruptions (loud noises, student behaviors, short power outage, etc.) Local Documentation N/A Major disruptions (long power outage, fire alarm, bomb threat, etc.) Contact State Student Cheating Complete RI (Option A) Adult Cheating Complete RI Potential Breach of Security (Option H) Received accommodation and is not a SPED student Complete RI (Option G) Depends on accommodation Complete RI (Option G) Post-Test Procedures Nullify if Read Aloud Accommodations not given to a student with an IEP Post-Test Procedures

RI Issues – Lessons Learned Teachers need to review materials with students and ensure that all responses are entered into the answer booklet. Students whose responses are in the test booklet must be transcribed and an RI must be entered. Students found with cell phones or other electronic devices will require an RI, we will only nullify the test if there is evidence that the device was used for cheating. Incomplete tests…

Student Demographic Data Verification (SDDV) Education Information System (EIS) Discrepancy reports

SDDV – Just Four Steps

Step One – Invalid USID This first step is just a year old… but clearing these up cures a number of issues down the line.

Step Two – Scan File Duplicates If you are unsure about the resolution, and it could be Part I and Part II, keep both records. TDOE will review them.

Step Three – Scan Only / EIS Only

Step Four – Demographic Discrepancies

Step Four – Demographic Discrepancies Five data points are required on paper answer documents: USID (Unique Student Identification Number) First & Last Name Date of Birth Grade Gender Discrepancies for these will have data for both EIS and Scan data

SDDV - Search

Student Record (1 of 3) Five data points are required on paper answer documents: USID (Unique Student Identification Number) First & Last Name Date of Birth Grade Gender

Student Record (2 of 3)

Student Record (3 of 3) Accommodations are not available in EIS and must be entered in EdTools.

SDDV – Mass Edit

Mass Edit Step #1 – Choose your search criteria.

Mass Edit Step #2 – You can select additional fields you want to change.

Mass Edit Step #3 – Results list based on your selections.

Teacher Student Claiming (TSC) This will be familiar territory. The process hasn’t changed.

Reactions What are you excited about? What are your concerns?

Reporting

Timeline for Data Return Year One Part I and Part II will be reported as single score. Quick Scores – February and June 2016 Standards setting will not be complete, so quick scores have no relevance to PLDs. TVAAS – July 2016 Achievement Reports Student Reports – October 2016 School Report – October 2016 District Reports – October 2016

Business Rules for Participation Scoring and Reporting

Participation Rate Calculation Numerator: Test-takers enrolled for Part I only + Test-takers enrolled for Part II only + Test-takers continuously enrolled for Part I & II Denominator: Enrolled students for Part I only + Enrolled students for Part II only + Continuously enrolled students for Part I & II

Enrollment Determination Part I: Last EIS pull prior to testing + students manually entered into MIST or provided on paper response document – students marked with non-participation codes in MIST. Part II: Last EIS pull prior to testing + students manually entered into MIST or provided on paper response document – students marked with non-participation codes in MIST or marked ‘not enrolled’ during SDDV. Last EIS pull scheduled for April 29. Please make updates to EIS before that date, to ensure smooth SDDV process.

Any questions about assessment logistics can be directed to: tned.assessment@tn.gov or (615) 741-0720

Thank You