Understanding Quick Scores & This Year’s ChangeUnderstanding Quick Scores & This Year’s Change Dr. Nakia TownsDr. Nakia Towns Assistant Commissioner for Data and ResearchAssistant Commissioner for Data and Research June 1, 2015June 1, 2015

Essential Questions  What is a quick score?  What changed about calculating quick scores this year in grades 3-8 and why?  What is the impact of the change in methodology for calculating quick scores this year in grades 3-8?  How is the department communicating directly and how is it supporting districts to communicate with parents and teachers regarding the change in quick score methodology?  What is the plan regarding quick scores for , with the transition to TNReady, and beyond? 2

Key Terms  Raw Score: the number of questions a student answered correctly.  Scale Score: generated from the raw score; this allows a comparison of student performance from year to year. Because the questions on the test change each year, scale scores give us a metric to compare apples to apples.  Cut Score: generated from the scale score; determines a student’s performance level (i.e., below basic, basic, proficient, advanced).  Quick Score: generated from the raw score; only to be factored into a student’s end of year grade, as required by law. This score is on a 100-point scale because student grades are on a 100-point scale.  Performance Levels: generated from the scale score to represent student’s level of proficiency. Performance levels are categorized as below basic, basic, proficient, or advanced. 3

TCAP Scoring Flow Chart 4

What is a Quick Score? 3 rd Grade RLA – What is proficient?  Though 72 percent correct means a student is proficient, this would be a low pass to failing grade on most A-F grading scales for 3-8.  Quick scores convert the number of questions students answered correctly (raw score) on TCAP into a measure on a 100-point grading scale as typically used by teachers.  A quick score is not the percent correct on TCAP.  A quick score is not a percentile rank. 72% correct 760 Scale Score 43 Raw Score 43 correct answers out of 60 questions Quick Score? 5

Quick Score Methodology Cube Root Scale In grades 9-12, the state has a uniform grading policy. The quick score calculation for EOCs has used the cube root scaling method since

Quick Score Methodology Interval Scale Grades 3-8 used the interval scaling method to calculate quick scores prior to this year, This method fixed quick scores to TCAP performance levels for grades and subject. (1) Number Correct = 0, then Numeric Score (or Quick Score)=0, (2) If Raw Score Basic Cut = 29 (2011 RLA, Grade=5), then 70 (Quick Score), (3) If Raw Score Proficient Cut = 48 (2011 RLA, Grade=5), then 85 (Quick Score), 7

Quick Scores  This past fall, prior to leadership transition, the office of assessment logistics made the decision to stop including TCAP performance levels with quick scores.  The department also decided to use the cube root scaling method for both high school EOCs and 3-8 Achievement.  These changes were made to streamline the calculations, providing a consistent methodology in high school EOCs and 3-8 Achievement.  It’s important to note, the interval scaling method cannot be used during assessment transitions before teachers complete standards setting to determine scale score cuts.  As the cube root methodology was used for EOCs in prior years, there was no change this year for students, teachers, and parents in grades

Quick Scores  The change in quick score methodology for grades 3-8 was not adequately or widely communicated in fall 2014, resulting in confusion for students, teachers, and parents in grades 3-8 with regard to interpreting quick scores.  Though no performance levels were published with quick scores, many relied on the prior interval scaling method and believed that a quick score of 85 was equivalent to proficient.  Possibly, some teachers may have communicated to parents that students were proficient who may not have been.  Possibly, some schools may have prematurely estimated their TCAP proficiency rates in advance of official accountability releases in June and July. 9

Quick Score Methodology Implications  The bar for proficiency has not changed. Scale score cut points are the same in 2015 as they were in  The quick score methodology does not impact TCAP performance levels.  The reality of student performance with regard to scale score cuts does not depend on the quick score. 10

Quick Score Methodology Implications How do quick scores impact teachers?  Quick scores do not impact evaluation or TVAAS, and quick scores have no bearing on personnel decisions tied to performance.  Quick scores have no impact on teacher, school, or district accountability metrics. How do quick scores impact students?  For grades 3-8, there is no statewide uniform grading policy. This means that a quick score of 84 in one district may be considered a B, while in another district it may be considered a C.  For grades 9-12, there is a statewide uniform grading policy, so that the quick score range for an A is consistent in all districts.  Quick scores count for 15-25% of the student’s semester grade or 7.5% to 12.5% of the end of year grade in

Additional Information and Support  Monday, May 18 – Quick scores using cube root scaling methodology released to districts for grades 3-8.  Tuesday, May 19 – Commissioner learns of fall 2014 decision to change 3-8 quick score calculation and determines this change has not been previously communicated.  Wednesday, May 20 – Commissioner s directors indicating quick score methodology change and timeline for additional information. Also included in Director Update.  Friday, May 22 – Detailed information released to districts regarding the quick score methodology for grades 3-8 achievement and high school EOCs, as well as tables showing quick score that is equivalent to scale score proficiency cuts.  Wednesday, May 27 – Conference call with directors and district data and accountability personnel to answer questions. May 22 and conference call information included in Director Update.  Thursday, May 28 – ed detailed quick score information to Tennessee Education Association, which was posted to TEA website. 12

Additional Information and Support  Thursday, May 28 – Released raw scores for spring 2015 achievement 3-8 and high school EOCs to all districts. Second conference call with district data and accountability personnel.  Friday, May 29 – Launched Understanding Quick Scores website.  Friday, May 29 – Released Educator Update and Principal Update with message from Commissioner and additional quick score information.  Friday, May 29 – Released Classroom Chronicles blog post from Commissioner with apology for department communication failure and additional quick score information for all educators.  Saturday, May 30 – Released raw scores and corresponding quick score using interval scaling methodology for spring 2015 achievement 3-8.  June 1-15 – Regional face-to-face meetings scheduled in East, Central and West Tennessee to answer additional questions for district personnel regarding quick scores and TCAP.  June 1-15 – Additional meetings with various stakeholder groups. 13

Quick Scores in and Beyond  New TNReady assessments for math and English language arts, as well as new social studies and US History assessment will be introduced next year.  Teachers will not complete standards setting to define scale scores cuts for any of these subjects until summer  Though quick scores can be calculated in advanced of standards setting, the department will seek additional stakeholder feedback to determine what is most appropriate.  Over the next few months, the department will engage TOSS (superintendents) Working Group in discussion about quick scores for 3-8 achievement and high school EOCs for  In August 2015, the department will communicate initial decision regarding quick scores and follow-up in September with more details during regional assessment meetings, as well via department updates and on public websites. 14

Resources  Learn more about Understanding Quick Scores:  Learn more about TNReady:  Preview questions for TNReady via MICA Item Sampler: All teachers have logins – Commissioner on 5/19. Contact district testing coordinator or Public access via “About” menu.  Learn more about all TCAP Assessments: QUESTIONS? 15