NYCDOE’s Advance: Measures of Student Learning

NYCDOE’s Advance: Measures of Student Learning
CFN 109 Leadership Meeting, August 26, 2013

CFN109 Leadership MoSL Training
Welcome, Review Agenda, Norms & Goals 10:45 – 11:00 a.m. Overview of MoSL Long & Short Term Priorities 11:00 – 11:10 a.m. Assessment Types 11:10 – 11:30 am Baseline Measures 11:30 – 11:35 am Goal Setting vs. Growth Model 11:35 – 11:45 am The Default Option 11:45 – 11:55 am Review of Next Steps 11:55 am – 12:00 pm Q & A 12:00 – 12:10 pm Feedback 12:10 – 12:15 pm

  Goals for the Session School Leaders will:
Be able to work with Local Measures Committees to make decisions about Assessments, Target Populations & Growth Measures for MoSL that can be feasibly implemented within their schools. Understand the short and term critical dates and deadlines for implementing MoSL   Welcome/introductions Why are we here together for the next few days and what are we hoping to accomplish

Session Norms High Engagement, Low Tech – Please turn off cell phones.
Come Together – Please raise your hand when facilitator calls for attention. Contribute to the Learning – Throughout the day, write any outstanding questions on index cards and place them in the center of your table. Q&A at the end or follow up s and conversations will address these and any other lingering questions. Maximize Our Time Together – Use this day to collaborate, troubleshoot and brainstorm with your colleagues to prepare your school for implementing MoSL. Time:1- 2 min Facilitator: Jessica Materials: None Overview: Introducing the norms will begin to assimilate them to our culture and set a standard of expectations for all them as a cohort. This is extremely important to state up front. This can impact the level of quality all participants experience. Facilitation: ~ <1 min Facilitator reads the text in the blue boxes and emphasizes the importance of norms and expectations of active participation which will impact what they take away as learners and what their colleagues take away. Next, ask 1 TC each to read the four bullet points. Before transitioning, quickly check that all TCs agree to participate within these norms. ~1 min Facilitator transitions: For the next activity we want to get to know each other a little better and know what we are bringing to the party. What experiences make up what is known as the Office of Teacher Effectiveness? What journeys have we all taken to get to this destination?

New York State Education Law 3012c
Passed in 2010, New York State Education Law 3012c introduced significant changes to the Annual Professional Performance Review for teachers and principals. The law is intended to foster clearer expectations for practice aligned to improvements in student achievement. Summary of State Policy on Teacher Evaluation Teachers will be evaluated annually on a 4-point rating scale (Highly Effective, Effective, Developing, Ineffective) 40% of a teacher’s evaluation will be based on an assessment of student growth All classroom teachers will be evaluated against uniform qualitative rating criteria through use of a state-approved research-based rubric of teacher practice Teachers will receive timely and constructive feedback, including use of improvement plans for tenured teachers who receive a Developing or Ineffective End-of-Year rating Time:2 min Facilitation: Lisa Materials: Overview of the activity: Overview of the DOE, framing of the division of talent, labor and innovation, learnings from the first 3 years of this work, policy context, etc. (include successes/impact from the history of the work – data) Facilitation: Say: While the details of our new teacher evaluation and development system may seem like new information to some, the DOE has engaged in intensive work and piloting efforts in the past 3 years. The Teacher Effectiveness Program helped New York City prepare to implement NYSED Law 3012 c for the past 3 years. Passed in 2010, the law changed the requirements for teachers’ performance review The law is intended to foster clear understandings of excellent teaching practice, aligned to high standards for student achievement Four parts of the law Evaluation on a 4 point HEDI scale Portion of evaluation based on student growth (EMPHASIZE THIS MOSL IS LAW) Use of a rubric to measure teaching practice Provision of feedback to teachers, catered to need 5

All teachers will receive: Teachers in Grade 3-12 will receive:
Sixty percent of a teacher’s overall rating will be based on Measures of Teacher Practice (MOTP). All teachers will receive: Initial planning conference and summative end of year conference, including artifact review Choice between two observation approaches Written and/or verbal feedback and observation reports Student Feedback via Tripod Student Survey (no stakes pilot in ; worth 5 of 60 points beginning in ) Let’s look more closely at the Measures of Teacher Practice, which will make up 60% of a teacher’s overall rating. Facilitator describes the Measures of Teacher Practice, which account for 60% of a teacher’s evaluation rating. One on one initial and end of year conferences with their principal or assistant principal to support the evaluation and development process. Conferences are a valuable opportunity for teacher and school leader together to reflect on a teacher’s practice and how to make improvements to it to further support student learning. In addition, teachers will be able to choose how they want to receive their observations. For teachers in grades 3-12, we will also pilot the use of a student survey in and this survey will count for 5 of a teacher’s 60 measures of teacher practice points beginning in the following school year. Teachers in Grade 3-12 will receive:

All teachers will receive:
Forty percent of a teacher’s overall rating will be based on Measures of Student Learning (MOSL). All teachers will receive: Two different measures of student learning (40%) State Measure Local Measure Multiple measures provide a more valid, robust picture of teacher performance, providing teachers with multiple sources of feedback Facilitator briefly previews the 40% Measures of Student Learning rating, and indicates that this will be covered more extensively in the next session.

Other Measures of Teacher Practice (60%) Summative Evaluation Rating
Every teacher will receive an overall summative rating based on multiple measures of teacher effectiveness. Other Measures of Teacher Practice (60%) State Measure (20%) Local Measure (20%) Summative Evaluation Rating Example Highly Effective 60% 20% Highly Effective Overall Time:2 min Facilitation: Lisa Materials: none Overview of the activity: Facilitator summarizes the overall summative rating using the talking points listed below This next slide gives you a sense of how these scores are going to be combined. The [Measures of] Teacher Practice, the State Growth Score, and Locally Selected Measure will each have a point-value associated with them. They will sum up together to get an overall rating. You see in this example that the teacher was “Highly Effective” on each of those, and ended up at “Highly Effective” overall. Now, there is one element in the law I want to note, that’s important here and that’s gotten a lot of attention: the law specifically says that if a teacher is “Ineffective” on both their State Measure and their Locally Selected Measure, then they’re “Ineffective” overall. We’ve done a lot of analysis of the data, and it is a very rare occurrence, based on the projections that we’re seeing, that someone will be “Ineffective” in both of these and their principal has also rated them highly. It happens in less than half of 1% of the cases, so we don’t think this will affect many people, but it is an important rule for folks to be aware of.

Focus on the Forty Percent:
Transition slide: Now we will focus specifically on the 40% of a Teacher’s Advance rating that is based on student outcomes or Measures of Student Learning.

Advance MoSL – Implementation Steps
September - November Make and submit Selections Program students in STARS Train Teachers and Relevant Support Staff Inform teachers of State and Local Measures Administer, Norm & Score (when necessary) baseline assessments Complete Goal Setting Process (schoolwide and teacher level) December – March Complete data entry of teacher programs, assessment selections, and student-level goals (High Schools) Prepare for 2nd semester April – June 3-8 NYS ELA and Math Testing & Scoring NYSESLAT Testing & Scoring Grades 4 & 8 Science Performance & Written Testing and Scoring NYC Performance Assessment & 3rd Party Assessment Administration & Scoring (Teachers cannot score the exams of their own students) Summer / Fall 2013 State test scores, NYS Growth Scores, NYC Growth Scores released Finalization of teacher scores for Local and State Measures Teachers informed of overall performance ratings 2 min Review key dates. Draw attention to the “optional” nature of the 10/4 date and what this means for inclusion in goal-setting.

Upcoming Dates By 9/9 Principal and School Local Measure Committee makes selections. Principal enters selections into Advance Web Application Students must be programmed in STARS for first teacher-student-course data capture September (ongoing) Schools train teachers on any new protocols and/or assessments in place. Schools administer all baseline assessments, keeping in mind any new security and administration protocols. Schools norm and score NYC Performance Assessments. September 16 Principals receive Teacher-Level Selection Tool populated with 9/9 data. Principals correct data, where required. Schools begin printing answer sheets from ATS where teachers record NYC Performance Assessments scores for upload. September 24 Principals must inform teachers of State and Local Measures of Student Learning (will be available for print from Advance Web App) For GOAL SETTING, Principals and Teachers begin to set classroom, grade and school-level goals. October 10/3 – Data Capture for Version #2 of Teacher-:Level Selection Tool 10/4 – For teachers doing GOAL SETTING: enter results of baseline assessment (if school wants this data included in DOE-predicted targets). 10/15 – For GOAL SETTING ,School and grade goals due to Superintendent 10/31 – Deadline for all baseline assessment administration and submission. November 11/1 For GOAL SETTING: Goal setting worksheets with DOE-produced targets available for download from STARS. 11/15 For GOAL SETTING: Deadline for Principals to approve teachers’ classroom level goals

School Local Measures Committee - Role and Responsibilities
Before September 9: Principal and UFT chapter leader has each selected 4 members.* School Local Measures Committee uses six-step process (one-hour per step) to make recommendations for Local Measures: Discuss important introductory information. Review assessment options. Finalize assessment selection. Review growth measurements Finalize growth measurements. Present recommended approach to principal Facilitator Note: Principals are encouraged to be part of their committee. *Principals and UFT Chapter Chairs are encouraged to be part of their School Local Measures Committee. Overview Timeline Roles & Resp. Application

After September 9th - Roles and Responsibilities
Principal: If implementing goal-setting, work with teachers to adjust targets and approve goals. Discuss progress toward goals throughout the year. Meet with teachers for summative discussion of results. Teacher: Understand the measures that will be used and for which courses taught. Understand the assessment that must be administered and the timeline for doing so. Administer and, where necessary, score baselines. Norm on NYC Performance Assessment scoring rubrics. If goal-setting, adjust DOE targets and submit goals to principal. Overview Timeline Roles & Resp. Application

Key Considerations for School Local Measures Committees
The following are key considerations schools will need to keep in mind for the first year of implementation: The school community will select its Measures of Student Learning annually. Schools will be able to make different selections for subsequent school years. Schools should use the DOE-supplied tools to focus on critical questions: What assessments and target populations do we use? E.g., Do we want to use an existing formative assessment, such as running records for evaluative purposes, or would we rather use a school-wide measure? What will we do for growth measurements? While goal-setting gives teachers more control, it requires more time and is often less predictive of student growth than growth models. For both decisions, consider: How much time will this take? Do we want to invest this time in light of the other critical work in our school? How does this decision build upon or change current practice in our schools? Overview Timeline Roles & Resp. Application

 Assessment Options NYC Assessment Type Example (K-5)
Target Population Options* State Assessments 4-5 Math and ELA State Assessments Individual Grade School 3rd Party Assessments Performance Series NYC Performance Assessments New! (for release this year)  NYC * There are some exceptions to these options – see the Measures of Student Learning Guide for target population options by assessment. Make sure facilitator references additional bulleted rules on in guides (page 23 of elementary guide). For every assessment selected, there must also be a target population selected. Target Population refers to the students who are included in the measure for a given assessment.

Assessment List – Changes from 8/1
Added “ELA Performance Assessments” in K-8 now include an option between Running Records and writing-based Performance Assessments. 8th Grade Science NYC Performance Assessment SANDI/FAST Always included in D75 supplement Integrated Algebra SVMI New Visions schools only Additional clarification for allowable State and Local options AP Local Measure use only Discovery Math Limited use for K-2 only schools Changed 3 min - Review changes to the assessment list.

Assessment Options: State Assessments
 State Assessments: Measure the performance of students based on state-created assessments. Considerations: Can be used as a school-wide measure by most teachers in a school where a State Assessment is administered. If you select the same assessment for Local Measures and State Measures, the target population must be a different set of students. No new assessments or workflow needed (in some cases, a baseline test must be administered) M2: Assessments State 3rd Party Performance

 State Assessments include: Grades 3-8 ELA and Math State Assessment Grade 4 and Grade 8 State Science Assessment NYSESLAT (ESL) NYSAA in ELA and Math State Regents (English, Math, Science, Social Studies) Where a state assessment is administered by a teacher, it is required as the State Measures assessment (for common branch K-8, if both ELA and Math are taught, they both must be used for the State Measure*). * For schools that departmentalize, teachers are only responsible for the content they teach. M2: Assessments State 3rd Party Performance

 State assessments may be selected as the Local Measures for: Teachers who administer state assessments In this situation, it must target a different population of students (options: grade, school, or lowest third of performers in the class) Teachers of ELA in grades 4-8 must use a NYC Performance Assessment as one of the Local Measures, but this may be supplemented with other state assessment measures Teachers who do not administer state assessments Can use the state assessment as either a grade-level or school-wide measure. This will frequently be a good option for teachers: Of subjects without individual assessments Who teach courses which largely affect performance in a state-tested grade/subject M2: Assessments State 3rd Party Performance

Assessment Options: 3rd Party Assessments
3rd Party Assessments: Created by assessment experts; assessment format varies (multiple choice, performance based, etc.) Considerations: Only state-approved 3rd Party Assessments (in Guide) can be used. Not available for all grades and subjects. Include both baseline and end-of-year assessments. Additional administration procedures needed. Teachers (per law) cannot score their own end-of-year assessment results. M2: Assessments State 3rd Party Performance

Assessment Options: 3rd Party Assessments
Considerations for Use: Schools may choose to focus on 3rd Party Assessments that are already in use, rather than implementing new assessments. Some schools use 3rd Party Assessments for formative purposes throughout the year and may not want to alter their intent by using them for evaluation. Teachers may not score their own students’ end-of-year assessments, so additional procedures must be put in place to account for this. For teachers using GOAL SETTING, baseline assessments must be administered and scores uploaded by October 4th. All baselines must be administered by October 31st. M2: Assessments State 3rd Party Performance

3rd Party Assessments – Scantron Performance Series
Performance Series (Scantron) MOSL options: ELA: Grades 3, 9-11 Math: Grades 3 through Geometry Considerations: Administration/scoring will not change for MOSL, with the exception of a pre-determined window for pre-/post-test administration. Can also use for Periodic Assessments; beginning- and end-of- year administration will count toward MOSL. No additional baseline selection needed. Online, web-based administration; schools must have adequate bandwidth and hardware to administer. 2 min - (Still need confirmation) - If administering twice within testing window, first administration will count as baseline and last administration will count as final assessment.

3rd Party Assessments – Discovery K-2
Considerations: An option for Math in K-2 only schools. K-5 and K-8 schools should use school wide performance on the state math assessment for K-2 teachers’ State Measure. For K-2 only schools: May choose to use beginning-of-year Discovery administration for baseline along with growth model or goal-setting. For Kindergarten ONLY: May choose school-created baseline rather than administering Discovery pre-test in beginning-of- year (must use goal-setting in this instance). 2 min This is new information, so review each point carefully. Explain why this decision was made (K-2 only). Explain baseline implications and why it is different for Kindergarteners (don’t know how to take test) – can select portfolio of work even, does not have to be a traditional assessment, but in this case, must use goal-setting.

3rd Party Assessments: Turn & Talk
At your tables, there are examples of some of these 3rd Party Assessments. Take time to review the assessments most applicable to your school. While you do this, discuss: What third party assessments do we currently use and are we considering using for MoSL? What will be the implications be for the school if we a third party assessment? What work would we have to make sure is in place? Who will do this and when? 10 min Activity led by table facilitators Make sure participants know where at their tables to find the samples that are provided.

Assessment Options: NYC Performance Assessments
NYC Performance Assessments: Authentic tasks (e.g., evidence-based essays), scored against a common rubric. Created by the DOE, NYC teachers, and curriculum and assessment experts to be used as Measures of Student Learning in teacher evaluation. Considerations: Not available for all grades and subjects. Include both baseline and end of year assessments. Require additional time for training, scoring, and recording results. Teachers (per law) cannot score their own end-of-year assessment results. M2: Assessments State 3rd Party Performance

Assessment Options: NYC Performance Assessments
Background Created and piloted selected sample assessments in SY ’12-13. Goal: Design assessments that achieved guiding principles 80 schools participated NYC Performance Assessments: Required for the State Measures where they exist and no state assessment exists – Grades 6-8 Social Studies & 6-7 Science The ELA Performance Assessment is required as at least one of the Local Measures for teachers of ELA in grades 4-8. May be selected as baseline assessments for specific state assessment (e.g. 4th Gr. Science). For complete listing, see Guide. M2: Assessments State 3rd Party Performance

Performance Assessments – Overview
NYC Performance Assessments are available in multiple K-12 subjects, some as optional, others as required. These include baseline and end-of-year assessments (no additional selections for baseline needed). If the principal and/or committee chooses the default option, the mandated NYC Performance Assessments in grades 4-8 ELA for the Local Measure do not need to be administered as all teachers in the school will use the default measure. There are Performance Assessments required in some grades/subjects for the State Measure. Those assessments are not affected by the selection of the default option for the Local Measure. In most grades/subjects with Performance Assessments, there is one type of assessment. However, in K-8 ELA, committees have two options within the Performance Assessment category: Running Records NYC Performance Assessment (using one of three reading programs) Writing-based NYC Performance Assessment 3 min Review all points on slide Focus on the optional vs. mandatory nature of performance assessments and how choosing the default option gets rid of the 4-8 ELA Local Measure requirement (but not in other places where performance assessment mandated for State Requirement) Focus on the fact that there are two options for the K-8 ELA performance assessments – will review both today

Running Records (K-8) – Overview
Three programs may be used for MOSL Running Records. Running Record program options for MOSL include: Fountas and Pinnell (F&P) Developmental Reading Assessment (DRA2), or Teacher’s College (TCRWP) Text funding is included in Periodic Assessment funding for elementary schools. Schools must establish structures so that teachers do not score final assessments of students whose scores will factor into their evaluation rating. Text Security – Texts used for Running Record assessment purposes should be kept separate from other texts (i.e., not used for instructional purposes). 2 min - Review running record program options

Running Records: Implementation Steps
By 9/9 Principals and School Local Measure Committees work together to select between Running Records and writing-based Performance Assessments for ELA K-8. September Schools administer first assessment as baseline. Must consider options for scoring: by the classroom teacher (keeping in mind that the spring administration CANNOT be scored by the classroom teacher) or by someone other than the classroom teacher. October By 10/4 – Submit results of baseline assessment in order to receive DOE-predicted targets with this data included (optional). By 10/31 – All baseline assessments administered, scored, and submitted. 2 min - Review timeline. Note that this is the same timeline for all assessments, not just RRs.

Running Records: Turn and Talk
Thinking about Running Records: What are the implications if our school chooses Running Records? What new systems must be put in place in order to ensure testing security & compliance with mandate that teachers cannot score their own students’ end of year assessments? 10 min - Led by table facilitators

Performance Assessments: NYC Performance Assessments
Considerations: Review of where they exist on assessment list ELA (writing-based): K-12 Math: Grade 3, Integrated Algebra Social Studies: 6-8, Global History, U.S. History Science: 4, 6-8, Living Environment In K-8 ELA, also have choice of Running Records. All NYC Performance Assessments (except ELA) translated into 9 languages. Require norming on rubrics at the school level. Samples available for download from Advance Intranet Site (final versions downloaded from Advance Web Application by 9/9). 3 min - These were all on the old assessment list except 8th grade science, which was added in the new version - Need to remind people on this slide that we are now talking about the “other” type of performance assessment – this type exists across several grades and subjects, whereas the running records are just specific to ELA K-8

NYC Performance Assessments: Turn and Talk
Thinking about NYC Performance Assessments: How will baseline performance assessments be administered in September? How will scoring be normed across the school (to prepare for requirement that teachers cannot score their own end of year assessments)? What preparation, training, and time is required to administer and score these assessments reliably? 10 min - Led by table facilitators

Performance Assessments: Implementation Steps
By 9/9 Final performance assessments released online (samples by 8/15). School Local Measure Committees recommend whether or not to administer Performance Assessments where optional. Principal approves or chooses default. For ELA K-8, principals and committees decide between Running Records and a writing-based performance assessment. Principals plan necessary teacher support around administration and scoring. September Schools develop scoring protocols. Schools norm teachers on Performance Assessment rubrics to ensure a shared understanding of demands of the rubric and scoring across student work. Teachers administer Performance Assessments. October Schools score and scan Performance Assessments. Must submit scores by 10/4 for inclusion in DOE targets (otherwise by 10/31). 2 min - Review points on slide. Timeline isn’t different for performance assessments, but does include some additional work in terms of norming in September.

Baseline Assessments Where choices exist, schools may opt to use a DOE-recommended baseline assessment or a school-selected baseline. Remember: Baselines only need to be selected for a few state assessments (4th grade ELA, Math & Science) and AP exams.* All other 3rd party and Performance Assessments come with baseline assessments. Where baselines must be selected, the PRINCIPAL chooses. If the selected baseline is not on the DOE list, schools must use goal-setting. DOE targets provided to those schools will not include results from the baseline. For schools using goal-setting: By October 4: Administer and submit baseline scores if schools want data to be used in the DOE calculation of target suggestions. By November 15: Goal-setting approval deadline. Schools must administer baseline assessments in time to write and approve goals for this date. All schools must administer, score, and submit baseline assessments by October 31. 5 min Review information about baselines Draw attention to the fact that only principals can select baselines (there are very few instances where a committee would have to select a baseline, but where this comes up, principal needs to select – ex. If using 4th grade Science Assessment for Local but not State, or AP exams in HS) Draw attention to the fact that goal-setting MUST be the growth measurement if baseline is school selected. * Principals also have the option of selecting an alternative baseline assessment for Discovery Math in K only.

Growth Measurement Options
After choosing assessments and target populations, the next step is choosing one of two growth measurements for each assessment. Growth Models: DOE calculates student targets, results, and teachers’ scores. Results are shared after assessments have been administered so student growth can be compared to similar students’ performance on assessments. Goal-Setting: DOE provides targets for how students will perform on assessments that principals and teachers can adjust based on their knowledge of students. Principals approve targets. 5 minutes M3: Growth Growth Models Goal-Setting Practice

Growth Models: Overview
39 Growth models allow us to compare the progress that students make in a year to similar students. In the State Growth Model, similar students are defined by four student-level characteristics at the student and classroom levels: Academic history Economic disadvantage Disability status Language learner status Citywide models account for similar characteristics to the State model. Growth models control for the degree to which students are expected to make gains given their achievement history and demographic characteristics. 3 minutes M3: Growth Growth Models Goal-Setting Practice

Defining Characteristic of “Similar Students”
State Growth Model Category Defining Characteristic of “Similar Students” Academic History: Prior year test score, same subject Prior year test score, different subject Retained in grade New to school in year other than entry year Average prior achievement and range around average prior score in student’s class/course (same subject) Student with Disability (SWD) Student has an IEP SWD spends less than 40% of time in general education setting Percent SWD in student’s class English Language Learner (ELL) Student is an ELL NYSESLAT scores Percent ELL in student’s class Poverty Student poverty indicator (yes/no) Percent poverty in a student’s class The State Growth Model compares the progress that students make in a year to a group of similar students statewide. In the State Growth Model, similar students are defined by 13 characteristics divided into four categories at the student and classroom level. To be included, a student has to be enrolled for a minimum of 60% of the year. Student scores are weighted by the percent of time they were enrolled in the course and their attendance 5 minutes M3: Growth Growth Models Goal-Setting Practice

Growth Models: Considerations
41 Does not introduce additional work in schools. Enables schools to compare their students’ performance to similar students. Gives teachers credit for the degree to which students exceed predicted growth. Better able to account for unexpected outcomes resulting from unfamiliar, new assessments. Growth model score results are not available until after assessments have been administered (i.e., the following spring/summer). Does not actively facilitate brainstorming and discussions about individual student performance and goals. May be more difficult to explain to schools. Benefits Challenges 5 minutes M3: Growth Growth Models Goal-Setting Practice

State Growth Model: Scoring and Results
Most teachers scored Effective on the state growth score in The results for NYC were slightly higher than the rest of the state: NYC Teachers (n=10,544) Rest of State (n = 22,585) Highly Effective 8% 6% Effective 76% 77% Developing 10% Ineffective Note: SED has stated that it expects the future distribution of ratings to remain similar to what it was in 5 minutes

Citywide Growth Model In addition to the State Growth Model, which covers grades 4-8 ELA and Math at this time, the DOE is also producing a Citywide Growth Model. This is available as a growth measurement option for any state or local assessment not covered by the State Growth Model. Like the State Growth Model, the citywide model will control for multiple factors including academic history, English language learner status, disability status, and poverty. 3 minutes M3: Growth Growth Models Goal-Setting Practice

Citywide Growth Models Process
44 DOE Reports Results and Scores DOE Measures Students’ Growth Administer Baseline and End-of-Year Assessments Fall: Baseline assessments administered to students (not required for all assessments). Spring: End-of-year assessment administered to students Growth models measure students’ growth compared to similar students on the end-of-year assessment. Similar students are determined by baseline performance, academic history, and demographic characteristics. DOE provides student targets, results, and teachers’ scores to teachers and principals after assessments have been administered. 5 minutes M3: Growth Growth Models Goal-Setting Practice

Goal-Setting Process Administer baseline assessment
DOE sends predicted student targets Teachers review DOE predicted targets Principals approve or adjust targets Administer end of year assessment Teachers’ Ratings Baseline assessment administered (not required for all assessments). Report baseline assessment results. DOE sends predictions for how individual students will perform. Predictions are based on baseline performance, student achievement history, and student demographic characteristics. Teachers may choose to adjust these targets based on additional information about their students. Teachers submit student targets to principal. Principal (or designee) report finalized student targets. 10 minutes End-of-year assessment administered to students. Teachers’ HEDI ratings are calculated with a conversion chart based on students’ performance on outcome assessments relative to their targets. M3: Growth Growth Models Goal-Setting Practice

Goal-Setting Considerations
Particularly valuable for teachers/schools with unique student populations or high mobility. Allows teachers and principals to individually tailor student goals. Requires additional time/resources Teacher’s rating is based on the percentage of students who meet their target but does not consider the degree to which students fall below or exceed their target. Targets must be set early in the school year, possibly before much diagnostic info is gathered Setting goals may be challenging if: Teachers are not familiar with the comparability between assessments Assessments are new or changing Benefits Challenges 5 minutes M3: Growth Growth Models Goal-Setting Practice

Goal-Setting Considerations
This is not the same as the goal-setting you may typically see in schools. Goals are scored against a state conversion chart which makes the target-setting process difficult and non-intuitive. Before considering goal-setting, make sure you understand the work and additional training this entails. M3: Growth Growth Models Goal-Setting Practice

Goal-Setting – Who Sets Goals
Teachers set goals for students in their classroom (target population: individual), but when goals go beyond an individual teacher’s students (target population: grade or school), the principal sets the goal and the superintendent approves. Target Population Teachers Principals Super. Individual Level By student; or By subgroups of students; or Whole class goal Set Approve* Grade Level Whole grade goal Approve School Level Whole school goal 2 minutes - Several goal types by target population, will go into detail in a minute. Most important thing here is the set vs. approve for different target populations. For an “individual” target population teacher sets, discusses and adjusts with principal, principal approves. However, and this is new information, for school and grade target populations, the principal sets the goals and the superintendent approves. * If a teacher fails to submit goals, the principal will set the goals for that teacher. The principal should not simply “approve,” but rather work with the teacher on adjusting targets as necessary.

All targets must be set as ranges.
Goal-Setting – Ranges All targets must be set as ranges. Example #1: A teacher creates goals for each student within a class: Johnny will score between 50 and 75. Jane will score between 60 and 80. Jose will score between 55 and 80. (and so forth…) Example #2: A teacher creates goals for a class of students: All students (i.e., each student) will score between 60 and 85. Note: Given how the SED conversion chart works, this is the same as setting a target such as “Johnny will score 50 or higher.” 3 min. - Key points: goals must be set as ranges, not single numbers, not percentages. Ultimately, it is the lower number that counts when deciding if a student “hit” a target per the state scoring chart.

Goal-Setting – Types of Targets
Teachers (with principal approval) can set three types of targets for individual-level MOSL goal-setting* Classroom Level Example: “All students will score between 50 and 75.” Note: Teachers cannot set targets in the format “85% of students will score 75% or higher.” Subgroup Level Example: “All students with 8th grade scores of 1.50 to 2.49 will score 55 to 80; all students with 8th grade scores of 2.50 to 3.49 will score 65 to 85, etc.” Note: Only subgroups based on incoming performance are allowed per SED rules. Student Level Example: “Johnny will score between 55 and 75.” 3 minutes - Discuss the three types of goals teachers can set for students in their classroom. *This includes “Individual – Lowest Third” targets, which must be the same target as is set for the broader target population.

Goal-Setting – Types of Targets
Principals (with superintendent approval) can set two types of targets for grade or school target populations. Whole Group Level (grade or school target population) Example: “All students in school/grade will score between 50 and 75.” Subgroup Level (grade or school target population) Example: “All students with 8th grade scores of 1.50 to 2.49 will score 55 to 80.” Note: These goals are set on a slightly different timeline. Principals should use reference tables released on 9/24 in order to draft and submit targets to superintendents by 10/15 for a final approval date of 11/15. 3 minutes Discuss the two types of goals that principals can set for grade or school level goals. Note: Individual student targets are not an option. Also note the difference in timeline for approval Principals should follow DOE suggested target ranges for subgroup and whole school and not adjust too much – this will have implications on what may be approved at the superintendent level (clarify wording of this with ORD).

From Goals to Ratings: Scoring
Teachers’ MOSL subcomponent scores are calculated from the chart below using the following process: Calculate the percent of students who met or exceeded their target. Use the following table to determine the teacher’s HEDI points and rating: Note: 60% of a teacher’s students need to meet or exceed their target to earn a rating above “Ineffective.” If a teacher sets goals above the DOE-predicted student target or the assessment is more difficult than expected, the teacher may be at risk of receiving a low rating. HEDI Conversion Chart % of students meeting or exceeding target Highly Effective Effective Developing Ineffective HEDI Pts 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 % at Target 100-97 96-93 92-90 89-85 84-80 79-75 74-67 66-60 59-55 54-49 48-44 43-39 38-34 33-29 28-25 24-21 20-17 16-13 12-9 8-5 4-0 3 min This is the scoring chart for the MOSL subcomponent of goal setting. Once scores are returned, the percentage of students hitting the target is calculated and then mapped to this chart to determine rating for this subcomponent of the overall Advance rating. As you can see here, at least 60% of students must meet their target in order to get above an Ineffective on the SED scoring chart.

DOE Target Suggestions
Target suggestions will be provided by the DOE. DOE target suggestions for school and grade target population (for principal use in goal-setting): By September 24 – Reference tables available. Tables with target ranges by assessment administered in each school. Tables with subgroup target ranges by assessment. By October 15 – Principals submit targets to superintendents for review. DOE target suggestions for individual classroom target population (for teacher use in goal-setting): By November 1 – Student-level targets available that reflect any baseline data submitted by October 4. Teachers may also use reference tables mentioned above prior to receiving student-level targets in order to draft goals prior to November 1. All goals must be finalized by November 15. 3 min Review differences in timeline and resources for different levels of goals. Explain that they will see an example of a reference table in the upcoming activity.

Goal-Setting: Fall Implementation Steps
By 9/9 School Local Measure Committees recommend whether or not to use goal-setting for Local Measure. Principals decide the same for State Measure (where the option exists). September Schools who choose goal-setting norm on expectations across classrooms. Schools administer baseline assessments. 9/24 – DOE target reference tables and goal-setting worksheets released on Intranet. October By 10/4 – Schools submit results of baseline assessments if they want data included in DOE-suggested targets (otherwise submit by 10/31). By 10/15 – Principals submit grade/school level goals to superintendent for review. Teachers start process of goal-setting using baseline data and reference tables. November 11/1 – Schools can access goal-setting worksheets which include baseline results and DOE-suggested targets. 11/15 – Principals finalize teacher-set goals. Superintendents finalize school/grade goals. 3 min Remind participants that apart from these dates, should consider additional time needed to norm on target setting, gather background data for adjusting goals, printing forms, completing targets, approving targets, etc.

Turn and Talk: Goal-Setting Implications
Given this activity, what support do you think schools will need when setting targets? Resources? Help interpreting reference tables? Help collecting classroom-based evidence? Help adjusting targets? How can you help principals to think through what it means to set grade or school level targets and work with teachers to adjust targets for classroom goals? How will you provide this level of support (coaching sessions, PD, etc.)? 10 min - With your tables, reflect on this activity and the supports you can provide schools in your network to help with this process.

Default Measures If a committee cannot reach decisions about Local Measures, or the principal does not approve the committee’s recommendation, the default option will be put in place. Grade Local Measures All Teachers in a Building with Grades 4-8 Assessment: State ELA & Math Assessments (School) Measurement: Growth Model (SED) All Teachers in a Building without Grades 4-8 Assessment: School-wide: All assessments at the school for State Measures Measurement: Student average of school growth measures

Immediate Next Steps By September 9 principals must submit decisions in the Web Application regarding measures for teachers in their schools. Before September 9: Principals must select assessments (and, where applicable, baseline assessments), target population, and growth measurements for State Measures for grades/subjects without required measures.* School Local Measures Committees must select assessments, target population, and growth measurements for Local Measures for all grades/subjects. School Local Measures Committees must present their recommendation to the principal (if not on the committee) for approval. Principals must either approve the School Local Measures Committee recommendation or choose the default option. * In some cases, an assessment may be required, but the principal will select a baseline assessment and the growth measurement. Overview Timeline Roles & Resp. Application

NYCDOE’s Advance: Measures of Student Learning

Similar presentations

Presentation on theme: "NYCDOE’s Advance: Measures of Student Learning"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

NYCDOE’s Advance: Measures of Student Learning

Similar presentations

Presentation on theme: "NYCDOE’s Advance: Measures of Student Learning"— Presentation transcript:

Similar presentations

About project

Feedback