Welcome! Teacher Evaluator Training Session 5 Please sit in a seat where there is a folder.

On this day… in 2009…we had already administered 3-8 ELA and Math tests in 2010…ipads didn’t exist in 2011…most of us had never heard of SLO’s Last month…SLO’s disappeared Last week…it was 82 degrees

Since we last met… What have you been up to related to APPR? Round Robin, beginning with the person with the longest title. 5 minutes to share.

Draw a House Roof 20 points Chimney 5 points When appraising your house, score yourself accordingly: Roof 20 points Chimney 5 points Smoke coming out of chimney 5 points Windows 5 points each (maximum 4) Door 10 points Garage 30 points Trees 50 points Patio 60 points Pool 100 points

Outcomes for today Advance our common language around SLOs Examine critically SLO components, their integration, and their relationship to teacher practice Gain direct experience with crafting SLOs Understand the connections between teacher observations and student learning objectives

Self-check Based on what you know about Student Learning Objectives, complete the self-check to provide a BASELINE for where we are with SLO’s. What other scenarios do you need clarified?

100-Point Evaluation System 60 Other Measures of Effectiveness 20 Growth Local 100 Points 100-Point Evaluation System SLO’s

SLO Review: State 20% Three types of teachers: Teacher 1: Those who have a State provided growth measure and are not required to have an SLO. (4-8 ELA and Math Teachers) Teacher 2: Those who have a State provided growth measure, and yet, are required to have an SLO because less than 50% of their students are covered by the State provided growth measure. Teacher 3: Those who are required to have an SLO and do not have a State provided growth measure.

SLO Framework All SLOs MUST include the following basic components: Student Population Which students are being addressed? Learning Content What is being taught? CCSS/National/State standards? Will this goal apply to all standards applicable to a course or just to specific priority standards? Interval of Instructional Time What is the instructional period covered (if not a year, rationale for semester/quarter/etc)? Evidence What assessment(s) or student work product(s) will be used to measure this goal? Baseline What is the starting level of learning for students covered by this SLO? Target(s) What is the expected outcome (target) by the end of the instructional period? HEDI Criteria How will evaluators determine what range of student performance “meets” the goal (effective) versus “well-below” (ineffective) , “below” (developing), and “well-above” (highly effective)? Rationale Why choose this learning content, evidence and target?

NYSED SLO Framework A student learning objective is an academic goal for a teacher’s students that is set at the start of a course. It represents the most important learning for the year (or, semester, where applicable). It must be specific and measurable, based on available prior student learning data, and aligned to Common Core, State, or national standards, as well as any other school and district priorities. Teachers’ scores are based upon the degree to which their goals were attained.

Learning Content This is the content to be taught in the SLO. Identifies the course name and source of standards (Common Core, national, state, local) associated with this SLO Specifies the exact standards, performance indicators, etc., that will be taught, learned, and assessed. But first, what is the most important learning for the course?

Learning Content - Application As a team: discuss and select a particular course identify the most important learning for the course identify the course’s source of standards (Common Core Learning Standards for ELA and Literacy) list the exact items (standards, indicators, etc.) to be taught, learned, and assessed.

Learning Content - For instance… Course: Grade 9 ELA Most Important Learning: Read and understand developmentally appropriate text. Construct evidence-based arguments. Standards: P-12 CCLS for ELA and Literacy Reading Anchor Standard #10: Read and comprehend complex literary and informational texts independently and proficiently. Writing Anchor Standard #1: Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence.

Target(s) and HEDI scoring Learning Content Population Three sections of ELA 9, heterogeneously grouped, 70 students. Learning Content Read and comprehend complex literary and information texts independently and proficiently. Write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence. Interval Evidence Baseline Target(s) and HEDI scoring 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Rationale CCLS Anchor Standards

Learning Content - Application As a team: discuss and select a particular course identify the most important learning for the course identify the course’s source of standards (Common Core Learning Standards for ELA and Literacy) list the exact items (standards, indicators, etc.) to be taught, learned, and assessed.

Learning Content – Considerations To what extent is the content central to this and future coursework, as well as college and career readiness? Approximately what percentage of the course’s standards is selected? What are the most critical aspects for the development of these students in this content this year? To what extent is the learning content specific and measurable?

Evidence These are the assessments used for determining students’ levels of learning. List the specific pre-assessment(s) and summative assessment(s) that will be used for providing baseline and summative data for the SLO. What assessments would accurately measure a student’s ability to meet the standards that you have identified?

Evidence State-determined RULES: Assessments must be either: From a state-approved 3rd party vendor District, regional, or BOCES developed (Teacher created is not an option.) Districts and BOCES must verify comparability and rigor for any assessments they develop. Teachers who have a vested interest in the results cannot score the assessments

Evidence – Application As a team: Discuss and identify what pre-assessment or diagnostic data you would gather and analyze at the beginning of the course Discuss and identify what summative assessments would best measure the selected learning content for the course and standards/performance indicators you selected.

Evidence-For instance… Population Three sections of ELA 9, heterogeneously grouped, 70 students. Learning Content Read and comprehend complex literary and information texts independently and proficiently. Write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence. Interval 2012-2013 school year. Evidence Baseline assessment: 8th Grade ELA results. Common writing prompt: Students provide an objective summary of Frederick Douglass’s Narrative. They analyze how the central idea regarding the evils of slavery is conveyed through supporting ideas and developed over the course of the text. Summative assessment: Ten reading comprehension questions based on the selection rom Things Fall Apart. Ten reading comprehension questions based on Quindlen, Anna. “A Quilt of a Country.” Newsweek September 27, 2001. Students determine the purpose and point of view in Martin Luther King, Jr.’s, “I Have a Dream” speech and analyze how King uses rhetoric to advance his position (in writing). Baseline Target(s) and HEDI scoring 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Rationale CCLS Anchor Standards Appendix B

Evidence – Application As a team: Discuss and identify what pre-assessment or diagnostic data you would gather and analyze at the beginning of the course Discuss and identify what summative assessments would best measure the selected learning content for the course and standards/performance indicators you selected.

Evidence – Considerations How aligned and authentic are the assessment items to the learning content? How valid and reliable are the assessments? Are the selected assessments from an approved list of allowable options? Are they verified as comparable and rigorous? What, if any, administration accommodations must legally be made for students? Have procedures been established to ensure those with vested interest do not score students’ assessments?

Student Population and Baseline These are the students included in the SLO. Specifies your assigned students who are included in this SLO along with the course sections and student names and/or identification numbers. Baseline This is the level of students’ knowledge and skill in the targeted learning content at the beginning of the interval of instructional time. Describes how students performed on the identified pre-assessment(s) for the learning content.

Student Population and Baseline As a team: For your selected course, identify a sample student population. Generate a sample baseline for the students in your sample student population.

Student Population and Baseline Three sections of ELA 9, heterogeneously grouped, 70 students. Learning Content Read and comprehend complex literary and information texts independently and proficiently. Write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence. Interval 2012-2013 school year. Evidence Baseline assessment: 8th Grade ELA results. Common writing prompt: Students provide an objective summary of Frederick Douglass’s Narrative. They analyze how the central idea regarding the evils of slavery is conveyed through supporting ideas and developed over the course of the text. Summative assessment: Ten reading comprehension questions based on the selection rom Things Fall Apart. Ten reading comprehension questions based on Quindlen, Anna. “A Quilt of a Country.” Newsweek September 27, 2001. Students determine the purpose and point of view in Martin Luther King, Jr.’s, “I Have a Dream” speech and analyze how King uses rhetoric to advance his position (in writing). Baseline On last year’s ELA 8: 4% scored 1; 18% scored 2; 67% scored 3, 11% scored 4. On the four-point district-wide writing rubric: 15% scored 1; 40% scored 2; 30% scored 3, 15% scored 4. Target(s) and HEDI scoring 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Rationale

Student Population and Baseline Considerations Population considerations: Within and across classes, what are students’ current skill and knowledge levels which may impact the target and the learning content. Within and across classes, how many and what types of identifiable subgroups exist? How will instruction be differentiated to meet specific student learning needs? Baseline considerations: In addition to the pre-assessment(s), what other data sources were reviewed to confirm student learning needs? How strongly do the pre-assessment(s) and other data analyses indicate the need for the learning content?

Target Target(s) This is the level of knowledge and skill that students are expected to achieve at the end point of the interval of instructional time. Define numerical growth goals for student performance on identified summative assessment(s) which measure student knowledge and skill in the learning content. These data will be reviewed by the teacher and Evaluator at the conclusion of the course.

Target(s) Training SLO Target Approach 1: Set a common growth target. 90% of students, including special populations, will grow by 60 percentage points or more on their summative assessment compared to their pre-test for the standards. (e.g., Student E’s target is 60 more than 30, or 90.) Student Pre-Test Score Summative Target Student A 10 70 Student B 20 80 Student C 5 65 Student D 60 Student E 30 90 Student F

Target(s) Training SLO Target Approach 2: Set a growth to mastery target. 85% of students, including special populations, will grow to score 75% or higher on the summative assessment for the selected standards. Student Pre-Test Score Summative Target Student A 10 75 Student B 20 Student C 5 Student D Student E 30 Student F

Target(s) Training SLO Target Approach 3: Set differentiated growth targets by student. 85% of students, including special populations, will meet or exceed their individualized target. Student Pre-Test Score Summative Target Student A 10 80 Student B 20 Student C 5 75 Student D 70 Student E 30 85 Student F

Target(s)-Application As a team: Decide what type of approach you are using to set growth targets for your SLO. Generate a sample target for the students in your sample student population.

Target(s)-For Instance… Population Three sections of ELA 9, heterogeneously grouped, 70 students. Learning Content Read and comprehend complex literary and information texts independently and proficiently. Write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence. Interval 2012-2013 school year. Evidence Baseline assessment: 8th Grade ELA results. Common writing prompt: Students provide an objective summary of Frederick Douglass’s Narrative. They analyze how the central idea regarding the evils of slavery is conveyed through supporting ideas and developed over the course of the text. Summative assessment: Ten reading comprehension questions based on the selection rom Things Fall Apart. Ten reading comprehension questions based on Quindlen, Anna. “A Quilt of a Country.” Newsweek September 27, 2001. Students determine the purpose and point of view in Martin Luther King, Jr.’s, “I Have a Dream” speech and analyze how King uses rhetoric to advance his position (in writing). Baseline On last year’s ELA 8: 4% scored 1; 18% scored 2; 67% scored 3, 11% scored 4. On the four-point district-wide writing rubric: 15% scored 1; 40% scored 2; 30% scored 3, 15% scored 4. Target(s) and HEDI scoring Eighty percent of all students will score 55 points or higher on the summative assessment (out of a possible 64 points). 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Rationale

Target Considerations Is the target rigorous enough to ensure that all students are on track to achieve college and career readiness? To what extent does the target align with your standards?

HEDI This is how different levels of student growth will translate into one of four rating categories: Highly effective Effective Developing Ineffective Provides specific measures of student learning for each rating category. 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1

HEDI – Application As a team: For your selected course and target, design a sample HEDI structure.

Target(s) and HEDI scoring Population Three sections of ELA 9, heterogeneously grouped, 70 students. Learning Content Read and comprehend complex literary and information texts independently and proficiently. Write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence. Interval 2012-2013 school year. Evidence Baseline assessment: 8th Grade ELA results. Common writing prompt: Students provide an objective summary of Frederick Douglass’s Narrative. They analyze how the central idea regarding the evils of slavery is conveyed through supporting ideas and developed over the course of the text. Summative assessment: Ten reading comprehension questions based on the selection rom Things Fall Apart. Ten reading comprehension questions based on Quindlen, Anna. “A Quilt of a Country.” Newsweek September 27, 2001. Students determine the purpose and point of view in Martin Luther King, Jr.’s, “I Have a Dream” speech and analyze how King uses rhetoric to advance his position (in writing). Baseline On last year’s ELA 8: 4% scored 1; 18% scored 2; 67% scored 3, 11% scored 4. On the four-point district-wide writing rubric: 15% scored 1; 40% scored 2; 30% scored 3, 15% scored 4. Target(s) and HEDI scoring Eighty percent of all students will score 55 points or higher on the summative assessment (out of a possible 64 points). 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 99-100% 97-98% 96-96% 92-94% 88-91% 85-87% 82-84% 79-81% 76-78% 73-75% 71-72% 68-70% 64-67% 60-63% 57-59% 53-56% 49-52% 45-48% 40-44% 30-39% <30% Rationale

HEDI – Considerations How high are expectations for all students? How clear are the distinctions between each rating category? Within a HEDI rating category, how clear and objective is the point allocation? What is the rationale that is used for the differentiations in points?

Interval of Instructional Time and Rationale This is the timeframe within which the learning content will be taught. (This is generally one academic year unless the course is set as a semester, quarter, etc.) Rationale This describes the reasoning behind the choices regarding learning content, evidence, and target.

Interval and Rationale – Application As a team: For your selected course, determine the interval and craft the rationale, reflecting on the considerations raised throughout our session.

Interval and Rationale Population Three sections of ELA 9, heterogeneously grouped, 70 students. Learning Content Read and comprehend complex literary and information texts independently and proficiently. Write arguments to support claims in an analysis of substantive topics or texts using valid reasoning and relevant and sufficient evidence. Interval 2012-2013 school year. Evidence Baseline assessment: 8th Grade ELA results. Common writing prompt: Students provide an objective summary of Frederick Douglass’s Narrative. They analyze how the central idea regarding the evils of slavery is conveyed through supporting ideas and developed over the course of the text. Summative assessment: Ten reading comprehension questions based on the selection rom Things Fall Apart. Ten reading comprehension questions based on Quindlen, Anna. “A Quilt of a Country.” Newsweek September 27, 2001. Students determine the purpose and point of view in Martin Luther King, Jr.’s, “I Have a Dream” speech and analyze how King uses rhetoric to advance his position (in writing). Baseline On last year’s ELA 8: 4% scored 1; 18% scored 2; 67% scored 3, 11% scored 4. On the four-point district-wide writing rubric: 15% scored 1; 40% scored 2; 30% scored 3, 15% scored 4. Target(s) and HEDI scoring Eighty percent of all students will score 55 points or higher on the summative assessment (out of a possible 64 points). 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 99-100% 97-98% 96-96% 92-94% 88-91% 85-87% 82-84% 79-81% 76-78% 73-75% 71-72% 68-70% 64-67% 60-63% 57-59% 53-56% 49-52% 45-48% 40-44% 30-39% <30% Rationale The Learning Content is based on the most important CCLS anchor standards. The baseline evidence combines state test scores with an on-demand assessment taken from the 8th grade performance tasks in Appendix B. Similarly, the summative assessment is based on the performance tasks for 9th grade in Appendix B. The summative score is calculated by adding twice of the number of comprehension questions answered correctly with the total score on the district-wide writing rubric (which has 6 elements on a 1-2-3-4 scale which translates to a maximum 24 points).

Discussing District Decisions Decision 5: Determine district-wide processes for setting, reviewing, and assessing SLOs in schools. Given the sample SLO you and your colleagues have developed, review the element selections made and why they were selected. Discuss at what level these decisions should be made for SLOs, and how this work will begin to operate within the district framework. What collaboration or training would be necessary to inform those decisions?

The Evaluator’s Role Work with teachers in setting SLO(s) Approve SLO Ensure security and integrity of assessments Monitor and assess SLO results Award points based on results of SLO

Thank you for your feedback. What’s next? Assessing the quality of SLO’s Implementation of SLO’s Ratcheting up the accountability for objective evidence collection and alignment to the rubric. Summer offerings will be forthcoming. Thank you for your feedback.