REVIEWING AND EVALUATING STUDENT LEARNING OBJECTIVES CURRICULUM, INSTRUCTION, TECHNOLOGY OCTOBER 12, 2012 Everything an Administrator Needs to Know About Student Learning Objectives Created by Jane Boyd, Program Coordinator, Nassau BOCES Student Support Services

1.Assess and identify priorities and academic needs. 2.Identify who will have State-provided growth measures and who must have SLOs as “comparable growth measures.” 3.Determine District rules for how specific SLOs will get set. 4.Establish expectations for scoring SLOs and for determining teacher ratings for the growth component. 5.Determine District-wide processes for setting, reviewing, and assessing SLOs in schools. Components of a SLO: 5 District Decisions From NY DOE engageny.org Introduction to SLO Deck

Translating SLO’s to the HEDI Scale This table illustrates the relationship between the Growth or Comparable Measure component to the Overall Composite Score.

Components of a SLO: NYSED SLO Framework All SLOs MUST include the following eight basic components: Student PopulationWhich students are being addressed? Learning Content What is being taught? CCSS/National/State standards? Will this goal apply to all standards applicable to a course or just to specific priority standards? Interval of Instructional Time What is the instructional period covered (if not a year, rationale for semester/quarter/etc)? Evidence What assessment(s) or student work product(s) will be used to measure this goal? Baseline What is the starting level of learning for students covered by this SLO? Target(s) What is the expected outcome (target) by the end of the instructional period? HEDI Criteria How will evaluators determine what range of student performance “meets” the goal (effective) versus “well-below” (ineffective), “below” (developing), and “well-above” (highly effective)? RationaleWhy choose this learning content, evidence and target? 4 From NY DOE engageny.org Introduction to SLO Deck

S tudent L earning O bjective: SLO Academic Goal Set at the start of the course Represents the most important learning Specific and measurable Based on prior student learning data Aligned to standards Aligned to school or district priorities Guidance on NYS Dist. Goal-Setting Process: SLO. March 2012 Definition:

S tudent L earning O bjective: SLO Today!  Where you are and where you are going  What are the criteria for each aspect of the SLO  How can you manage this process  What are your concerns

District rules for how specific SLOs will get set Will the District require the use of existing, common District-wide assessments for a specific grade/subject? Are there grades/subjects where the District wants to prioritize building or buying additional District-wide assessments? Are there groups of teachers where group or team results based on state assessments are appropriate? Are there grades or subjects where the District can identify priority learning standards or other District-wide guidance for schools and teachers? What will the District require for any remaining teachers not covered by the above?

District Expectations for Scoring For each group of grade/subject teachers with similar growth goal approaches, Districts must determine and communicate the District’s expectations for student learning growth relative to baselines and specify how teachers will be awarded HEDI ratings and earn from 0-20 points based on the results obtained, consistent with State Regulations and guidance.

District-wide Processes What tools will principals use to assess the rigor of teacher- determined targets? Will Districts review all goals or spot check goals and targets set by schools and teachers in any way to ensure rigor and comparability? How will procedures to monitor progress of students on SLO targets and final results reviews be handled? Districts may want to consider other teacher evaluation procedures like evaluator/teacher conferences, and procedures around classroom observations as well as District processes for data-driven inquiry meetings to be sure that sufficient time and coordination is provided.

Room for Decision Making D23, p.40. In some cases, district and BOCES leaders may develop entire SLOs (or just targets and/or HEDI expectations) for all teachers or certain subject area teachers, and in other instances, district and BOCES leaders may leave the development of the SLO (or just targets and/or HEDI expectations) to principals and/or principals and teachers. In some cases, the district/BOCES may decide that all SLOs must use the same target expectations based on growth progress norms while in other instances the district/BOCES may allow principals and/or teachers to propose a target based on the starting level of the students covered within the SLO. Decisions as to whether all teachers of the same subject will use a common SLO (or whether principals may even dictate this) are allowable; however, these are decisions first to be made by district/BOCES leaders.

S tudent L earning O bjective: SLO When setting SLO’s by course or grade:  Learning content, Interval of Instructional Time, Evidence, Targets, HEDI Criteria are all the same.  Population, Baseline data, Differentiation of targets based upon baseline data, and rationale will be unique to the roster.

S tudent L earning O bjective: SLO Different district approaches for setting SLO’s: 1. Director’s and Chairpersons create the SLO’s and assessments with Central Office guidance and disseminate them to faculty 2. Director’s and Chairpersons train and over see the creation of SLO’s and assessments within departments or across grade levels with Central Office guidance 3. Teacher’s create SLO’s using district decisions and SLO’s are evaluated by Supervisors – principals, chairpersons, etc. In all situations, there comes a point where a review needs to take place.

S tudent L earning O bjective: SLO Director’s and Chairpersons create the SLO’s and assessments and disseminate them to faculty  May select Learning Content – standards  Make decisions regarding evidence:  Create baseline assessments or make decisions regarding data to be used as the baseline  Create the summative assessments  Apply district decisions for target setting and HEDI scores Teachers can:  Teacher’s describe their population  Analyze the data and record that analysis in the “Baseline”  Add any information to “Rationale”

S tudent L earning O bjective: SLO Director’s and Chairpersons train and oversee the creation of SLO’s and assessments within departments or across grade levels  Highly collaborative within the department or grade level  Director’s/Chairpersons facilitate as teachers contribute to conversations regarding learning content for courses/grades  Director’s/Chairpersons create a format for the creation of assessments or use of baseline data  Data is analyzed by teachers and discussed with Director’s/Chairpersons to create appropriate targets.

S tudent L earning O bjective: SLO Teacher’s create SLO’s using district decisions and SLO’s are evaluated by Supervisors – principals, chairpersons, etc.  Teachers receive training and create SLO’s for their courses using district decisions regarding any learning content, targets and the HEDI scoring.  Appropriate assessments are a collaborative effort and finalized by a supervisor  Data is gathered and the SLO is written with the idea of recommending appropriate targets, etc. with the given district guidelines.

S tudent L earning O bjective: SLO Who needs an SLO?  Right now, it is anyone that is not receiving a state-provided growth measure which is 4-8 ELA and Math. See “Purple Memo” and “Assessment Options for SLO’s” More than one SLO? Maybe How do you know what you need?  50% or more of your students must be covered under your SLO’s. That might mean you need two or maybe even three.  Begin with the courses taught that have the largest number of students, combining sections with common assessments.* *p10 & 12 SLO Guidance March

Minimum Group Size Who needs an SLO? Another situation -  When the minimum number of students is not met. Other situations?  Part-time teachers? § – Does not discuss part time classroom teaching service.  Long term subs?  Librarians – Speech?  Second or third SLO just in case my population changes

Who Needs an SLO? D43. How are SLOs for Library/Media Specialists established if these teachers do not have regular classes scheduled and only schedule on-demand/teacher-requested basis for specific topics and projects? Districts/BOCES will need to determine their specific rules around which courses must have SLOs when contact time varies following the State’s rules and the general principle of including the courses with the most students first and making practical judgments about how to consider different course meeting schedules like those in this example.

Classroom Teaching Service Teacher of Record New Guidance - B12. When must school librarians be evaluated under Education Law § 3012-c? "Classroom teacher" is defined as a teacher in the classroom teaching service who is a teacher of record. Librarians who are certified as a library media specialist or school media specialist (library) are teachers in the classroom teaching service. For the school year, teacher of record is defined as a teacher who is primarily and directly responsible for a student’s learning activities that are aligned to the performance measures of a course, consistent with guidance. Therefore, a certified librarian who is not a teacher of record is not a "classroom teacher“ and therefore would not need to be evaluated under Education Law § 3012-c. However, if a certified librarian is a teacher of record, he/she would be considered a "classroom teacher" and therefore must be evaluated under Education Law § 3012-c.

Is a Librarian a Classroom Teacher? Certified Library Media Specialist or School Media Specialist Teacher of Record APPR and SLO’s Teachers in Classroom Teaching Service Not a Teacher of Record Other evaluation processes A teacher who is primarily and directly responsible for a student’s learning activities that are aligned to the performance measures of a course consistent with guidance M2. What kinds of data will districts and BOCES need to collect in order to determine who is the teacher of record for evaluation purposes? Districts and BOCES will need to collect additional data elements to support teacher of record determinations. (Teacher-Student Data Linkage)

Speech Teacher – Addendum August APPR B10. If a speech teacher (i.e., certified teacher of the speech and hearing handicapped or certified teacher of the speech and language disabilities) is just providing related services, is he/she required to be evaluated under Education Law §3012-c? No. If a certified speech and language therapist is providing related services only, as defined in Education Law §4401 and section 200.1(qq) of the Commissioner's Regulations, he/she would not be considered a classroom teacher and would therefore not be required to be evaluated under Education Law §3012-c. B11. If I am certified as a teacher of the speech and hearing handicapped and provide instructional services, am I required to be evaluated under Education Law §3012-c? If a teacher who is certified as a teacher of the speech and hearing handicapped is assigned to provide instructional services and he/she is a teacher of record, then he/she is considered a classroom teacher and therefore must be evaluated under Education Law §3012-c.

S tudent L earning O bjective: SLO Student Population: Students included in the SLO  There is no minimum number of students  The population is set on BEDS day  Even if the class enrollment changes it is not abandoned. D27D27  District set date for adding students  Addresses the growth of “each student” (A ROSTER IS REQUIRED) but then it is the aggregate growth of all the students that determines whether or not the target is met.  Other student information to describe the population may be added (SLO Development Guide) P37, Section D. APPR Guidance April 2012

S tudent L earning O bjective: SLO Population examples  Health and Occupations Technology 11 th Grade; 24 students, including 2 with learning disabilities on attached roster  18 Kindergarten Students listed on the attached roster  23 8 th grade Math AIS students from 3 sections of the course. See roster  All ___ students enrolled in 2 sections of AP Calculus. See attached roster  All ___ students in 10 th Grade English. See attached rosters which includes course sections, student names, and ID numbers  11 th and 12 th grade students enrolled in Fitness for Life, see attached roster which includes students with disabilities Population* These are the students assigned to the course section(s) in this SLO - all students who are assigned to the course section(s) must be included in the SLO. (Full class rosters of all students must be provided for all included course sections.)

Population

S tudent L earning O bjective: SLO Learning Content: Content to be taught in the SLO  Identify the source for the standards: Common Core, State, National, College Board, Industry, AASL Standards, etc.  Include any relevant Common Core Literacy/Math standards with NYS standards (Literacy in History/SS or Science & Technical subjects) – might include district decisions  Name the exact standards and performance indicators  As appropriate, choose the “most important learning” or a “subset of power standards.” What do you traditionally assess and does it give you a picture of the year?  Align to district or school goals and vertically align SLO Development Checklist OCMBOCES *SLO Template Learning Content* What is being taught over the instructional period covered? Standards? Will this goal apply to all standards applicable to a course or just to specific priority standards?

S tudent L earning O bjective: SLO Where can I find my standards? standards/ for Common Core standards/ for Arts, MST, Social Studies, Science, Health, PE, Family & Consumer Science, etc. for Career Development and Occupational Studies /marketing.html for Business and Marketing /marketing.html Where else?

Learning Content

S tudent L earning O bjective: SLO Interval of Instructional Time: Timeframe within which the learning content is being taught  One academic year, a semester? a quarter?a quarter?  Every other day? - Describe  Start and end date SLO Development Checklist OCMBOCES, SLO Guidance March 2012, APPR Guidance April 2012

Interval of Instructional Time

S tudent L earning O bjective: SLO Evidence: Assessments used to determine students’ level of learning  List the specific baseline assessments (pre-assessment) and summative assessment(s) that will be used to provide baseline and summative data for the SLO. 1. If the course ends in a NYS assessment or a Regents exam, it must be used as the summative assessment 2. List of 3 rd party assessments and Regents equivalents - FLACS3 rd party assessments 3. District, Regional, or BOCES developed. Districts or BOCES must verify comparability and rigor. What could the other option have been?  (School-or BOCES-wide, group, or team results based on State assessments) New York SLO Development Guide SLO Development Checklist OCMBOCES

S tudent L earning O bjective: SLO What if the third option was chosen?  District, Regional, or BOCES developed. Align tightly to learning content Include components of good assessment practices Scoring procedures – “vested interest”

BASELINES VESTED INTEREST G4: Does vested interest rule apply to pre-tests given to establish a baseline for a SLO? Answer: To the extent practicable, districts or BOCES should ensure that any assessments or measures, including those used for performance- based or performance task assessments that are used to establish a baseline for student growth are not disseminated to students before administration and that teachers and principals do not have a vested interest in the outcome of the assessments they score. CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

BASELINES VESTED INTEREST G4: Does vested interest rule apply to pre-tests given to establish a baseline for a SLO? Answer continued: If it is impracticable to comply with this requirement for pre-tests, such as in certain cases when using a performance-based or performance task assessment, the district or BOCES must have adequate procedures in place to ensure that the security of such assessments is not compromised (i.e., as with all SLOs the principal and/or supervisor must ensure the rigor and fairness of the targets and set the goals based on the assessment that is used as the baseline and ensure that such goals are adequately met based on summative data). CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

Performance Tasks?

Performance Tasks How do SLO’s meet the requirement for comparability? “Specify priority learning standards in a grade or subject around which assessments or performance tasks for students will be constructed by District Teams” (SLO Guidance March, Page 14) Includes a “standards-based” rubric (SLO Guidance March, Page 30)

S tudent L earning O bjective: SLO OR – other baseline options ! Algebra 2/Trig for Physics Grade 8 NYS English Assessment for 9 th grade English Second Grade Math Summative for Third grade

BASELINES D28: Question The SLO documents state that SLOs “must be specific and measurable, based on available prior student learning data" – what is meant by prior ? APPR Guidance June 2012, p37-38 Response Teachers are encouraged to use a variety of student historical academic data, if available. This historical academic data might include prior course results, or it might only include the current baseline data from the course taken at the beginning of the interval of instruction. CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

BASELINES D29: Question What should teachers use as a pre-assessment for establishing a baseline when students enter a sequence for the very first time and have little to no background information of a subject? APPR Guidance June 2012, p38 Response...it will be important for teachers to see what other courses they can draw on to provide other historical data. Any information about what entering students understand from other subject areas based on academic history in other courses, can give a teacher a wealth of information as to where the students may struggle or thrive. The teacher may want to also collect other baseline information (e.g., collect a writing sample) in order to more broadly assess the student's skills that may impact instruction for the year. CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

BASELINES D47: Question Can SLOs for students with disabilities have a different target for growth? APPR Guidance June 2012, p43 Response The target for students in any SLO may be differentiated because of the baseline (starting point of learning) and historical academic data. It is important to keep in mind that targets for all students, regardless of any special education classification, should be differentiated because of baseline data and not because of any special education classification. CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

SUMMATIVE – VESTED INTEREST This can be an issue. Teachers may not score the summative assessment. Talk of scoring guidelines for Regents exams to become more rigorous Performance task issues Questions regarding teachers’ involvement in the creation of summative assessments CIT, Lupinskie Center, One Merrick Ave. Westbury, NY 11590

Evidence

S tudent L earning O bjective: SLO Baseline: Level of students’ knowledge and skill in the targeted learning content at the beginning of the interval of instructional time. Describes how students performed on the pre- assessment or other previous summative assessment Include any other data sources being considered – that may impact your “target” Roster  Actual baseline scores for each student are required

Baselines On the Roster

S tudent L earning O bjective: SLO Target: Level of growth and skill that students are expected to achieve at the end point of the interval of instructional time. Numeric Roster – growth goal and scores for each student Written in the form of a statementon the SLO  Different formats consider the individual students growth, might connect baseline to summative, might be a “ n percent of students will ____________.” Aligned to school/district expectations Includes all populations: “including special populations ”

Important to Consider: The Number of Students Look at Co-taught Living Environment SLO exampleo-taught Living Environment SLO How many students are in this class? Describe what you see in the baseline component. What is the target statement? What HEDI rating is aligned to the target?

Important to Consider: The Number of Students 90% of all students = 90% of 19, which is aligned to 14 points on the HEDI scale Do the math. 18/19 would be 95% or 18 points 17/19 students would be 89% or 13 points 16/19 students would be 84% or 8 HEDI points What do you observe?

Differentiated Targets Take a second look at the Co-taught Regents Living Environment SLO. Examine the Baseline and Target section again. (Regionally developed pre-assessment) District decisions regarding Targets can be used while differentiating for clusters of student results.  Raw score of – achieve raw score of 80 or higher  Raw score of – achieve a raw score of 66 or higher (mastery)  Raw score of 29 or below – achieve a raw score of 41 or higher (passing) Target statement: 90% of all students will meet or exceed their goal.

Differentiated Targets What is the likelihood that students will hit their targets when they are created in this way? How is this more realistic?  Raw score of – achieve raw score of 80 or higher  Raw score of – achieve a raw score of 66 or higher (mastery)  Raw score of 29 or below – achieve a raw score of 41 or higher (passing)  Target statement: 90% of all students will meet or exceed their goal.

Differentiated Targets – Another Example

Student Population, Baseline, and Target(s) Training SLO Target Approach 1: Set a common growth target. 90% of students, including special populations, will grow by 60 percentage points or more on their summative assessment compared to their pre-test for the standards. (e.g., Student E’s target is 60 more than 30, or 90.) 50 StudentPre-Test ScoreSummative Target Student A1070 Student B2080 Student C565 Student D060 Student E3090 Student F1070

Student Population, Baseline, and Target(s) Training SLO Target Approach 2: Set a growth to mastery target. 85% of students, including special populations, will grow to score 75% or higher on the summative assessment for the selected standards. 51 StudentPre-Test ScoreSummative Target Student A1075 Student B2075 Student C575 Student D075 Student E3075 Student F1075

Student Population, Baseline, and Target(s) Training SLO Target Approach 3: Set differentiated growth targets by student. 85% of students, including special populations, will meet or exceed their individualized target. 52 StudentPre-Test ScoreSummative Target Student A1080 Student B2080 Student C575 Student D070 Student E3085 Student F1080

Target

S tudent L earning O bjective: SLO HEDI Criteria: Translates different levels of student growth to the four categories, Highly Effective, Effective, Developing, Ineffective. Allocates points clearly and objectively It must be mathematically possible for a given teacher to obtain any point value in the scale Description  Highly effective – exceeds district/BOCES expectations  Effective – meets district/BOCES expectations  Developing – below district/BOCES expectations  Ineffective – well-below district/BOCES expectations Guidance on NYS Dist. Goal-Setting Process: SLO. March 2012

More than one SLO?

HEDI Criteria

S tudent L earning O bjective: SLO Rationale – Include reasoning behind decisions. Particularly targets and HEDI alignment. That rationale could include  Reference to the population and content  Baseline data recorded on the roster  Other data considered – historical, cohort  National Any information that will help the reader grasp the rigor of the SLO Rate your SLO! – using the Annotated SLO Rubric Form

Rationale

Resources APPR Guidance Document, June, 2012 SLO Guidance Document, March, 2012 From NY DOE engageny.org Introduction to SLO Assessment Options for SLO’s on engageny.org Nys-eval-plans-guidance-Purple memo.pdf Ira Schwartz Update on Accountability DATAG July = =1518 SLO Connections to NYS Teaching Standards.pdf Annotated SLO Rubric Form

Special Ed, ESL, and More objectives-webinar-series-ii-fall-2012/ objectives-webinar-series-ii-fall-2012/