Student Learning Objectives: Process for Developing SLOs Developed by Pamela Paek and Jeri Thompson, Center for Assessment The purpose of this presentation is to walk through the process for developing Student Learning Objectives (SLOs). This process is based on a set of materials developed by the Center for Assessment.

Agenda Time Agenda Item 9:00-9:30 am Overview of SLOs 9:30-10:30 am   Introduction to SLO template Learning Goals 10:30-10:45 am Break 10:45-12:00 pm  Assessments & Targets 12:00-12:30 pm Lunch 12:30-1:30 pm Rubric 1:30-1:45 pm 1:45-2:45 pm Analysis and Revisions of Example SLO 2:45-3:00 pm Closure/Evaluation This table provides an overview of the day and what we plan to cover in each section. As you can see, we have many things on the agenda to cover today, and will walk you through each piece, to give you a background of SLOs, and walk through the template to describe each aspect of an SLO, show you how to use the rubric to rate an SLO, and provide you an opportunity to rate and improve an example SLO.

Presentation Objectives Understand the process of creating SLOs Defining learning goals in an SLO Understanding the use of assessments in SLOs Defining and understanding targets Understand and successfully use the SLO Template Understand and successfully use the SLO Rubric for evaluating the different aspects of the SLO template We have three main objectives for this presentation today. We want you to walk away with a thorough understanding of what an SLO is comprised of and how to use the template to help you develop a strong SLO. Understanding of the process for creating SLOs. This includes defining learning goals in an SLO, understanding the use of assessments in SLOs, and defining and understanding targets Understanding to successfully use the SLO Template Understanding to successfully use the SLO Rubric for evaluating and improving the different aspects of the SLO template [Note: slides 3-24 are scheduled to be done between 9:30-10:30 am]

Template: Page 1 Teacher Information Course/Grade Level Information Teacher Name School Name District Name Course/Grade Level Information Course Name Brief Course Description Grade Level(s) We will now walk you through each of the aspects of the SLO template, starting with the first page. We expect SLOs to be completed electronically. As such, we’ve created the header on the top of this template to repeat on every page of the SLO. This information includes the teacher name, school name, and district name. This reason for this teacher information repeated on each page is to ensure that if this document is printed, that all pages of the document are included correctly with the corresponding teacher. Note that you only have to fill out the teacher information section once. As soon as you do that, this information will automatically repeat on all subsequent pages. The next section lists the course or grade level information. The purpose of this section is to provide what course or class this SLO is written toward under course name. Then a brief course description is next, to indicate what you will teach in this course. Detailed description is vital to here, so someone who is not familiar about what is taught will have a solid understanding of the objectives of the course. And then be able to see how the standards and learning goal are core to student success in this course. Specifying grade level is also important, to be clear what grade level and subject matter is taught, and as another check point to make sure the included standards are appropriate for what is described.

Template: Page 1 Process, Implementation Timeline, and Sign-Offs List the names and current job positions of those developing this SLO. Administrator Name and Title Administrator sign-off of initial SLO Date final SLO is due for teaching effectiveness rating This next section on page 1 lists some of the logistics that are important to include as part of the auditing and sign-off process for SLO implementation. First, we want to know the names and current job positions of those developing this SLO. The reason for this is so someone reviewing the SLO can address the authors of the SLO for clarity. Second, the administrator name and title is to be included. This documents who will be the person approving this SLO at the beginning of the year as well as providing a teacher rating at the end of the year. The “sign-off of initial SLO” requires the administrator signature and date, to show that the administrator has used the SLO Rubric (we will discuss this later) and agrees all aspects of it are rated “acceptable” for use. This sign off means that no more edits are needed to improve this SLO and thus, it can be used for a teaching effectiveness rating later on. We call this the initial SLO because it is the plan for what will be done at the beginning of the year. The last piece of this section is the date the final SLO is due for the teaching effectiveness rating. This date is put here to indicate when teachers must have their final data collected in time for inclusion in this SLO. The final SLO includes the actual performance of the students, how that data is related to the expected performance, and the teacher’s effectiveness ratings.

Learning Goal Definition a description of what students will be able to do at the end of the course/grade based on the intended standards and curriculum that are being taught and learned as close to the individual student as possible, allowing for a variation based on the current achievement levels of individual or groups of students An SLO comprises three aspects: learning goals, assessments, and targets. Let’s talk about learning goals first. A learning goal is a description of what students will be able to do at the end of the course/grade based on the intended standards and curriculum that are being taught and learned as close to the individual student as possible, allowing for a variation based on the current achievement levels of individual or groups of students

Template: Learning Goal Which big idea is supported by the learning goal? Which content standards are associated with this big idea? List all standards that apply, including the text of the standards (not just the code). Why is this learning goal important and meaningful for students to learn? In what ways does the learning goal require students to demonstrate deep understanding of the knowledge and skills of the standards or big idea being measured? If you look at page 2 of the blank template, you’ll see a table with questions populating the left column. These questions are used to help you piece together all of the elements that need to be considered to write a strong learning goal. The first question asks: “Which big idea is supported by the learning goal?” The reason we ask this question is to make sure you are thinking about a concept that is beyond a single standard. This big idea is to be an overarching concept that integrates many of the course or grade level standards. This big idea is not just a general statement but tries to encompass a larger view of combining standards together. The next question asks, “Which content standards are associated with this big idea?” We ask that you list all standards that apply, including the text of the standards (not just the code). The reason we ask for the text of the standards is so that the reviewer has all of the information necessary in front of him/her to see what exactly is covered within this SLO. They will not need to cross reference any other document. In other words, we want anything about this SLO to be included in this document. The third question asks “Why is this learning goal important and meaningful for students to learn?” This question is used to push your thinking on what you’re trying to teach. In other words, is the big idea and set of standards you’re including core to what students should learn in this course? If not, then we’d want to rethink your big idea and corresponding standards. This question can also be used as a checking point to review the actual learning goal you write at the end of this section. The fourth question asks, “In what ways does the learning goal require students to demonstrate deep understanding of the knowledge and skills of the standards or big idea being measured?” To ensure that the learning goal assess higher-level thinking, this question is meant to have you justify how this will be done.

Template: Learning Goal Describe the instruction and strategies you will use to teach this learning goal. Be specific to the different aspects of the learning goal. Identify the time span for teaching the learning goal (e.g., daily class-45 minutes for the entire school year). Explain how this time span is appropriate and sufficient for teaching the learning goal. Describe the learning goal that includes and aligns to the planning information. Continuing on with the learning goal, the next few pieces asks for descriptions around instruction. The first description states: “Describe the instruction and strategies you will use to teach this learning goal. Be specific to the different aspects of the learning goal.” While this can be used as a generic section, we encourage you to be as specific as you can about how you’d teach this learning goal. There is obviously content and grade-level specific standards that are being addressed, and rather than write out a daily lesson plan here, or a very generalized statement, we want you to document specific things that will help you teach this goal particularly. Now we move onto time span. We ask for you to “identify the time span for teaching the learning goal (e.g., daily class-45 minutes for the entire school year).” The reason for this is so we have an understanding of the time frame for teaching this learning goal. We want specifics here so you can then answer the next request, which is “Explain how this time span is appropriate and sufficient for teaching the learning goal.” For instance, if you say you’re going to teach addition of fractions for an entire year in 8th grade, I’d want you to be able to justify how this is appropriate. Not only would this content be below what is needed in 8th grade math, it is also a specific set of skills that is discrete and not integrated into a bigger concept within mathematics. So, spending a whole year every day on this would not be appropriate, as other standards that are grade-level appropriate would not be addressed. All of this work has now hopefully helped you shape what your learning goal should be. So the final piece of this section asks you to “Describe the learning goal that includes and aligns to the planning information.”

The “SMART” Approach Specific: The learning goal is focused, for example, by content standards; by learners’ needs. Measurable: An appropriate instrument/measure is selected to assess the learning goal . Appropriate: The learning goal is within the teacher’s control to effect change and is a worthwhile focus for the students’ academic year. Realistic: The learning goal is feasible for the teacher. Time limited: The learning goal is contained within a single school year or appropriate unit of time. To help you map out the different pieces of the learning goal section, we advise using the “SMART” approach. We will go through each letter of this acronym and show how each is tied to different parts of the template. The SMART Approach stands for the following: S is for Specific: The learning goal is focused, for example, by content standards; by learners’ needs. M is for Measurable: An appropriate instrument/measure is selected to assess the learning goal . A is for Appropriate: The learning goal is within the teacher’s control to effect change and is a worthwhile focus for the students’ academic year. R is for Realistic: The learning goal is feasible for the teacher. T is for Time limited: The learning goal is contained within a single school year or appropriate unit of time.

Specific The learning goal is focused, for example, by content standards; by learners’ needs. Goldilocks Dilemma: Too broad  difficult to measure well. Too narrow  too discrete and piecemeal. What is “just right?” (Addressed by “big idea” and “content standards” in the template) Let’s talk more about Specific. When we say we want the learning goal to be specific, we are also saying we want the learning goal to be focused. For example, a learning goal can be specific by content standards (we have a section for specific standards in the template) or by learners’ needs (let’s say there is a high level idea based on a pre-test that students are struggling with). The issue with Specific is about what the right grain size is. If the learning goal is too broad, then it’s difficult to measure well because there is too much to cover and too few items to truly measure something that big. If the learning goal is too narrow, then it may not be measuring anything beyond a single standard. This could lead to teaching content in piecemeal or fragmented ways, which won’t help students see connections in what they learn. So, one of the things you learn to finesse in writing a learning goal is finding out what is “just right.” We call this the Goldilocks Dilemma. As she did, you need to keep trying to refine it to the right size. Specific is addressed by the “big idea” and “content standards” parts of the template.

Measureable An appropriate instrument/measure is selected to assess the learning goal (assessment). It is critical that we find valid assessments to measure the degree to which students achieve the learning goals. (Addressed in the Assessments section with high quality assessments that support and measure the learning goal. ) Measureable relates to assessments. We want to know that there is something being used that can actually measure the goal. Not only that, the instrument/measure selected to assess the learning goal must be appropriate and valid. It is critical that we find valid tools to measure the degree to which students achieve the learning goals. We discuss this in the assessments section when we discuss high quality assessments that support and measure the learning goal.

Appropriate The learning goal is within the teacher’s control to effect change and is a worthwhile focus for the students’ academic year. must be designed in ways that the teacher feels like they have a fair chance to succeed should be ambitious enough to require high quality teaching to achieve. (Addresses “important and meaningful” and “deep understanding” in the template) When we talk about appropriate, we want to make sure that: The learning goal is within the teacher’s control to effect change and is a worthwhile focus for the students’ academic year. The learning goal must be designed in ways that the teacher feels like s/he has a fair chance to succeed The learning goal should also be ambitious enough to require high quality teaching to achieve. Appropriate is addressed in the “important and meaningful” and “deep understanding” sections of the template)

Realistic The learning goal is feasible for the teacher to teach. While ambitious, the learning goals must be achievable, not just for the extraordinary teacher, but also for most effective teachers. (Addresses the “instruction and strategies” and “time span” pieces of the template) R stands for Realistic. This is to ensure that the learning goal is feasible for the teacher to teach. While we want the learning goal to be ambitious, they must also be achievable, not just for the extraordinary teacher, but also for most effective teachers. Realistic is addresses by the “instruction and strategies” and “time span” pieces of the template.

Time Limited The learning goal is contained within a single school year or appropriate unit of time. The learning goal must be written so they can be summatively evaluated within the time under the teacher’s control. (Addresses the two “time span” pieces of the template) Lastly, we have Time Limited. As you saw in the template, we have two time span pieces. When we include time limited here, we want to be sure the learning goal is contained within a single school year or appropriate unit of time (for instance, a semester long course would have the time frame limited to within when a teacher has students to teach). Similarly, the learning goal must be written so it can be summatively evaluated within the time under the teacher’s control. The reason for this is so when you get to the Targets section, you have data you can collect to show what progress your students have made.

Learning Goal Summary Using the planning information and the SMART format, the learning goal should now be the right size, detail, and depth, and be teachable within the time frame allotted. Explaining the learning goal with enough specificity is critical to writing a solid SLO. It’s the foundation that everything else is built on. To sum up what a learning goal is and how to do it, we think that by using the planning information in the template and the SMART format, the learning goal should now be the right size, detail, and depth, and be teachable within the time frame allotted. Explaining the learning goal with enough specificity is critical to writing a solid SLO. It’s the foundation that everything else is built on. [cue in picture of house with crumbling foundation]. This house here has a poor foundation. We know that means that anything built on top or around this bad foundation is unstable and not something you would trust. So… [cue in NOT sign], do not build a house with a poor foundation. That means, you need to make sure your foundation is solid. Your foundation is your learning goal. Get that right to start off. Then everything you build around it will be stable and strong.

Questions? Any questions you have about learning goals so far?

Social Studies Example: Learning Goal Describe the learning goal for this SLO. Students will independently use primary and secondary sources to explain, generalize, connect, and/or form an argument based on historical and contemporary issues related to civics and government. The learning goal is then detailed as follows: Students will independently use primary and secondary sources to explain, generalize, connect, and/or form an argument based on historical and contemporary issues related to civics and government

Social Studies Example: Big Idea Which big idea is to be measured by the learning goal? Demonstrating an understanding of civics and government, including the purpose and functions of politics and government, the establishment of the United States government, the role of individuals and groups in a civic society, and the interconnectedness of the political, social, cultural, and environment of nations and states. Let’s go through each piece of the social studies example so you have a concrete example of how each part of the learning goal section of the template can be completed. The first question asks: Which big idea is to be measured by the learning goal? This example’s response to this question is: Demonstrating an understanding of civics and government, including the purpose and functions of politics and government, the establishment of the United States government, the role of individuals and groups in a civic society, and the interconnectedness of the political, social, cultural, and environment of nations and states.

Social Studies Example: Standards Which content standards are associated with this big idea? State Social Studies Standards: Standard 1 – Students will understand the significance and impact of the Constitution on everyday life. Standard 2 – Students will understand the protections and privileges of individuals and groups in the United States. Standard 3 – Students will understand the distribution of power among the national, state, and local governments in the United States federal system, or compound constitutional republic. Standard 4 – Students will understand the responsibilities of citizens in the United States. Common Core State Standards for Literacy in History/Social Studies The standards associated with the big idea are then listed: State Social Studies Standards: Standard 1 – Students will understand the significance and impact of the Constitution on everyday life. Standard 2 – Students will understand the protections and privileges of individuals and groups in the United States. Standard 3 – Students will understand the distribution of power among the national, state, and local governments in the United States federal system, or compound constitutional republic. Standard 4 – Students will understand the responsibilities of citizens in the United States. Common Core State Standards for Literacy in History/Social Studies

Social Studies Example: Important Why is this learning goal important and meaningful for students to learn? It is critical for students to be able to read and comprehend primary and secondary sources in social studies in order to be college and career ready. Specifically, students should understand the types of evidence used in history sources, including domain-specific words and phrases, descriptions of events, concepts, and arguments that are analyzed from various sources. Students must be able to read complex informational texts in social studies with independence and confidence because the vast majority of reading in college and workforce training programs will be sophisticated nonfiction. Additionally, in order for students to be contributors to society, they must understand the structure of a democratic government and how they can contribute to and positively impact society. To address why this is an important and meaningful learning goal for students to learn: It is critical for students to be able to read and comprehend primary and secondary sources in social studies in order to be college and career ready. Specifically, students should understand the types of evidence used in history sources, including domain-specific words and phrases, descriptions of events, concepts, and arguments that are analyzed from various sources. Students must be able to read complex informational texts in social studies with independence and confidence because the vast majority of reading in college and workforce training programs will be sophisticated nonfiction. Additionally, in order for students to be contributors to society, they must understand the structure of a democratic government and how they can contribute to and positively impact society.

Social Studies Example: Deep Learning In what ways does the learning goal require students to demonstrate deep understanding of the knowledge and skills of the standards or big idea being measured? Analyzing multiple sources of texts and citing evidence requires strategic thinking and reasoning and/or extended thinking. Given the combination of these higher level thinking skills, this learning goal measures a high depth of knowledge (DOK), with a DOK level 3 or 4. * For more information about DOK, see http://www.nciea.org/publications/cognitiverigorpaper_KH11.pdf The way this learning goal requires deep understanding of the knowledge and skills include: Analyzing multiple sources of texts and citing evidence requires strategic thinking and reasoning and/or extended thinking. Given the combination of these higher level thinking skills, this learning goal measures a high depth of knowledge (DOK), with a DOK level 3 or 4. * For more information about DOK, see http://www.nciea.org/publications/cognitiverigorpaper_KH11.pdf

Social Studies Example: Instruction Describe the instruction and strategies you will use to teach this learning goal. The use of a civics and government high school textbook, primary sources and news articles are used as text resources, along with multi-media resources. Direct instruction on understanding the essential social studies’ civics and government ideas and concepts will take place on a daily basis. Instruction also includes modeling and scaffolding the application of the essential ideas in authentic situations presented as prompts. This includes completing graphic organizers and constructing informational argument writing on government and civics topics. The instruction and strategies to teach this learning goal include: The use of a civics and government high school textbook, primary sources and news articles are used as text resources, along with multi-media resources. Direct instruction on understanding the essential social studies’ civics and government ideas and concepts will take place on a daily basis. Instruction also includes modeling and scaffolding the application of the essential ideas in authentic situations presented as prompts. This includes completing graphic organizers and constructing informational argument writing on government and civics topics.

Social Studies Example: Time Span Identify the time span for teaching the learning goal (e.g., daily class-45 minutes for the entire school year). Students in grade 12 are engaged in civics and government instruction for 1 hour a day, 5 days a week for an entire school year. Instruction on constructing informational and argument writing is interwoven into this time period, approximately two times per month. The time span for teaching the learning goal is listed as: Students in grade 12 are engaged in civics and government instruction for 1 hour a day, 5 days a week for an entire school year. Instruction on constructing informational and argument writing is interwoven into this time period, approximately two times per month.

Social Studies Example: Time Span Explain how this time span is appropriate and sufficient for teaching the learning goal. By grade 12, students have had multiple opportunities to read and demonstrate understanding of complex historical text as well as to write informational and argument writing. In addition to learning civics and government concepts, students will have opportunities to practice and refine these reading and writing skills during this course. Since not all of the civics and government standards for this course are included in this learning goal, the amount of time designated to this learning goal is appropriate and sufficient. This time span is appropriate and sufficient, because by grade 12, students have had multiple opportunities to read and demonstrate understanding of complex historical text as well as to write informational and argument writing. In addition to learning civics and government concepts, students will have opportunities to practice and refine these reading and writing skills during this course. Since not all of the civics and government standards for this course are included in this learning goal, the amount of time designate to this learning goal is appropriate and sufficient.

Social Studies Example: SMART Review Use the SMART protocol to confirm that the Learning Goal has the right size, detail, and depth necessary. Check the boxes that apply. The Learning Goal is: Specific –focused on the big idea and content standards. Measurable – able to be appropriately and adequately assessed (note the Assessments section will identify the specific assessment to be used). Appropriate – within the teacher’s control to effect change and is important, meaningful for students to learn during the identified time span. Realistic – while ambitious, it is achievable for both teachers and students, during the time span identified. Time Limited – can be summatively evaluated within the time under the teacher’s control. To ensure that the learning goal has the right size, detail, and depth necessary, we provide a checklist using the SMART format. If any of these are not checked off, then the learning goal should be revised to ensure all the pieces are in place. The Learning Goal is: Specific –focused on the big idea and content standards. Measurable – able to be appropriately and adequately assessed (note the Assessments section will identify the specific assessment to be used). Appropriate – within the teacher’s control to effect change and is important, meaningful for students to learn during the identified time span. Realistic – while ambitious, it is achievable for both teachers and students, during the time span identified. Time Limited – can be summatively evaluated within the time under the teacher’s control.

15 Minute Break Let’s take a 15 minute break before we move onto the assessment and target sections.

Assessment Definition Assessments should be: standards-based designed to best measure the knowledge and skills found in the learning goal accompanied by clear criteria or rubrics to determine student learning from the assessment high quality measures used to evaluate the degree to which students achieved the developed learning goals Assessments should be used to support and measure the learning goal. Not vice versa. The two remaining aspects of the SLO are the assessments and targets. Let’s stat off with defining assessments for use with SLOs. Assessments should be: standards-based designed to best measure the knowledge and skills found in the learning goal accompanied by clear criteria or rubrics to determine student learning from the assessment high quality measures used to evaluate the degree to which students achieved the developed learning goals Assessments should be used to support and measure the learning goal. Not vice versa. If you are working with the QPA (Quality Performance Assessment) group, you will know much of the information we are going to be talking about. [Note: slides 26-50 are scheduled to be done between 10:45-12]

Template: Assessment Describe the assessments (such as performance tasks and their corresponding rubrics) that measure students’ understanding of the learning goal. Assessments and rubrics need to be established as high quality, such as through the High Quality Assessment Review Tool: Is the assessment clearly aligned to the identified big idea and standards? Does the assessment engage students appropriately considering both Depth-of-Knowledge and difficulty? Will the scoring tool allow for different scorers to reach the same score? Is the assessment fair and unbiased? Opening your template to the Assessment section, you’ll see that the assessment description is first. Describe the assessments (such as performance tasks and their corresponding rubrics) that measure students’ understanding of the learning goal. Assessments and rubrics need to be established as high quality, such as through the High Quality Content Review Tool. Consider Content validity – is the assessment aligned to the identified big idea and standards? Does the assessment engage students appropriately in the DOK and level of difficulty? Consider reliability – Will the scoring tool allow for different scorers to reach the same score? Is the assessment fair and unbiased? The Assessment Validation Protocol and Checklist from the QPA work will assist in determining a high quality assessment. While we believe a good part of the planning information is needed to write the assessment section, we know this is also an iterative process, where you nee to check your assessments against the learning goal and criteria for high quality.

Social Studies Example: Assessment Describe the assessments (such as performance tasks and their corresponding rubrics) that measure students’ understanding of the learning goal. Assessments and rubrics need to be established as high quality, such as through the High Quality Assessment Review Tool. A variety of common performance tasks were developed by the district Social Studies curriculum department and are in a bank of approved assessment prompts that can be used to measure student success. All tasks have been validated through the Assessment Review Tool. These tasks are aligned to the Social Studies state standards and the Common Core reading and writing standards. An example of an assessment and the Assessment Review Tool are attached. Let’s look at our Social Studies example. This is how this description was addressed: when describing the assessment, the response states “A variety of common performance tasks were developed by the district Social Studies curriculum department and are in a bank of approved assessment prompts that can be used to measure student success. All tasks have been validated through the Assessment Review Tool. These tasks are aligned to the Social Studies state standards and the Common Core reading and writing standards. An example of an assessment and the Assessment Review Tool are attached.” Let’s pull out the example assessment task and rubric from your folder so we can review these together using the Assessment Validation Checklist.

Social Studies Example: Scoring Explain how student performance is defined and scored using the assessments. Include the specific rubric and/or scoring criteria to be used. The use of a school-wide analytical 4-point argumentative writing rubric will be used to score student responses. This rubric was created by a Social Studies committee formed by the district level Social Studies curriculum department and was validated through the rubric section of the Assessment Review Tool. Now that we’ve seen the assessment and the high quality assessment tool, we’ll want to make sure we understand the rubric and scoring criteria. The next part of the template asks just that: Explain how student performance is defined and scored using the assessments. Include the specific rubric and/or scoring criteria to be used. The explanation of the rubric states “The use of a school-wide analytical 4-point argumentative writing rubric will be used to score student response. This rubric was created by a Social Studies committee formed by the district level Social Studies curriculum department and was validated through the rubric section of the Assessment Review Tool.” Review the rubric using the Assessment Validation Checklist.

Social Studies Example: Data Collect How often will you collect data to monitor student progress toward this learning goal? Within the second quarter and throughout the remainder of the school year, students will have monthly opportunities to demonstrate their ability to use various identified texts (primary and secondary sources) to respond to informal and formal teacher developed prompts requiring them to form an argument regarding a civic/government concept. At least three times during the year students will respond to a prompt on a formal assessment. To get a better sense of how often data is collected toward measuring student progress, part of the template maps that out specifically: How often will you collect data to monitor student progress toward this learning goal? The response in this example says “Within the second quarter and throughout the remainder of the school year, students will have monthly opportunities to demonstrate their ability to use various identified texts (primary and secondary sources) to respond to informal and formal teacher developed prompts requiring them to form an argument regarding a civic/government concept. At least three times during the year students will respond to a prompt on a formal assessment.”

Social Studies Example: Monitor Progress How will you use this information to monitor student progress and to differentiate instruction for all students toward this learning goal? Student work, both informal and formal, will be analyzed for ability to demonstrate each of the criteria on the rubric, including the ability to identify relevant content evidence. Students who are struggling readers and writers will have assistance with reading sources. A variety of levels of sources will be used to engage struggling readers. A variety of graphic organizers will be used to help students capture their thoughts prior to answering the assessment prompts. Students struggling with key social studies concepts will work with the teacher and/or peers in small groups or will have individual instruction that may include the use of organizers as a means of recording key information needed to respond to the question. Advanced students will have opportunities to research, read, and analyze complex texts with greater independence as well as consider the information in extended self-generated questions. The follow-up question to how often the data is collected is how it will be used. More specifically, the template asks: How will you use this information to monitor student progress and to differentiate instruction for all students toward this learning goal? The response here talks about formal and informal data: Student work, both informal and formal, will be analyzed for ability to demonstrate each of the criteria on the rubric, including the ability to identify relevant content evidence. Students who are struggling readers and writers will have assistance with reading sources. A variety of levels of sources will be used to engage struggling readers. A variety of graphic organizers will be used to help students capture their thoughts prior to answering the assessment prompts. Students struggling with key social studies concepts will work with the teacher and/or peers in small groups or will have individual instruction that may include the use of organizers as a means of recording key information needed to respond to the question. Advanced students will have opportunities to research, read, and analyze complex texts with greater independence as well as consider the information in extended self-generated questions.

Targets Definition Identify the expected outcome by the end of the instructional period. May differ for subgroups of students. There are two key components of the targets associated with SLO: End Goal: What performance demonstrates that students met the learning goal using your assessments? Starting Level: If we expect all students to all achieve the same end goal, then we can skip this step, but more likely there will be some differentiation of goals. Now that we have assessments, we now need to use the data from those to establish our targets. Targets identify the expected outcome by the end of the instructional period. They may differ for subgroups of students. There are two key components of the targets associated with SLO: End Goal: What performance demonstrates that students met the learning goal using your assessments? Starting Level: If we expect all students to all achieve the same end goal, then we can skip this step, but more likely there will be some differentiation of goals.

Social Studies Example: Establishing Describe the courses, assessments, and/or experiences used to establish starting points and expected outcomes for students’ understanding of the learning goal. Based upon a review of the students prior social studies course grades (9th grade-World History and Geography, 10th grade-American Studies, 11th grade social studies course), 11th grade reading and writing state test scores, and a school writing portfolio which contains district developed performance assessments that are scored using the district argument rubric, students have been placed in high, average, and low performing groups for the SLO targets. Let’s go through each piece of the social studies example so you have a concrete example of how each part of the learning goal section of the template can be completed. The template asks to describe the courses, assessments, and/or experiences used to establish expected outcomes for students’ understanding of the learning goal. The example SLO describes these as follows: Based upon a review of the students prior social studies course grades (9th grade-World History and Geography, 10th grade-American Studies, 11th grade social studies course), 11th grade reading and writing state test scores, and a school writing portfolio which contains district developed performance assessments that are scored using the district argument rubric, students have been placed in high, average, and low performing groups for the SLO targets.

Template: Targets Using prior performance, classify students into “performance” groups Different levels of achievement (e.g., basic, proficient) Different proportions of students reaching the same target (e.g., 80% of Level 3 students will achieve target) SLO targets would then be differentiated according to the students’ starting group. To establish targets, one way to do this is by differentiating students on where they start off the course or grade using previous years’ data. So, using prior performance, you can classify students into “performance” groups. You can group students is into different levels of achievement (e.g., basic, proficient). Then you can target different proportions of students reaching the same target (e.g., 80% of Level 3 students will achieve target). SLO targets would then be differentiated according to the students’ starting group.

Social Studies Example: Starting Points Identify the actual performance (e.g., grades, test scores, etc.) to establish starting point for students. 35% of the students were below proficiency in reading and writing on their grade 11 state test received Ds and Fs in their 11th grade English and social studies courses (similar performance in 9th and 10th) includes students with disabilities Includes English Language Learners (ELL) Using this prior performance information, this group of students is placed in the low performing level. The first piece of the Targets section asks for a description of data that is used to get an understanding of where students are coming into the course by asking you to identify the actual performance (e.g., grades, test scores, etc.) of students for the identified courses, assessments, or other sources of information to categorize student levels as their starting points. This description includes a way to differentiate students with their past test scores and grades: 35% of the students entering the 12th grade social studies class are below grade level in reading and writing (Level 1) on their grade 11 state test, received Ds and Fs in their 11th grade English and social studies courses, and performed at a Level 1 (below) on a significant number of performance assessments from previous grades. They had similar performance in 9th and 10th grade. These struggling students include students with disabilities and those classified as English Language Learners (ELL). Using this prior performance information, this group of students is placed in the low performing level.

Social Studies Example: Starting Points Level Starting # of Students Low 21 Average 33 High 6 This 35% from the previous slide equates to 21 students, as seen in the table in the low row. The remaining 65% of the students are at either nearing proficient (35%) or at proficient (30%) on their grade 11 state test and received a C or higher in their 11th grade English and social studies courses. They had similar performance in 9th and 10th grade. Of these students, 10% received an A in their 11th English and/or social studies courses: this 10% has been placed in the high performing level. The remaining 50% of students are placed in the average performing level. There are no students who are reading above grade level or who scored at the exceeded proficiency level for reading comprehension of informational text, informational or argument writing on their state test.

Template: Targets Expected Targets: Using students’ starting points, identify the number or percentage of students expected at each achievement level based on their assessment performance(s). Be sure to include any appropriate subgroups. The expected targets are your goals for where students end up, which may be differentiated from where they start. The template asks you to explain your expected targets: Using students’ starting points, identify the number or percentage of students expected at each achievement level based on their assessment performance(s). Be sure to include any appropriate subgroups.

Social Studies Example: Expected Targets Using students’ starting points, identify the … expected Based on the argumentative writing rubric scores for teacher created tasks three (3) formal performance tasks Expected targets are NOT an average of scores, it’s demonstrated consistent improvement over time in the criteria of the rubric.  expected target based on each of the criteria in the rubric The example starts by explaining where the data comes from and how scores are not averaged: These targets are based on the argumentative writing rubric scores for the teacher created tasks, as well as the three (3) formal performance tasks administered in this course over the entire school year. This is not an average of scores, but rather an evaluation of students' demonstrated consistent improvement over time in the criteria of the rubric.  Since there is no state assessment for grade 12, these target grouping decisions are based on the evaluation of each of the criteria in the rubric for argumentative writing, across both teacher created and formal performance tasks throughout the school year.

Social Studies Example: Expected Targets Using students’ starting points, identify the … expected Using the different rubric criteria, students with scores: below or nearing proficient = low performing group  mainly proficient = average performing group mainly advanced = high performing group The template indicates to: “Record the actual number or percentage of students who achieved the targets. Be sure to include any appropriate subgroups.” The example goes into detail on which data was used and how they were grouped based on expected targets: These targets are based on the argumentative writing rubric scores for the teacher created tasks, as well as the three (3) formal performance tasks administered in this course over the entire school year. This is not an average of scores, but rather an evaluation of students' demonstrated consistent improvement over time in the criteria of the rubric. Based on students' performance on the different rubric criteria across teacher created and formal performance tasks throughout the year, students are classified in the low, average, or high performing groups. Students with scores below or nearing proficient on the different rubric criteria are classified in the low performing group. Students with scores that are mainly proficient on the different rubric criteria are classified in the average performing group. Students with scores that are mainly advanced on the different rubric criteria are part of the high performing group.

Social Studies Example: Expected Targets (Low) Level Starting # of Students Expected # of Students Low 21 5 Average 33 16 High 6 Approximately 75% of the 21 students starting in the low performing group at the beginning of the year will move into the average group (16 students) by the end of the year, reaching the proficient level on the majority of the rubric’s criteria across both teacher created and formal performance tasks.  The students remaining in the low performing group (5 students) will score at the nearing proficiency level on the majority of the rubric’s criteria across both teacher created and formal performance tasks.

Social Studies Example: Expected Targets (Average) Level Starting # of Students Expected # of Students Low 21 5 Average 33 16 30 High 6 3 The number and percentage of students expected in the average performing group is also defined by performance using the rubric: Approximately 10% of the 33 students in the average performing group at the beginning of the year will move into the high performing group (3 students) by the end of the year, reaching the advanced level on the majority of the rubric’s criteria.  The students remaining in the average performing group (30 students) will score at the proficient on the majority of the rubric’s criteria with some demonstration of performance at the advanced level across both teacher created and formal performance tasks.

Social Studies Example: Expected Targets (High) Level Starting # of Students Expected # of Students Low 21 5 Average 33 16 30 High 6 3 The number and percentage of students expected in the high performing group is also defined by performance using the rubric: 100% of the 6 students starting in the high performing group at the beginning of the year will remain in the high performing group at the end of the year, reaching the advanced level on the majority of the rubric’s criteria across both teacher created and formal performance tasks.

Social Studies Example: Expected Targets Identify your expected targets below and explain how these targets demonstrate ambitious, yet realistic goals, for measuring students’ understanding of the learning goal. These targets are ambitious and realistic Several of the struggling students are scoring as nearing proficiency on the state assessment in 11th grade. With the differentiated instruction, these students will be able to become proficient. Students significantly below proficient progress at least one level on the rubric (e.g., novice to apprentice, or novice to proficient). The next piece is two-fold: after listing what you expect students to be able to do on the assessments you provide, you need to explain how these expected targets are challenging for you to achieve and make sure you’re setting the bar high, not low just so you can pass it: Identify your expected targets below and explain how these targets demonstrate ambitious, yet realistic goals, for measuring students’ understanding of the learning goal? The response says: These targets are ambitious and realistic for a 12th grade yearlong study of civics and government. Several of the struggling students, based on the above-identified data, are scoring as nearing proficiency. With the differentiated instruction, these students will be able to also score as proficient. Students who are significantly below proficient should be able to demonstrate growth by at least one level on the rubric (e.g., novice to apprentice, or novice to proficient).

Template: Targets Actual Outcomes: Record the actual number or percentage of students who achieved the targets. Be sure to include any appropriate subgroups. Please provide any comments you wish to include about actual outcomes Until now, everything that has been completed in the SLO template is goal setting. The actual targets are how your students actually performed at the end of the year. The template asks you to Record the actual number or percentage of students who achieved the targets. Be sure to include any appropriate subgroups (this assumes you had differentiated targets, maybe by disability category or another classification. To explain your actual outcomes, the template provides a place for you to do so: Please provide any comments you wish to include about actual outcomes

Actual Performance Level Starting # of Students Expected # of Students Actual # of Students Low 21 5 6 Average 33 46 49 High 9 The template asks for “actual performance” of students: “Record the actual number or percentage of students who achieved the targets. Be sure to include any appropriate subgroups.” Approximately 71% of the 21 students starting in the low performing group at the beginning of the year moved into the average group (15 students) at the end of the year, reaching the proficient level on the majority of the rubric’s criteria across both teacher created and formal performance tasks.  Of the 6 students remaining in the low performing group, 3 students scored at the nearing proficiency level on the majority of the rubric’s criteria across both teacher created and formal performance tasks, while the other 3 students scored at the below proficiency level on the majority of the rubric’s criteria across both teacher created and formal performance tasks. Approximately 3% of the 33 students in the average performing group at the beginning of the year moved into the high performing group (1 student) at the end of the year, reaching the advanced level on the majority of the rubric’s criteria.  The students remaining in the average performing group (33 students) scored at the proficient level on the majority of the rubric’s criteria with minimal demonstration of performance at the advanced level across both teacher created and formal performance tasks. And further to talk about movement from the high to average group: Of the 6 students starting in the high performing group at the beginning of the year, 67% remained in this group, reaching the advanced level on the majority of the rubric’s criteria across both teacher created and formal performance tasks.  The other 23% (2 students) moved into the average performing group with the majority of the rubric’s criteria across both teacher created and formal performance tasks at the proficient level. Although the percentages have some discrepancies (e.g., expected 10%; actual 3%), the actual number of students in each group illustrates the overall improvement of student performance. However, the actual targets fell slightly below the expected targets.

Template: Teacher Ratings Based on the actual versus expected targets: Does Not Meet: Based on the students’ starting points, students performed worse than expected. Meets: Based on the students’ starting points, students performed as expected. Exceeds: Based on the students’ starting points, students performed better than expected. Administrator comments Administrator and Teacher Signature Finally, the comparison of the actual and expected targets then determines a teacher’s rating as does not meet, meets, or exceeds their targets (goals). If the actual data falls below the expected, then the teacher is classified as “does not meet” while if they surpass their goals, they are rated as “exceeds.” There is room for administrator comment as well as signatures of both administrator and teacher, which then finalizes the SLO.

Social Studies Example: Teacher Ratings: Based on the results of the learning goal, assessments/tasks, and targets of this SLO, a teacher rating is noted below.  Meets Based on the students’ starting points, students performed as expected. Administrator comments: Students demonstrated some improvement in this course; however anticipated targets were not met. In addition several students moved to a lower level than where they began. This example shows that the teacher met their targets, even though the expected targets were not met exactly. The administrator indicates that there was improvement in the course, but not as much as what was anticipated. Students demonstrated some improvement in this course; however anticipated targets were not met. In addition several students moved to a lower level than where they began.

Questions? What other questions do you have?

30 Minute Lunch Break It is now time for lunch. Let’s try to spend 30 minutes on lunch, since we have such a short time together today, but lots to do.

Rubric: Rating the Quality of SLOs Acceptable Needs Improvement Unacceptable Learning Goal Assessments Targets You have a single page document entitled “Rubric for Rating the Quality of Student Learning Objectives.” The purpose of this rubric is for use by teachers, school administration, and district administration to evaluate the different aspects of Student Learning Objectives (SLOs). The left column consists of the three aspects of the SLO: learning goal, assessments, and targets. The top row consists of the three ratings: acceptable, needs improvement, and unacceptable. For any SLO to be used for teacher performance ratings, all three aspects must meet the “acceptable quality” rating on this rubric. [Note: Slides 52-60 are scheduled for 12:30-1:30 pm] For an SLO to be used for teacher ratings, all three aspects of the rubric must meet the “acceptable” rating.

Rubric: Learning Goal Reviewing the Learning Goal row of the rubric, highlight or underline the words and phrases that help you differentiate the three levels of ratings: What are the differences between and Acceptable and Needs Improvement? What are the differences between Needs Improvement and Unacceptable? To get clarity on the differences of the three levels for each SLO aspect, let’s look at one row at a time. Reviewing the Learning Goal row of the rubric, highlight or underline the words and phrases that help you differentiate the three levels of ratings: What are the differences between and Acceptable and Needs Improvement? What are the differences between Needs Improvement and Unacceptable? Take about 5 minutes to highlight differences and to see if you agree with your table on how the three rating levels are different. Where do you need more clarity? How can you ensure that your rating of acceptable for Learning Goal would truly be the same as your peers at your table?

Learning Goal Example Review this example and use the rubric to help rate the quality of this section. How would you rate the different aspects of the learning goals? Where do you and your table disagree? Discuss the different ratings within your group. What evidence did you use to come to your ratings? How would you raise the quality of the learning goal? Let’s pull out the Social Studies example we walked through this morning. Now that you’re familiar with it, we want you to review just the Learning Goal section of this example and use the rubric to help rate the quality of this section. Take about 10 minutes to address these questions. How would you rate the different aspects of the learning goals section? Where do you and your tablemates disagree? Discuss the different ratings within your group. What evidence did you use to come to your ratings?

Rubric: Assessment Reviewing the Assessment row of the rubric, highlight or underline the words and phrases that help you differentiate the three levels of ratings: What are the differences between and Acceptable and Needs Improvement? What are the differences between Needs Improvement and Unacceptable? Reviewing the Assessment row of the rubric, highlight or underline the words and phrases that help you differentiate the three levels of ratings: What are the differences between and Acceptable and Needs Improvement? What are the differences between Needs Improvement and Unacceptable? Take about 5 minutes to highlight differences and to see if you agree with your table with how the three rating levels are different. Where do you need more clarity? How can you ensure that your rating of acceptable for Assessments would truly be the same as your peers at your table?

Assessment Example Review this example and use the rubric to help rate the quality of this section. How would you rate the different aspects of the assessment? Where do you and your table disagree? Discuss the different ratings within your group. What evidence did you use to come to your ratings? Now review just the Assessment section of this example and use the rubric to help rate the quality of this section. Take about 10 minutes to address these questions. How would you rate the different aspects of the Assessment section? Where do you and your tablemates disagree? Discuss the different ratings within your group. What evidence did you use to come to your ratings?

Rubric: Targets Reviewing the Targets row of the rubric, highlight or underline the words and phrases that help you differentiate the three levels of ratings: What are the differences between and Acceptable and Needs Improvement? What are the differences between Needs Improvement and Unacceptable? Reviewing the Targets row of the rubric, highlight or underline the words and phrases that help you differentiate the three levels of ratings: What are the differences between and Acceptable and Needs Improvement? What are the differences between Needs Improvement and Unacceptable? Take about 5 minutes to highlight differences and to see if you agree with your table on how the three rating levels are different. Where do you need more clarity? How can you ensure that your rating of acceptable for Targets would truly be the same as your peers at your table?

Targets Example Review this example and use the rubric to help rate the quality of this section. How would you rate the different aspects of the targets? Where do you and your table disagree? Discuss the different ratings within your group. What evidence did you use to come to your ratings? What about the teacher’s rating? Do you agree with this categorization? Why or why not? Now use the rubric to help rate the quality of the Targets section. Take about 10 minutes to address these questions. How would you rate the different aspects of the Targets section? Where do you and your tablemates disagree? Discuss the different ratings within your group. What evidence did you use to come to your ratings? What about the teacher’s rating? Do you agree with this categorization? Why or why not?

Discuss Ratings Let’s see what the overall ratings were by table for each of these sections. What do you think about the rating of this example for the Learning Goal section? Assessment section? Targets section? Let’s now reconvene as a large group and see if we can get a sense of what the ratings were by table (take 5 minutes max). For learning goals, how many rated acceptable? Needs improvement? Unacceptable? [take a tally] How many rated the Assessment section as acceptable? Needs improvement? Unacceptable? [take a tally] How many rated the Targets section as acceptable? Needs improvement? Unacceptable? [take a tally] [Now summarize the majority of ratings for each of these three sections.]

Annotations An annotated version of this Social Studies SLO details the comments to justify the ratings it received. Review your ratings and comments against these annotations that lists the issues we raised. What changes did you find that were not addressed in this document? What ones were on this document that you did not include? An annotated version of this Social Studies SLO details the comments to justify the ratings it received. Take the next 10-15 minutes to review your ratings and comments against these annotations that list the issues we raised. What changes did you find that were not addressed in this document? What ones were on this document that you did not include?

Questions? What other questions do you have?

ELA Example Review this new example and use the rubric to help rate the quality of the Learning Goal, Assessments, and Targets Review the teacher ratings based on their expected versus actual targets. What evidence did you use to come to your ratings? How would you raise the quality of each section? Make edits directly onto the document in how you would change the language to improve the clarity of this section. You have an hour to review this new example and use the rubric to help rate the quality of the Learning Goal, Assessments, and Targets Review the teacher ratings based on their expected versus actual targets. What evidence did you use to come to your ratings? How would you raise the quality of each section? Make edits directly onto the document in how you would change the language to improve the clarity of this section. We will circulate around tables and provide feedback and answer questions. [Note: slide 62 will set up what they will do for the hour after the break from 1:45-2:45].

15 Minute Break If you need a break now, please take one. The rest of the afternoon we’re going to have you work at your tables on rating this second example.

ELA Example: Check In Review your ratings and comments against the Annotated sheet that lists the issues we raised. What changes did you find that were not addressed in this document? What ones were on this document that you did not include? [After about 35-40 minutes of them working, let them know about this document they can check their work against.] For the next 20 minutes, review your ratings and comments against the ELA annotations that lists the issues we raised. What changes did you find that were not addressed in this document? What ones were on this document that you did not include?

Writing an SLO If time permits, begin drafting your own learning goal using the blank template, with the beginning of it seen above.

Thank you! Please go to http://www.nciea.org for access to more resources in our SLO Toolkit. We will be updating it regularly. [With 15 minutes left, let panelists know you want them to finish up their discussion – 2 minute warning. Then spend a couple minutes discussing the Center’s SLO toolkit] For more information about SLOs, please go to http://www.nciea.org for access to the SLO Toolkit and its resources. It will be updated regularly, to include further resources and examples for use. Thank you so much for your hard work today. This is tough stuff, and we appreciate you getting into the work.