1. Overview for School Leaders

2. Session Objectives I.Review Teacher Effectiveness System II.Define SLO process III.Exploring SLO Templates -Assessment Literacy- IV.Identifying Key Points for School Leaders V.Action Planning

4. (B) FOR PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONAL EMPLOYES WHO SERVE AS CLASSROOM TEACHERS, THE FOLLOWING SHALL APPLY: (1) BEGINNING IN THE 2013-201 4 SCHOOL YEAR, THE EVALUATIO N OF THE EFFECTIVENESS OF PROFESSIONAL EMPLOYES AND TEMPORARY PROFESSIONAL EMPLOYES SERVING AS CLASSROOM TEACHERS SHALL GIVE DUE CONSIDERATION TO THE FOLLOWING: (I) CLASSROOM OBSERVATION AND PRACTICE MODELS THAT ARE RELATED TO STUDENT ACHIEVEMENT IN EACH OF THE FOLLOWING AREAS: (A) PLANNING AND PREPARATION. (B) CLASSROOM ENVIRONMENT. (C) INSTRUCTION. (D) PROFESSIONAL RESPONSIBILITIES. (II) STUDENT PERFORMANCE, WHICH SHALL COMPRISE FIFTY PER CENTUM (50%) OF THE OVERALL RATING OF THE PROFESSIONAL EMPLOYE OR TEMPORARY PROFESSIONAL EMPLOYE SERVING AS A CLASSROOM TEACHER AND SHALL BE BASED UPON MULTIPLE MEASURES OF House Bill 1901 Race to the Top Act 82

5. 5

6. 6

7. Educator Effectiveness Prezi http://prezi.com/2wjeukgle6ja/?utm_campaig n=share&utm_medium=copy&rc=ex0sharehttp://prezi.com/2wjeukgle6ja/?utm_campaig n=share&utm_medium=copy&rc=ex0sharehttp://prezi.com/2wjeukgle6ja/?utm_campaig n=share&utm_medium=copy&rc=ex0sharehttp://prezi.com/2wjeukgle6ja/?utm_campaig n=share&utm_medium=copy&rc=ex0share

8. Observation/Evidence (50%) 4 Domains, 22 Components Principal/Evaluator Observes

9. Multiple Measures of Student Achievement Multiple Measures of Student Achievement 1.Building Level Data (School Performance Profile) Academic Achievement, Graduation/Promotion Rate, Attendance, AP-IB Courses offered, PSAT, Building Level PSSA and Keystone Assessment Data 2.Correlation Data Based on Teacher Level Measures PSSA, Keystone Data 3. Elective Data (SLOs)

11. SLO Process A process to document a measure of educator effectiveness based on student achievement of content standards.

12. SLO Concepts Student achievement can be measured in ways that reflect authentic learning of content standards. Educator effectiveness can be measured through use of student achievement measures.

13. The SLO in PA is written to a specific teacher and a specific class/course/content area for which that teacher provides instruction in the area they are certified to teach.

14. Many factors can influence the size of an SLO, but the process remains the same……….. Time Frame Course Content Important Learning Needs

15. SLO Process Design Indicators Performance Measures Goal- Standards SLO Goal Assessment #1a Indicator #1 Indicator #2 Assessment #1b Assessment #2

16. SLOs should: 1.Represent the diversity of students and courses/content areas taught. 2.Align to a set of approved indicators/targets related to selected academic content standards. 3.Be based upon two time-bound events/data collection periods and/or performance defined levels of “mastery”. 4.Be supported by verifiable data that can be collected and scored in a standardized manner. 5.Include a set of independent performance measures. SLO Process Criteria

17. SLO Process Steps: Teacher 1.Identify subject and students 2.Select the “big idea” from the content standards 3.Establish a goal 4.Identify indicators associated with the goal 5.Select and/or create performance measures for each indicator 6.Create performance expectations across all indicators

19. SLO Template A process tool used to identify goals, indicators, and performance measures for use in the greater Teacher Effectiveness System Handouts: SLO Template, Help Desk, & Performance Task Framework

20. SLO Template Design Context Goal Measures Indicators Expectations

21. 1.Goals are based upon the “big ideas” within the content standards. 2.Performance indicators are specific, measureable, attainable, and realistic. 3.Performance measures should be valid, reliable, and rigorous assessments. 4.Data should be collected, organized, and reported in a consistent manner. 5.Teacher expectations of student achievement should be demanding. SLO Template Criteria

22. SLO Template Steps: Teacher 1.Classroom Context 1a. Name1b. School1c. District 1d. Class/ Course Title 1e. Grade Level 1f. Total # of Students 1g. Typical Class Size 1h. Class Frequency 1i. Typical Class Duration 2. SLO Goal 2a. Goal Statement 2b. PA Standards 2c. Rationale

23. Spanish 1 Students will be able demonstrate effective communication in the target language by speaking and listening, writing, and reading. 8 th Grade Art Students will demonstrate the ability to manipulate visual art materials and tools to create works based on the ideas of other artists and to evaluate the processes and products of themselves and other artists. Grade 5 Library Students will demonstrate the ability to use online D.P.S. databases and search engines, Britannica Elementary, Culture Grams, and Nettrekker toward support real world experiences and determining which is the best source for specific information.

24. Targeted content standards used in developing the SLO. Arts and Humanities: 9.1, 9.2, 9.3, 9.4 http://pdesas.org/

25. Explains why the SLO is important and how students will demonstrate learning of the standards through this objective. Grade 8 Art: Developing the ability to manipulate visual art materials and tools are important to the artistic creation process, as is the ability to evaluate the process and product created by oneself and others. Child Development (FCS) Understanding how children grow and develop will prepare individuals and families to meet challenges associated with raising children.

26. SLO Template Steps: Teacher 3. Performance Measures (PM) 3a. Name  PM #1  PM #2  PM #3  PM #4  PM #5 3b. Type ____District-designed Measures and Examinations ____Nationally Recognized Standardized Tests ____Industry Certification Examinations ____Student Projects ____Student Portfolios ____ Other:______________________________ 3c. Purpose  PM #1  PM #2  PM #3  PM #4  PM #5 3d. Metric Growth (change in student performance across two or more points in time) Mastery (attainment of a defined level of achievement) Growth and Mastery 3e. Administration Frequency  PM #1  PM #2  PM #3  PM #4  PM #5 3f. Adaptations/ Accommodations IEP ELL Gifted IEP Other 3g. Resources/ Equipment  PM #1  PM #2  PM #3  PM #4  PM #5 3h. Scoring Tools  PM #1  PM #2  PM #3  PM #4  PM #5 3i. Administration & Scoring Personnel  PM #1  PM #2  PM #3  PM #4  PM #5 3j. Performance Reporting  PM #1  PM #2  PM #3  PM #4  PM #5

27. Many things must be considered when choosing or building quality assessments.

28. Choosing or Building Performance Measures and Tasks What must a Student know and do to complete a performance measure? What does a Teacher do to administer a performance measure? How does a Teacher score a performance measure?

29. SLO Template Steps: Teacher 4. Performance Indicators (PI) 4a. PI Targets: All Student Group  PI Target #1  PI Target #2  PI Target #3  PI Target #4  PI Target #5 4b. PI Targets: Subset Student Group (optional)  PI Target #1  PI Target #2  PI Target #3  PI Target #4  PI Target #5 4c. PI Linked (optional) 4d. PI Weighting (optional)

30. Describes individual student performance expectation 4a. What performance measure(s) – tests, assessments– will be used to measure student achievement of the standards, and what’s the expected student achievement level based on the scoring system for those measures? 4b. What’s the expected achievement level for unique populations? (IEP, students who did not do well on a pre-test, etc.)

31. Performance Indicator Statement HS Choral Individual Vocal Assessment Task Students will achieve proficient or advanced levels in 6 out of 8 criteria of the second scoring rubric. 5 th Grade ELA DRA text gradient chart Students will demonstrate one year of reading growth

32. A Temporary Detour…

33. Foundational Knowledge Basic Assessment Literacy

34. Test Specifications When developing test specifications consider: Sufficient sampling of targeted content standards Aim for a 3:1 items per standard ratio Developmental readiness of test-takers Type of items Multiple Choice (MC) Short Constructed Response (SCR) Extended Constructed Response (ECR)/Complex Performance tasks Time burden imposed on both educators and students 34

35. Test Specifications (cont.) When developing test specifications consider: Cognitive load Aim for a balance of DoK levels Objectivity of scoring Each constructed response item/task will need a well-developed rubric Weight of items (point values) Measures (tests) should consist of 25-35 total points; 35-50 points for high school Item cognitive demand level/DoK level Measures should reflect a variety of DoK levels as represented in the targeted content standards 35

36. Test Specifications Example 36 Content Strand(s)MCSCRECRTotal Expressions & Equations4004 Creating Equations5005 Structure in Expressions3003 Ratios & Proportions3205 Reasoning with Equations & Inequalities4105 Interpreting Functions3216 Real Number System5117 Grand Totals276235 *Performance measure contains 35 items/tasks. Content Strand(s)MC (1 pt.) SCR (2pts.) ECR (4pts.) Total Expressions & Equations4004 Creating Equations5005 Structure in Expressions3003 Ratios & Proportions3407 Reasoning with Equations & Inequalities4206 Interpreting Functions34411 Real Number System52411 Grand Totals2712847 *Performance measure score based upon 47 points.

37. Test Specifications Example (cont.) 37 Content Strand(s)DoK 1DoK 2DoK 3Total Expressions & Equations1214 Creating Equations1225 Structure in Expressions2013 Ratios & Proportions0505 Reasoning with Equations & Inequalities 3205 Interpreting Functions0246 Real Number System1427 Grand Totals8171035 *Performance measure contains items/tasks with the following Level/DoK distribution: DoK 1 = 23% DoK 2 & 3 = 77%

38. Stem (question) with four (4) answer choices Typically worth one (1) point towards overall score On the new PSSA there are MC questions with two answers Generally require about one (1) minute to answer, depending on rigor/DoK Pros Easy to administer Objective scoring Cons Students can guess the correct answer No information can be gathered on the process the student used to reach answer (error analysis) 38 Multiple Choice Items

39. Requires students to apply knowledge, skills, and critical thinking abilities to real-world performance tasks Entails students "constructing" or developing their own answers in the form of a few sentences or bullet points, a graphic organizer, or a drawing/diagram with explanation Worth 1-3 points Pros Allows for partial credit Provides more details about a student’s cognitive process Reduces the likelihood of guessing Cons Greater scoring subjectivity Requires more time to administer and score 39 Short Constructed Response Items

40. Requires students to apply knowledge, skills, and critical thinking abilities to real-world performance tasks by developing their own answers in the form of narrative text with supporting graphic organizers and/or illustrations Worth 4 or more points Entails more in-depth explanations than SCR items Pros Allows for partial credit Provides more details about a student’s cognitive process Reduces the likelihood of guessing Cons Greater scoring subjectivity Requires more time to administer and score 40 Extended Constructed Response Items

41. Depth of Knowledge is… The complexity of mental processing that must occur in order to construct an answer A critical factor in determining item/task rigor 41

42. Depth of Knowledge Chart 42

43. Process Steps 1.Review content standards from completed Targeted Content Standards Template and insert content strand(s) into specification table. 2.Determine the number of items by item type (i.e., Multiple Choice, Short Constructed Response, Extended Constructed Response) for each content strand. 3.Ensure item type and cognitive level (I, II, III)/depths of knowledge (DoK) are assigned. 4.Assign item weights to each item type. 5.Assign number of passages (by type) when using literary works. 43

44. QA Checklist There is a sufficient sampling of targeted standards. The specifications reflect a balance between developmental readiness and time constraints. Time is considered for both educators and students. The cognitive demands reflect those articulated in the targeted standards. The measure allows for both objective and subjective scoring procedures. The measure consists of 35-50 points with the Level I/DoK I limited to one-third of the items/tasks. 44

45. Blueprints Content ID # Content Statement Item Depth of Knowledge (DoK) Performance measures should reflect a variety of DoK levels. Sufficient sampling of content standards Aim for a 3:1 item to standard ratio (3 items for every standard). Cognitive load Aim for a balance of DoK levels among standards. Design measures with at least 50% DoK 2 or higher. 45

46. Blueprint Example 46

47. Process Steps 1.List the standards by number and statement in the appropriate columns. Remember to aim for a 3:1 item to standard ratio. 2.Decide on the item count for each standard and fill in the appropriate column. 3.Determine the number of DoKs for each standard following the specified guidelines for “rigor.” 4.Repeat Steps 1-3 ensuring that item and DoK counts meet the specification requirements. 47

48. The blueprint lists the content standard ID number. The blueprint lists or references the targeted content standards. The blueprint designates item counts for each standard. The blueprint reflects a range of DoK levels. The blueprint item/task distribution reflects that in the specification tables. 48 QA Checklist

49. All items/tasks articulated on the blueprint are represented within the Scoring Key. MC items have been validated to ensure only one correct answer among the possible options provided exists. MC answers do not create a discernible pattern. MC answers are “balanced” among the possible options. Scoring Key answers are revalidated after the final operational form reviews are complete. 49

50. 50 Scoring Rubrics

51. Holistic vs. Analytic Rubric Scoring Holistic Scoring Provides a single score based on an overall determination of the student’s performance Assesses a student’s response as a whole for the overall quality Most difficult to calibrate with different raters Analytic Scoring Identifies and assesses specific aspects of a response Multiple dimension scores are assigned Provides a logical combination of subscores to the overall assigned score 51

52. Rubric Scoring Considerations Describe whether spelling and/or grammar will impact the final score. Avoid using words like “many,” “some,” and “few” without adding numeric descriptors to quantify these terms. Avoid using words that are subjective, such as “creativity” or “effort.” Avoid subjective adjectives such as “excellent” or “inadequate.” 52

53. SCR Rubric Example 53 General Scoring Rubric 2 points The response gives evidence of a complete understanding of the problem. It is fully developed and clearly communicated. All parts of the problem are complete. There are no errors. 1 point The response gives evidence of a reasonable approach but also indicates gaps in conceptual understanding. Parts of the problem may be missing. The explanation may be incomplete. 0 points There is no response, or the work is completely incorrect or irrelevant.

54. SCR Rubric Example 54 Sample Response: “In two complete sentences, explain why people should help save the rainforests.” 2 points The student’s response is written in complete sentences and contains two valid reasons for saving the rainforest. “People must save the rainforest to save the animals’ homes. People need to save the rainforest because we get ingredients for many medicines from there.” 1 point The student’s response contains only one reason. “People should save the rainforest because it is important and because people and animals need it.”

55. Rubrics for ECR Tasks Create content-based descriptions of the expected answer for each level of performance on the rubric. Provide an example of a fully complete/correct response along with examples of partially correct responses. Reference the item expectations in the rubric. Make the rubric as clear and concise as possible so that other scorers would assign exact/adjacent scores to the performance/work under observation. 55

56. ECR Rubric Example 56 General Scoring Rubric 4 points The response provides all aspects of a complete interpretation and/or a correct solution. The response thoroughly addresses the points relevant to the concept or task. It provides strong evidence that information, reasoning, and conclusions have a definite logical relationship. It is clearly focused and organized, showing relevance to the concept, task, or solution process. 3 points The response provides the essential elements of an interpretation and/or a solution. It addresses the points relevant to the concept or task. It provides ample evidence that information, reasoning, and conclusions have a logical relationship. It is focused and organized, showing relevance to the concept, task, or solution process. 2 points The response provides a partial interpretation and/or solution. It somewhat addresses the points relevant to the concept or task. It provides some evidence that information, reasoning, and conclusions have a relationship. It is relevant to the concept and/or task, but there are gaps in focus and organization. 1 point The response provides an unclear, inaccurate interpretation and/or solution. It fails to address or omits significant aspects of the concept or task. It provides unrelated or unclear evidence that information, reasoning, and conclusions have a relationship. There is little evidence of focus or organization relevant to the concept, task, and/or solution process. 0 points The response does not meet the criteria required to earn one point. The student may have written on a different topic or written "I don't know."

57. ECR Rubric Example 57 Sample Response: “List the steps of the Scientific Method. Briefly explain each one.” 4 points 1.Ask a Question- Ask a question about something that you observe: How, What, When, Who, Which, Why, or Where? 2.Do Background Research- Use library and Internet research to help you find the best way to do things. 3.Construct a Hypothesis- Make an educated guess about how things work. 4.Test Your Hypothesis- Do an experiment. 5.Analyze Your Data and Draw a Conclusion- Collect your measurements and analyze them to see if your hypothesis is true or false. 6.Communicate Your Results- Publish a final report in a scientific journal or by presenting the results on a poster. 3 points 1.Ask a Question 2.Do Background Research-Use library and Internet research. 3.Construct a Hypothesis- An educated guess about how things work. 4.Test Your Hypothesis- Do an experiment. 5.Analyze Your Data and Draw a Conclusion 6.Communicate Your Results 2 points 1.Ask a Question 2.Do Background Research 3.Construct a Hypothesis 4.Test Your Hypothesis 5.Analyze Your Data and Draw a Conclusion 6.Communicate Your Results 1 point Ask a Question, Hypothesis, Do an Experiment, Analyze Your Data 0 points “I don’t know.”

58. QA Checklist CR items/tasks have scoring rubrics that reflect a performance continuum. CR items/tasks include sample responses for each level of performance. CR scoring rubrics are clear and concise. CR scoring rubrics include all dimensions (aspects) of the tasks presented to the students. CR scoring rubrics avoid including non-cognitive (motivation, effort, etc.) or content irrelevant attributes. Don’t use a rubric if a checklist or simpler tool will do! 58

59. Back to the SLO template…

60. SLO Template Steps: Teacher 5. Teacher Expectations 5a. Level Failing 0% to ___ % of students will meet the PI targets. Needs Improvement ___% to ___% of students will meet the PI targets. Proficient ___% to ___% of students will meet the PI targets. Distinguished ___% to 100% of students will meet the PI targets. 5b. Elective Rating Distinguished (3) Proficient (2) Needs Improvement (1) Failing (0) Notes/Explanation. Teacher Signature _________________________Date______ Evaluator Signature _____________________Date______. Teacher Signature _________________________Date______ Evaluator Signature _____________________Date______

61. Describes the number of students expected to meet the performance indicator criteria. 5a: Proficient 85% to 94% of students meet the performance indicator.

62. Proficient! 5a: Proficient 85% to 94% of this audience can explain the SLO process to their stakeholders!

63. SLO Online Resources pdesas.org IU8 Curriculum Network wiki Available Templates Available Rubrics Homeroom IU8 Wiggio (Online support group)

65. Key Process Points The SLO process facilitates a conversation about expectation between educators (principals and teachers)

66. Key Points (3) What is the subject or content focus? Who does it encompass? How can it improve instruction and educator practice? Goals- Standards Are they high quality measures? Who administers and scores the measures? What are the expectations for students? Performance Measures What are indicators of success? How are they being measured? Upon which students are they based? Indicators

67. 1.The SLO is based upon small numbers of students/data points. 2.Goals and indicators are linked to standards. 3.Indicators are vague without specific performance criteria. 4.Growth and/or mastery is not clearly defined 5.Performance measures are not well designed or lack rigor. 6.Overall student achievement expectations are extreme. IV. Areas of Caution

68. IV. Generic Process Steps: Leader 1.Establish SLO template completion timeline ASAP 2.Review complete template 3.Conduct review meeting with teacher 4.Agree on any revisions; submit materials 5.Establish “mid-cycle” spot review 6.End-of-Year review with supporting data

69. V. Action Planning Implementation Timeline Roles for administrative team Professional development for teacher Repurposing your schedule for job embedded SLO activities –SLO creation (Design, Build, Review) –Resource investigations –Time to work in grade level/content area teams Support systems for teachers from administrators and peers

70. Contact Info CURRICULUM Janel Vancas, Acting Assistant Director jvancas@iu08.org Laura J. Toki, Director ltoki@iu08.org ED PROGRAMS Jennifer Anderson, Assistant Director janderson@iu08.org

71. Frequently Asked Questions What are the definitions of “tested,” “non-tested?” Tested: Teachers with Eligible PVAAS Score (20% Elective) A PA certified educator with full or partial responsibility for content specific instruction of the assessed eligible content as measured by a Grade 4-8 PSSA or Keystone Exam. Non-tested: Teachers without Eligible PVAAS Score (35% Elective) Teachers who do not teach courses assessed by Grade 4-8 PSSA or Keystone exams.

72. Who develops the SLO? Is this an individual effort or a collaborative effort? Each educator will be responsible to develop SLOs as required by the LEA. Collaborative development of SLOs is encouraged (e.g., similar content area or grade level teachers, interdisciplinary groups of educators, collaboration through professional organizations educators, etc.). A PDE approved SLO Template is provided to help guide educators and administrators through the process. How will the final SLO measure be translated into a “score” that can be applied to the 20% or 35% of a teacher’s evaluation? This formula and computation process is currently under development by PDE and will be published in the PA Bulletin by June 30, 2013.

73. What is the SLO template and process designed to address in terms of instructional delivery time, number of students, or size of the objective? SLOs can be written to address the entire length of a grade or course, but could be tailored to a focused time period. Student achievement for large or select groups of students can be described. The template is designed to address a grade or course plan but could be used to address a meaningful, focused instructional objective or focused teaching practice. Will PDE recommend some performance measures and scoring tools? Model SLOs for a variety of content areas will be provided, utilizing a variety of performance measures and scoring tools. These models can be used as is or can be modified.

74. How many SLOs per teacher/per year/per grade? What about “co-taught” classes, teachers who travel between schools, and other unique instructional scenarios? Policy and guidelines on these issues are yet to be determined. How will the SLO process be monitored? A principal or LEA-assigned evaluator would monitor the SLO process, including (but not limited to) the timeline for development, approval for the SLO to be implemented and verification of the measure of educator effectiveness based on the completion of the SLO. Tools are currently being developed to assist principals toward efficiently and effectively monitoring this process.

75. How do “goals” and “performance indicators differ? The Goal Statement should address important learning content to be measured, and the performance indicators should describe expected levels of achievement. If a school is already having conversations about SLO and is having success, is it necessary to fill out this template or can we continue what we are doing? State regulations say that “LEAs shall use an SLO to document the process to determine and validate the weight assigned to the Elective Data measures that establish the Elective Rating.”

76. When will LEAs be expected to implement SLOs? Models will be available for school year 2013-2014, and LEAs have the option to use SLOs as a component for measuring educator effectiveness in school year 2013- 14. LEAs will be expected to implement SLOs in school year 2014-2015. First year teachers will not be expected to implement SLOs. What supports will be available to teachers and districts to develop and implement rigorous SLOs? An online training program and process/ definitions manual will be provided, as will an up-to-date template and content-specific models. Anticipated availability of these supports is August 2013.

77. Your Questions?