1. www.engageNY.org New York State’s Teacher and Principal Evaluation System VOLUME I: NYSED APPR PLAN SUBMISSION “TIPS”

2. www.engageNY.org Module Objectives Understand the common errors entered into Review Room by districts/BOCES. Understand ways in which districts/BOCES can avoid similar errors when using the Review Room functions. 2

3. www.engageNY.org Key Tools and Resources When Following This Module All of the following resources are located on EngageNY.org: APPR TIPS Memo Summary of regulations (i.e., “the purple memo”) APPR Guidance (see Section C, in particular) TLE Roadmaps SLO Guidance, Exemplars, and Webinars 3

4. www.engageNY.org If You Received a Letter That Included The Following… 4 Your district did not enter in the assessment box the name of the assessment corresponding to the valid option selected, followed by the words "unless [OFFICIAL NAME OF 3rd PARTY ASSESSMENT] is a State-approved 3rd party assessment."

5. www.engageNY.org Tip 1: 3 rd Party Assessments not yet on the State-Approved List 5 It is possible to describe an alternative plan for using a 3 rd party assessment that has not yet been approved by following the directions outlined in Tip 1.

6. www.engageNY.org Tip 1: 3 rd Party Assessments not yet on the State-Approved List (cont.) 6 Step 1: choose an approved alternative option. Step 2: in the assessment box, enter the name of the corresponding assessment Step 3: followed by the words, “unless [OFFICIAL NAME OF 3 rd PARTY ASSESSMENT] is a State- approved 3 rd party assessment.”

7. www.engageNY.org If You Received a Letter That Included The Following… 7 The name of the assessment is incomplete. Assessment names must include the grade and subject level. See Regents Rules §30- 2.5(c)(3)(i); APPR Guidance Document C20.

8. www.engageNY.org Tip 2: Incomplete/Inconsistent Assessment Names 8 Remember to double check that you have: 1.Named the specific assessment you are using. 2.Named the grade and subject level of the assessment. 3.Named an assessment that is consistent with the option selected from the drop-down menu. Remember to double check that you have: 1.Named the specific assessment you are using. 2.Named the grade and subject level of the assessment. 3.Named an assessment that is consistent with the option selected from the drop-down menu. For example, a regionally-developed 7 th grade Social Studies assessment would be written as the following within Review Room: XX NAMED Region-developed 7 th grade Social Studies assessment.

9. www.engageNY.org If You Received a Letter That Included The Following… 9 In Task 4.2 [9.4 for principals] your district assigned points to “State-Approved Survey Tools” but did not check the box in Task 4.3 [9.5 for principals] assuring that your district would use survey tools from the State-approved list or the approved NYSED survey variance process. See Education Law §3012-c(2)(h)(3)(iii); Regents Rules §30-2.5(d)(1)(iv)(c); APPR Guidance Document H13.

10. www.engageNY.org Tip 3: Student Surveys for use in “Other Measures” NYSED is regularly updating the surveys on the State-approved list. Follow Tip 2 if you decide to use surveys for teachers and/or principals. Remember to check the box in Task 4.3 if your district has assigned points to “State- approved survey tools.”

11. www.engageNY.org Tip 3: Student Surveys for use in “Other Measures” (cont.) Keep in Mind: Districts/BOCES will also need to upload an alternative points distribution for the Other Measures for Principals in case no principal surveys are approved by the State. Keep in Mind: Districts/BOCES will also need to upload an alternative points distribution for the Other Measures for Principals in case no principal surveys are approved by the State. Remember to check the assurance that you will use a State-approved survey when and if one is approved for principals.

12. www.engageNY.org If You Received a Letter That Included One or More of The Following… 12 Your district chose control factors other than poverty, ELL status, SWD status, or prior academic history. These factors are the only control factors allowed for the student growth subcomponent. See APPR Guidance Document D22. Your district’s rationale does not explain/address why your district chose each control factor. See APPR Guidance Document D22, E6, E7. Your district’s response does not describe the process used to mitigate potential problematic incentives. See APPR Guidance Document D22, E7. Your district listed control factors that make all points, specifically zero (0), unavailable. See Regents Rules §30-2.6(e)(4).

13. www.engageNY.org Tip 4: Using Adjustment Factors and Controls for State Growth or Locally-selected 13 See Guidance D22, E6, and E7 for a list of control variables that may be used for each subcomponent. Keep in mind that there is more than one way to use control variables in Growth or Locally-selected measures.

14. www.engageNY.org Tip 4: Using Adjustment Factors and Controls for State Growth or Locally-selected (cont.) 14 Keep in Mind: Districts/BOCES must also make a number of assurances about the use and application of any locally developed controls. Keep in Mind: Districts/BOCES must also make a number of assurances about the use and application of any locally developed controls.

15. www.engageNY.org Tip 4: Using Adjustment Factors and Controls for State Growth or Locally-selected (cont.) 15 One option is to add extra HEDI points based solely on the composition of the class or school – this is not the only option. If a school district or BOCES does assign additional points to certain educators based on the control variables, the assignment of points must still effectively differentiate educator performance in each of these subcomponents and in their overall ratings. The maximum allowable additional points under the State Growth or Locally-selected subcomponents is TWO POINTS.

16. www.engageNY.org If You Received a Letter That Included One or More of The Following… 16 The values specified in the HEDI categories and points assignment are non-consecutive and/or overlap. See Regents Rules §30-2.6(e); APPR Guidance Document I2. The description in the boxes is inconsistent with the description in your attachment. See Regents Rules §30- 2.6(e). Your district’s general description of HEDI categories does not sufficiently or clearly describe how a teacher will earn each of the four (4) HEDI rating categories. See Regents Rules §30-2.6(e); APPR Guidance Document I2. District expectations regarding the level of performance needed for each of the four (4) HEDI categories are not sufficiently defined. See Regents Rules §30-2.3(b)(4), §30- 2.6(e).

17. www.engageNY.org Tip 5: HEDI Criteria Sections 17 Be sure the points and scoring ranges in the HEDI charts do not overlap. Look at the scoring ranges in the top chart. Notice how there is an overlap.

18. www.engageNY.org Tip 5: HEDI Criteria Sections (cont.) 18 Remember to clearly explain how points will be assigned within each of the 4 HEDI rating categories. Be sure the descriptions and criteria in the HEDI boxes match any attachments you upload.

19. www.engageNY.org If You Received a Letter That Included One or More of The Following… 19 Your district’s HEDI scoring ranges do not match the Commissioner’s scoring ranges. See Regents Rules §30-2.6(b).

20. www.engageNY.org Tip 6: Points Available Based on Growth on Comparable Measures (SLOs) 20 SLOs as Comparable Growth measures can only earn from 0-20 points. The scoring ranges will never exceed 0-20 points for SLOs. When the Board of Regents approves a Value-added model for use as the State-provided growth measures in grades 4-8 ELA and Math, (or other grades/subjects in the future), the scoring range for State Growth for those educators will count for up to 25 points; the comparable growth measures for all other educators who do not have State-provided measures will continue to count for up to 20 points. Even when a teacher has some grades/subjects with State-provided Value-added measures but he/she does not cover at least 50% of the teacher’s students, the scoring range for the teacher’s Growth subcomponent will be 0-20 points, not 0-25 points.

21. www.engageNY.org If You Received a Letter That Included One or More of The Following… 21 Your district listed at least one school that does not contain any students in grades four (4) or above. School configurations that do not include grades four (4) and above do not have a Value-added score and should not be included in this task. See Regents Rules §30-2.5(b)(2); APPR Guidance Document D1. Your district has omitted a grade configuration in this Task that has a Value-added score. Grade configurations having a Value-added score must be included in this Task. See Regents Rules §30-2.5(b)(2). Your district’s HEDI submission does not measure growth. See Education Law §3012-c(2)(f)(1); Regents Rules §30-2.6(b).

22. www.engageNY.org Tip 7: Principal Evaluation: State Growth, Comparable Growth, and Locally-selected Measures of Student Achievement For high school principals, the State expects to have an approved Value-added Growth score for State Growth in 2012-13. 22 In Task 7.1, remember to also list all of the principals with high school grade configurations as those who will have Value-added measures.

23. www.engageNY.org Tip 7: Principal Evaluation: State Growth, Comparable Growth, and Locally-selected Measures of Student Achievement (cont.) 23 Be sure that the grade configurations listed in Task 7.3 (shown on top) match those listed in Task 8.2 (shown below)

24. www.engageNY.org Tip 7: Principal Evaluation: State Growth, Comparable Growth, and Locally-selected Measures of Student Achievement (cont.) Keep In Mind: Principal Comparable Growth (SLOs) and Principal Locally-selected measures of student growth or achievement must be based on student performance. These measures may not be based on teacher performance (e.g. 70% of teachers earn Effective on their Locally-selected measures is not acceptable for principals). 24

25. www.engageNY.org If You Received a Letter That Included One or More of The Following… 25 Your district selected a group/team/BOCES-wide measure but failed to name a State assessment in the free-form text box. See Regents Rules §30- 2.5(b)(1)(iv)(d). The name of the assessment is inconsistent with the option selected from the drop-down menu. See Regents Rules §30-2.5(c)(3)(i); APPR Guidance Document C20.

26. www.engageNY.org Graduation rates (including schoo lwide- growth or local measures constructed using graduation rates) are not an allowable option for the State Growth SLOs or Locally-selected subcomponent for teachers. See D58 of APPR Guidance. See D59 and D60 of APPR Guidance for allowable examples of how school-or BOCES-wide, group, or team measures could work for teachers. Tip 8: Graduation Rates for Growth or Locally- selected Subcomponents for Teachers 26

27. www.engageNY.org If You Received a Letter That Included One or More of The Following… 27 The description for training and certification of lead evaluators does not include a process for certifying and periodically recertifying all lead evaluators. See Education Law §3012-c(2)(d); Regents Rules §30-2.9(c) & (d); APPR Guidance Document J2 and J4. The description does not include a process for ensuring that lead evaluators maintain inter-rater reliability over time. See Education Law §3012-c(2)(d); Regents Rules §30-2.9; APPR Guidance Document J2. The description of training does not include the duration of the training your district provides to evaluators and lead evaluators. See Education Law §3012- c(2)(d); Regents Rules §30-2.9; APPR Guidance Document J2. The description of training does not specify the nature of the training your district provides to evaluators and lead evaluators. See Education Law §3012-c(2)(d); Regents Rules §30-2.9; APPR Guidance Document J2. The description your district submitted is unclear and/or lacks sufficient detail. See Education Law §3012-c(2)(d); Regents Rules §30-2.9; APPR Guidance Document J2. Your district listed the training requirements for lead evaluators and failed to include all the minimum requirements. See Regents Rules §30-2.9(b).

28. www.engageNY.org Tip 9: Training and Certification of Lead Evaluators In Tasks 6.4 and 11.4, make sure to describe, in detail: the duration and nature of evaluator training; the process for certifying lead evaluators; the process for re-certifying lead evaluators; and the process for ensuring inter-rater reliability. 28

29. www.engageNY.org Tip 10: Proofreading 29 Be careful to… Re-label as needed if you are copying similar answers from one section to another (e.g., avoid referring to each of the four HEDI categories as “highly effective” or referring to teachers in the principal evaluation sections). Include the grade level and subject for each assessment named in your submission. Verify these are spelled correctly. Always proofread thoroughly!

30. www.engageNY.org 30 Our goal is to help every student graduate ready for college and careers. The APPR process is an important step toward that goal. Send your questions to educatoreval@mail.nysed.gov.