1. Music Teacher Evaluation in Michigan Dr. Phillip M. Hash, Calvin College pmh3@calvin.edu pmh3@calvin.edu February 14, 2013

2. Overview of PM Workshop 1.New Legislation 2.Current Trends 3.Evaluation strategies 4.Assessment Strategies 5.Your Experience

3. Legislative Review All Teachers Evaluated Annually Percentage of Evaluation to Relate to Student Growth National, State, And Local Assessments Evaluations vs. Seniority in Personnel Decisions Michigan Council On Educator Effectiveness 2

4. MDE Will Provide Measures For every educator, regardless of subject taught, based on 2009-10 and 2010-11 data: – Student growth levels in reading and math – Student proficiency levels in math, reading, writing, science, social studies – Foundational measure of student proficiency and improvement (same for each teacher in a school) Understanding Michigan's Educator Evaluations, MDE (December 2010) How will this data be used for arts educators? – Currently up to school districts – Might be specified by the state after this year

5. Performance-Based Compensation A district shall implement a compensation method for teachers and administrators that includes “job performance and job accomplishments as a significant factor” to determine “compensation and additional compensation.” MCL 380.1250(1) Meaning for arts educators?

6. New Prohibited Bargaining Subjects 1. Teacher Placement 2. Reduction in Force/Recall 3. Classroom Observation 4. Performance Evaluation 5. Teacher Discharge/Discipline 6. Performance-Based Compensation 7. Parent Notification

7. Pilot Programs 2012-13 Pilot – 14 districts – 4 evaluation models – Standardized tests – Local measures for non- tested subjects – Recommendations by 2013-14 school year 4

8. Current Trends in MI Teacher Evaluation

9. Frameworks, Methods, Systems Used as part of Local Evaluation

10. % Student Growth Counted in Teacher Evaluation (2011-12) % of Growth in Local Evaluation Systems

11. Current Trends: Effectiveness Ratings for 2011-12

12. Teacher Ratings & Student Growth

13. Evaluation Strategies

14. Always have lesson plans connecting to standards – See MI GLCE – Incorporate as many standards as make sense for your class – but not just perform and read notation Study the evaluation form Plan lessons using evaluation rubric as a guide Be prepared to provide evidence of instructional & professional practices – Student work, rubrics, lesson plans, parent call log, etc. Use a variety of instructional practices. Focus on student engagement. Don’t try to put on a show for evaluator [Is it time to reconsider the number of performances per year??]

15. Danielson Example

16. Student Engagement in Rehearsal http://cart.bravomusicinc.com/ http://cart.bravomusicinc.com/ http://www.youtube.com/watch?v=VY3nJXCh WrY (student led warm-ups - breathing) http://www.youtube.com/watch?v=VY3nJXCh WrY http://www.youtube.com/watch?v=tgdksldrw kc (chorale) http://www.youtube.com/watch?v=tgdksldrw kc http://www.youtube.com/watch?v=_eOjYt1- 4-0 (student sectionals - feedback) http://www.youtube.com/watch?v=_eOjYt1- 4-0

17. Developing Local Assessment Strategies

18. Creating an Assessment Plan District Music Faculty (by area) – Est. curriculum based on MI Standards What should students in each grade level know and be able to do? How and when will objectives be assessed? – Perhaps not every grade every year How will assessments show growth? (e.g., difference in % b/w pre- post test, defined NP, PP, P, HP?) Take plan to administration for approval – Law says that “with the involvement of teachers” Pilot, Review, Revise, Implement

19. MI Grade Level Content Expectations (June 2011) What students should know and be able to do in grades K-8, & HS Aligned w/ VPAA & 21 st century skills Standards, & benchmarks by grade level Teachers evaluated on use of standards [See handout]

20. Assessment Terms Reliability = Consistency – Test/retest (regardless of yr., location, etc.) – Interrater (every judge the same) Validity = the extent to which an assessment measures what they purport to measure Authentic Assessment = Students demonstrate knowledge and skills in real-world context (e.g., performance) Quantitative – data is numerical (anything that can be counted, percentages) Qualitative – data is in words (descriptions, written critiques) Formative vs. Summative – Formal vs. Informal -

21. Assessment Terms - RTTT Rigorous – assessments that measure grade-level standards Two points in time – pre- & post-test – Proficiency from one year to the next – Ongoing assessments of musical skills (steady beat, pitch matching, singing, recorder, instrumental performance, sight-reading, etc.) Comparable across classrooms – same for all teachers at a particular level or area – Assessments comparable in rigor to other subjects

22. Student Growth Measures

23. Rubistar http://rubistar.4teachers.org/ http://rubistar.4teachers.org/ Create rubrics using existing descriptors Search other teachers’ rubrics for samples – Edit to fit your needs

24. Rubrics Types include: – Holistic (overall performance) – Analytic (specific dimensions of performance) – Additive Descriptors must be valid (meaningful) Scores – Must be reliable (consistent) – Should relate to actual levels of students learning Can be used by students for self-assessment and to assess the performance of other students Give to students b/f assessment 14

25. What does a rubric look like? BeginningBasicProficientAdvanced TONEBreathy; Unclear; Lacks focus; Unsupported Inconsistent; Beginning to be centered and clear; Breath support needs improvement Consistent breath support; Centered and clear; Beginning to be resonant Resonant; Centered; Vibrant; Projecting Adapted from: K. Dirth, Instituting Portfolio Assessment in Performing Ensembles, NYSSMA Winter Conference, Dec. 2, 1997. Features: Scale includes rating points (at least 4). See handout for sample headings Highest point represents exemplary performance Criterion—based categories Descriptors are provided for each level of student performance Pre- and/or Post-test. Teacher, peer, & self assessment 13

26. Holistic Rubric

28. Piano Rubric - Analytic Quiz #1 Scales Two octaves, hands together, ascending and descendingKeys ____________ 1/6/12

29. Sample Rating Scale 12

30. Showing Growth w/ Rubrics (or any other pre- post-test) Pre- & post-test average class posttest % - average class pretest % = % growth PostPre% growth 675710 796514 593227 908010 827210 584513 72.558.514

31. Est. Personal Reliability Record 10 students Grade w/ rubric Grade again in 2 weeks Measure the difference in score for each recording Calculate average difference Lower = better Trial 1Trial 2Difference 990 671 862 11101 972 12102 440 Av. Diff. 1.14

32. Rate these 6 recorder performances on a scale of 1-12 Rate the same examples using rubric in handout Trial 1 1 _____ 2 _____ 3 _____ 4 _____ 5 _____ 6 _____

33. Recorder Trial 2 Use rubric on loose sheet Add up score Match score from Trial 1 to Scores from Trial 2 Is there a difference? In which scores are you most confident?

34. Elementary General Music – Grade 3 Pre- & Post Test Sample [See handout] Paper/pencil, but relies on musical response Prompts can be different for pre-test Pre-test can be an abbreviated version Require 2-3 class periods to complete Music supervisor could issue musical examples & prompts before the test (avoid teaching to the test)

35. Creating Similar Elementary General Music Assessment For grades 3-5, determine what GLCEs can be measured through paper/pencil response Create question(s) for each benchmark – deliberately connect question to GLCEs (validity, rigor, comparable a/c classrooms) Decide # of questions needed to determine competency Create questions that fit different prompts

36. Performing Ensembles Semester Exam [see handout] Jason Lowe – Bay City HS Bands Mandy Smith – Rockford HS Choirs

37. Watkins – Farnum Performance Scale Sight reading – band Published by Hal Leonard Reliable & valid assessment Forms A & B Easy to score as per directions in handout 14 exercises worth X pts. Score until student earns 0 on 2 consecutive exercises

38. Royal Conservatory Music Development Program (see handout) Recorder, strings, woodwinds, brass, percussion, voice Graded preparatory, 1-10 – RC Grade 8 considered college entrance Includes solos, etudes, scales/arpeggios, ear training, sight reading, theory Curricula online Adapt for your program

39. Performing Ensembles

40. Excellence in Theory or Standard of Excellence Music Theory & History Workbooks Kjos - publisher 3 volumes (see handout sample) Includes theory, ear training, history Take MS & HS to complete 3 volumes Students work on lessons during down time in rehearsal Establish grade level expectations and written exam

41. Insuring Integrity

42. Self created, administered, and graded assessments Colleagues & administrators will ask Standards Based assessments Comparable across classrooms Demonstrate validity & reliability – Explain/demonstrate process for creating, administering, & grading – Demonstrate connection b/w state standards and assessments – Archive recordings

43. www.vocaroo.com Audio emails Archived up to 5 months Sends link to an email address Download as.WAV or.Ogg Useful for performance tests Very easy! http://vocaroo.com/?media=vAdx5RJr1DVC7upIc

44. Festival Ratings

45. NAfME Position Statement Successful music teacher evaluation must, where the most easily observable outcomes of student learning in music are customarily measured in a collective manner (e.g., adjudicated ratings of large ensemble performances), limit the use of these data to valid and reliable measures and should form only part of a teacher’s evaluation. (NAfME, 2011)

46. Festival Ratings: Advantages Provide quantitative third party assessment Can show growth over time in some circumstances – Individual judges’ ratings – Repertoire difficulty – 3 yr. period Valid to the extent that they measure the quality of an ensemble’s performance of three selected pieces & sight reading at one point in time Likely reliable over 3-yr. period based on previous research Probably adaptable to state-wide evaluation tool Assess a few performance standards

47. Ratings Growth Example Hypothetical Contest Ratings for One Ensemble over a Three-year Period Note. Roman numerals represent division ratings. a Total increase from year 1 to year 3 = 44%. Judge 1Judge 2Judge 3 Sight- Reading Average Annual Increase a Final Year 1IIIIIII 2.25-2 Year 2II I 1.7522%2 Year 3IIIII1.2529%1

48. Ratings ≠ MEAP or MME Exams MEAP & MME Same for all each yr. Rel. and val. established Many Standards Individual Mostly objective Reflect multiple levels of achievement Ratings Rep., adj. change Val. & rel. not est. Per. standards only Group Mostly subjective 90%+ earn I or II out of V ratings.

49. Festival/Contest Ratings: Challenges Reliability Curricular limitations Score Inflation Ratings Effectiveness in differentiating quality Influence of non-performance factors Group vs. Individual performance Other factors Role of MSBOA & MSVMA?

50. Experiences

51. Describe Your Situation In roundtables by area? How are you measuring student growth at your school? What support are you getting? What needs or concerns do you have?