Music Teacher Evaluation in Michigan Dr. Phillip M. Hash, Calvin College February 14, 2013

Overview of PM Workshop 1.New Legislation 2.Current Trends 3.Evaluation strategies 4.Assessment Strategies 5.Your Experience

Legislative Review All Teachers Evaluated Annually Percentage of Evaluation to Relate to Student Growth National, State, And Local Assessments Evaluations vs. Seniority in Personnel Decisions Michigan Council On Educator Effectiveness 2

MDE Will Provide Measures For every educator, regardless of subject taught, based on and data: – Student growth levels in reading and math – Student proficiency levels in math, reading, writing, science, social studies – Foundational measure of student proficiency and improvement (same for each teacher in a school) Understanding Michigan's Educator Evaluations, MDE (December 2010) How will this data be used for arts educators? – Currently up to school districts – Might be specified by the state after this year

Performance-Based Compensation A district shall implement a compensation method for teachers and administrators that includes “job performance and job accomplishments as a significant factor” to determine “compensation and additional compensation.” MCL (1) Meaning for arts educators?

New Prohibited Bargaining Subjects 1. Teacher Placement 2. Reduction in Force/Recall 3. Classroom Observation 4. Performance Evaluation 5. Teacher Discharge/Discipline 6. Performance-Based Compensation 7. Parent Notification

Pilot Programs Pilot – 14 districts – 4 evaluation models – Standardized tests – Local measures for non- tested subjects – Recommendations by school year 4

Current Trends in MI Teacher Evaluation

Frameworks, Methods, Systems Used as part of Local Evaluation

% Student Growth Counted in Teacher Evaluation ( ) % of Growth in Local Evaluation Systems

Current Trends: Effectiveness Ratings for

Teacher Ratings & Student Growth

Evaluation Strategies

Always have lesson plans connecting to standards – See MI GLCE – Incorporate as many standards as make sense for your class – but not just perform and read notation Study the evaluation form Plan lessons using evaluation rubric as a guide Be prepared to provide evidence of instructional & professional practices – Student work, rubrics, lesson plans, parent call log, etc. Use a variety of instructional practices. Focus on student engagement. Don’t try to put on a show for evaluator [Is it time to reconsider the number of performances per year??]

Danielson Example

Student Engagement in Rehearsal WrY (student led warm-ups - breathing) WrY kc (chorale) kc (student sectionals - feedback) 4-0

Developing Local Assessment Strategies

Creating an Assessment Plan District Music Faculty (by area) – Est. curriculum based on MI Standards What should students in each grade level know and be able to do? How and when will objectives be assessed? – Perhaps not every grade every year How will assessments show growth? (e.g., difference in % b/w pre- post test, defined NP, PP, P, HP?) Take plan to administration for approval – Law says that “with the involvement of teachers” Pilot, Review, Revise, Implement

MI Grade Level Content Expectations (June 2011) What students should know and be able to do in grades K-8, & HS Aligned w/ VPAA & 21 st century skills Standards, & benchmarks by grade level Teachers evaluated on use of standards [See handout]

Assessment Terms Reliability = Consistency – Test/retest (regardless of yr., location, etc.) – Interrater (every judge the same) Validity = the extent to which an assessment measures what they purport to measure Authentic Assessment = Students demonstrate knowledge and skills in real-world context (e.g., performance) Quantitative – data is numerical (anything that can be counted, percentages) Qualitative – data is in words (descriptions, written critiques) Formative vs. Summative – Formal vs. Informal -

Assessment Terms - RTTT Rigorous – assessments that measure grade-level standards Two points in time – pre- & post-test – Proficiency from one year to the next – Ongoing assessments of musical skills (steady beat, pitch matching, singing, recorder, instrumental performance, sight-reading, etc.) Comparable across classrooms – same for all teachers at a particular level or area – Assessments comparable in rigor to other subjects

Student Growth Measures

Rubistar Create rubrics using existing descriptors Search other teachers’ rubrics for samples – Edit to fit your needs

Rubrics Types include: – Holistic (overall performance) – Analytic (specific dimensions of performance) – Additive Descriptors must be valid (meaningful) Scores – Must be reliable (consistent) – Should relate to actual levels of students learning Can be used by students for self-assessment and to assess the performance of other students Give to students b/f assessment 14

What does a rubric look like? BeginningBasicProficientAdvanced TONEBreathy; Unclear; Lacks focus; Unsupported Inconsistent; Beginning to be centered and clear; Breath support needs improvement Consistent breath support; Centered and clear; Beginning to be resonant Resonant; Centered; Vibrant; Projecting Adapted from: K. Dirth, Instituting Portfolio Assessment in Performing Ensembles, NYSSMA Winter Conference, Dec. 2, Features: Scale includes rating points (at least 4). See handout for sample headings Highest point represents exemplary performance Criterion—based categories Descriptors are provided for each level of student performance Pre- and/or Post-test. Teacher, peer, & self assessment 13

Holistic Rubric

Piano Rubric - Analytic Quiz #1 Scales Two octaves, hands together, ascending and descendingKeys ____________ 1/6/12

Sample Rating Scale 12

Showing Growth w/ Rubrics (or any other pre- post-test) Pre- & post-test average class posttest % - average class pretest % = % growth PostPre% growth

Est. Personal Reliability Record 10 students Grade w/ rubric Grade again in 2 weeks Measure the difference in score for each recording Calculate average difference Lower = better Trial 1Trial 2Difference Av. Diff. 1.14

Rate these 6 recorder performances on a scale of 1-12 Rate the same examples using rubric in handout Trial 1 1 _____ 2 _____ 3 _____ 4 _____ 5 _____ 6 _____

Recorder Trial 2 Use rubric on loose sheet Add up score Match score from Trial 1 to Scores from Trial 2 Is there a difference? In which scores are you most confident?

Elementary General Music – Grade 3 Pre- & Post Test Sample [See handout] Paper/pencil, but relies on musical response Prompts can be different for pre-test Pre-test can be an abbreviated version Require 2-3 class periods to complete Music supervisor could issue musical examples & prompts before the test (avoid teaching to the test)

Creating Similar Elementary General Music Assessment For grades 3-5, determine what GLCEs can be measured through paper/pencil response Create question(s) for each benchmark – deliberately connect question to GLCEs (validity, rigor, comparable a/c classrooms) Decide # of questions needed to determine competency Create questions that fit different prompts

Performing Ensembles Semester Exam [see handout] Jason Lowe – Bay City HS Bands Mandy Smith – Rockford HS Choirs

Watkins – Farnum Performance Scale Sight reading – band Published by Hal Leonard Reliable & valid assessment Forms A & B Easy to score as per directions in handout 14 exercises worth X pts. Score until student earns 0 on 2 consecutive exercises

Royal Conservatory Music Development Program (see handout) Recorder, strings, woodwinds, brass, percussion, voice Graded preparatory, 1-10 – RC Grade 8 considered college entrance Includes solos, etudes, scales/arpeggios, ear training, sight reading, theory Curricula online Adapt for your program

Performing Ensembles

Excellence in Theory or Standard of Excellence Music Theory & History Workbooks Kjos - publisher 3 volumes (see handout sample) Includes theory, ear training, history Take MS & HS to complete 3 volumes Students work on lessons during down time in rehearsal Establish grade level expectations and written exam

Insuring Integrity

Self created, administered, and graded assessments Colleagues & administrators will ask Standards Based assessments Comparable across classrooms Demonstrate validity & reliability – Explain/demonstrate process for creating, administering, & grading – Demonstrate connection b/w state standards and assessments – Archive recordings

Audio s Archived up to 5 months Sends link to an address Download as.WAV or.Ogg Useful for performance tests Very easy!

Festival Ratings

NAfME Position Statement Successful music teacher evaluation must, where the most easily observable outcomes of student learning in music are customarily measured in a collective manner (e.g., adjudicated ratings of large ensemble performances), limit the use of these data to valid and reliable measures and should form only part of a teacher’s evaluation. (NAfME, 2011)

Festival Ratings: Advantages Provide quantitative third party assessment Can show growth over time in some circumstances – Individual judges’ ratings – Repertoire difficulty – 3 yr. period Valid to the extent that they measure the quality of an ensemble’s performance of three selected pieces & sight reading at one point in time Likely reliable over 3-yr. period based on previous research Probably adaptable to state-wide evaluation tool Assess a few performance standards

Ratings Growth Example Hypothetical Contest Ratings for One Ensemble over a Three-year Period Note. Roman numerals represent division ratings. a Total increase from year 1 to year 3 = 44%. Judge 1Judge 2Judge 3 Sight- Reading Average Annual Increase a Final Year 1IIIIIII Year 2II I %2 Year 3IIIII1.2529%1

Ratings ≠ MEAP or MME Exams MEAP & MME Same for all each yr. Rel. and val. established Many Standards Individual Mostly objective Reflect multiple levels of achievement Ratings Rep., adj. change Val. & rel. not est. Per. standards only Group Mostly subjective 90%+ earn I or II out of V ratings.

Festival/Contest Ratings: Challenges Reliability Curricular limitations Score Inflation Ratings Effectiveness in differentiating quality Influence of non-performance factors Group vs. Individual performance Other factors Role of MSBOA & MSVMA?

Experiences

Describe Your Situation In roundtables by area? How are you measuring student growth at your school? What support are you getting? What needs or concerns do you have?