Using Classroom Portfolios to Evaluate Arts Educators An Alternate Growth Measures System Dr. Ryan Fisher Music Education Division Head, University of Memphis Music Education Division Head, University of Memphis

What Prompted their Journey?

Options For Non-Tested Subjects? A. Measures of Collective Performance (TN) B. Student Learning Objectives Systems C. The Development/Adaptations of Other Assessments How do you provide measurable data while staying true to the holistic process of the arts?

D: None of the Above THE GOAL: A holistic and meaningful picture of the value teachers add to students, using the work that is already happening in classrooms. OUR SOLUTION: A flexible but rigorous portfolio of student work samples that demonstrate growth across standards-based learning domains. Actually… A Hybrid Approach

Stakeholders and Development Timeline Portfolio Peer Review System TN Fine Arts System Development Committee TDOE & Memphis City Schools 600 Fine Arts Teachers NAfME, Consultants, The Gladis Project, & Hope Street Group Nonprofit NAfME, Consultants, The Gladis Project, & Hope Street Group Nonprofit The Arts Community

Oct Jan 2011 Fine Arts Committee Created & Submitted Recommendations to the TN Dept. of Ed Fall 2011 System is Beta Tested then Piloted with 450 Teachers, Scoring Guides Created Spring 2012 Portfolio Submission and Peer Review Process, Data Report to TDOE, Meetings at USDOE Summer 2012 District Trainings at TN Arts Academy Fall 2012 TN Board of Ed Approval- Three Districts Implement More Meetings w USDOE Spring 2013 Peer Reviewers Trained Teachers Portfolios Scored, Evaluation Profiles Populated Fall ,500 Users With Individual Growth Scores for Evaluation, Feedback Loop Developed Stakeholders and Development Timeline

What Makes it Work? Teacher submits 5 evidence collections which include a “purposeful sampling” of students Each collection contains evidence of student growth (pre and post lesson/unit/year), and learning objectives/targets/supporting evidence Self-scored, then rated by content specific peer reviewer Built in secondary peer review in case of significant disagreement between teacher and first reviewer

Purposeful Sampling- A Key Element Meaningful representation of the classes and students Standards-Based Collections (Perform, Create, Respond, Connect… 3 of 4) Should be reflective of teacher course load Evidence of Differentiation…showing how you impact students of various learning levels

Self Portrait Growth Sample

Pre and Post Finding the Core…. Unit Project Skill or Concept Unit Project Skill or Concept

GLADiS Project Cloud-Based Evidence Collection Tool Allows for Double Blind Peer Review Tracks Rater Reliability and Monitors Audits Logistic Support

What are the Benefits? Flexible, yet rigorous method of measuring authentic student performances/products Combines evaluation with professional growth Creates additional career opportunities for teacher leaders Equips NTGS educators with the tools to advocate for a well- rounded student experience Cost Efficient

What are the Benefits? School-wide vs. individual growth findings from 2013 data Limited correlation between school-wide scores and arts teachers’ individually attributable growth School-wide scores = abnormal distribution (negatively skewed) Alternate growth measure = normal distribution 22.5% would have received the same score 54.1% received a lower score than school-wide score (over 50% of those schools assigned a score of 5) 23.4% received a higher score than school-wide score (many of which were Level 5 teachers who would have been denied recognition as master teachers)

What are the Required Pre-Conditions? The process must be time efficient for educators The implementation must account for technology limitations Flexibility embedded to account for inequity of resources/class time/curricular support across districts The peer review portion must be fair, consistent, rigorous, and provide teachers with feedback that allows for professional growth The system must expand access for quality arts education

Goals for Portfolio Review Increased Access to the NTGS “Core” Subject Areas Improved Instructional Practice Teachers Treated as Experts/Leaders Spread of Scores Increased Achievement in each subject area. Meaningful Generalized Gains Increased Student Creativity, Collaboration, Innovation

TN Arts Academy Panel Discussion Summary Response to some claims that the portfolio model isn’t fair since the portfolio model doesn’t mirror the standardized test model other teachers are evaluated by? Some tested subject teachers view portfolio “too soft” of an indicator. Many tested-subject teachers are envious and wish they could use this type of model – feel it’s more authentic to what they’re doing in their classroom. Some tested-subject teachers are not even aware arts teachers use a portfolio for student growth.

TN Arts Academy Panel Discussion Summary What are the weaknesses of the model or what are the problems teachers have encountered? Attitude – barrier they need help getting over – arts administrators and supervisors need to motivate them and help them get over the barrier in order to at least submit a portfolio that truly reflects their teaching. Many teachers who originally fought against this model have seen its benefit and its focus on showing success of students and putting students first through this process. Dropdown menu of feedback from reviewer that isn’t specific enough to the teacher – more specific feedback could help develop/improve teachers. Need more time/support to develop portfolio throughout year. Peer review – they would like more input in feedback (submit action plan to teachers)

TN Arts Academy Panel Discussion Summary What do you feel are the strengths of this portfolio model? Improved my teaching Professional development aspect (putting together the portfolio forces you to think about what you are doing in the classroom – more purposeful!) Peer assessors also learn and develop from evaluating portfolios of other teachers. Truly demonstrates the various levels of teachers (those who wait until last minute, those who focus throughout the year, etc.) Reinforces standards accountability Purposive sampling – teacher shows growth in students of various achievement levels Principals know arts educators have things they have to get accomplished in classroom to show in student growth measures, so they protect class time in arts more. Fewer pullouts (or they at least seek input/permission from that arts teacher). More legitimized now since there is a way to test student growth. Portfolio adoption seems to bring arts educators together for professional development and help in developing these portfolios.

TN Arts Academy Panel Discussion Summary What changes have been made or need to be made going forward? Peer reviewers now have to take a quiz after training in order to improve reviewer accountability. 4-5 collections (based on feedback) Co-teaching situations – focusing on areas in which they had the most impact (select different students though) – developing best practices for co-teaching situations. Scoring guide revisions. Teacher’s guide updated this year. Minor tweaks to GLADIS to make it more user-friendly for submitters and peer reviewers. Providing more examples for teachers in training.

Future of Portfolio Model North Carolina adapting TN Portfolio model for their non-tested subject teachers TN Physical Education teachers now exploring portfolio model Shelby County School Kindergarten teachers are piloting a portfolio model this coming school year

Using Classroom Portfolios to Evaluate Arts Educators An Alternate Growth Measures System Dr. Ryan Fisher Music Education Division Head, University of Memphis Music Education Division Head, University of Memphis Or contact: Dr. Dru Davison foliomodel/home