1. Michigan Arts Education Instruction and Assessment
Welcome to this Introduction to the Michigan Arts Education Instruction and Assessment Project.
In this module, I will describe some of the important characteristics of this ground-breaking project and introduce you to the resources you now have available to you.
Demonstrating Educator Effectiveness
Pilot Project Final Meeting

2. Introductions Local Educators MAC Staff Kathy Dewsbury-White
Jason O’Donnell
Ed Roeber
Heather Vaughan-Southard
Linda Wacyk

3. Overview of the Day Review of MAEIA Project Expectations
Overview of Available MAEIA DEE Activities & Resources
Reports of DEE Activities from DEE Project Participants
Actual Educator Effectiveness Teacher/Supervisor Meeting
Review of the Demonstrating the Effectiveness of Educators
Interest in Continued DEE Work in
Welcome to this Introduction to the Michigan Arts Education Instruction and Assessment Project.
In this module, I will describe some of the important characteristics of this ground-breaking project and introduce you to the resources you now have available to you.

4. Overview of the Pilot Project
Worked with volunteer arts educators and their supervisors interested in using MAEIA assessments to demonstrate their effectiveness
Learned about MAEIA assessment models
Examined pilot SLO work of a couple arts educators
Reviewed P.A. 173 of 2015
Considered educator evaluation plans
Continued learning throughout the school year
Meeting in the spring to advise one another (and MAEIA) on what has worked
Welcome to this Introduction to the Michigan Arts Education Instruction and Assessment Project.
In this module, I will describe some of the important characteristics of this ground-breaking project and introduce you to the resources you now have available to you.

5. MAEIA DEE Resources
An overview of the MAEIA resources for the DEE project and how they could be used

6. Overview of the DEE Resources
Four webinars were conducted
Logistics in the Demonstrating Educator Effectiveness Project
Writing a SLO for Demonstrating Educator Effectiveness
Documenting Instruction
Documenting Instruction in the Demonstrating Educator Effectiveness Pilot Project
Seven modules were released on selecting, administering, scoring and reporting the MAEIA assessments
A variety of resources created and available on the project website:
Describe the contents of the four webinars.
In DEE Webinar 1 – mmmmm
In DEE Webinar 2 – nnnnnn
Each of these webinars is archived and available on the project website.

7. DEE Project Summary
A project summary was prepared at the request of one of the DEE project participants
The project summary is for participant use to remind them about the DEE project, the activities carried out, and the resources provided
Use the summary not only to remind yourself about these but also to inform others, such as colleagues, supervisors, and/or administrators about the project and how the resources may be useful to them

8. DEE Project Summary

9. Other MAEIA Resources
A Teacher Booklet for every assessment, and a Student Booklet for most of them.
The Assessment Administration Manual provides general information about how to administer assessments.
A paper on Using the MAEIA Assessments to Demonstrate Educator Effectiveness is also available.
There are other resources you should be aware of:
There is a Teacher Booklet for every assessment, and a Student Booklet for most.
The Assessment Administration Manual provides overall general information about how to administer assessments, and
A paper titled “Using the MAEIA Assessments to Demonstrate Educator Effectiveness” is also available. This paperß suggest ways that the MAEIA assessments can meet educator evaluation requirements in Michigan

10. How can teachers use MAEIA Assessments?
to inform current instruction
to improve student learning and achievement
as a portion of educator
effectiveness demonstration
to improve future instruction and program improvement
MAEIA assessments can be used in several ways:
There are several ways that the MAEIA assessments can be used:
to inform current instruction
to improve student learning and achievement
as a portion of your demonstration of your effectiveness as an educator
to improve future instruction and program improvement

11. Overview of Michigan’s Educator Evaluation Law
Public Act 173 of 2015
See our PowerPoint from November 2 for a more detailed overview.

12. Evaluation Law: Moving Targets
In November 2015, legislators passed PA 173 of 2015
Amends MCL
Eases into changes, most starting in
Addresses evaluation requirements in two areas:
Professional Practice
Student Growth
(a) Evaluate teacher and school administrators at least annually
(b) Establish clear approaches to measuring student growth and provides teachers and school administrators with relevant data
(c) Use multiple rating categories (highly effective, effective, minimally effective, and ineffective) that take into account data on student growth as a significant factor (now replaced with specific percentages).
(d) Uses the evaluations, at a minimum, to inform decisions regarding all of the following:
(i) The effectiveness of teachers and administrators,
(ii) Promotion, retention, and development of teachers and school administrators
(iii) Granting of tenure or full certification, or both
(iv) Removing ineffective tenured and untenured teachers and administrators 64

13. Professional Practice
Portion of evaluation not based on growth data must be based “primarily” on a district-selected framework.
Frameworks:
MCEE-recommended: Danielson’s Framework for Teaching, Marzano Teacher Evaluation Model, The Thoughtful Classroom, or 5 Dimensions of Teaching and Learning.
MDE-approved: TBD Districts may choose a framework on the list, build their own, or modify a framework on the list
Training: All evaluators must receive framework training, delivered by the framework vendor or authorized trainer.
NOTE: Teachers and administrators being evaluated must also receive some training in how the framework will be used to evaluate them. This can be provided by the district or the vendor. 64

14. Student Growth Ratings
Percentage of evaluation based on student growth:
through : 25%
and beyond: 40%
Student growth data:
State assessment data does not have to be used until
State assessment data make up only half of the total growth data for teachers in tested grades and subjects.
Non-State (Local) growth measures must use multiple measures and be used consistently among similarly situated educators.
For 2015‐16 through 2017‐18, student growth is 25% of the overall performance rating.
In 2018‐19, the growth rating moves to 40%.
For teachers in state tested core areas, growth ratings will be based on a combination of ratings from State assessments and district measures.
For teachers in core or non‐core areas with no required state test, growth ratings will be based on district measures which may include optional State assessments where available.
For administrators, growth will be based on aggregation of growth measures for teachers within the administrator’s scope of responsibility.
Districts must develop a local growth model that is transparent and treats all similarly situated teachers and administrators equitably. 64

15. Student Growth Ratings, cont’d
Non-state (local) growth measures may include the following:
Student Learning Objectives (SLOs)
Other rigorous assessments that are comparable across the district
Nationally normed or locally developed assessments aligned to state standards
Research-based growth measures
IEP goals (where applicable)
Districts must develop a local growth model that is transparent and treats all similarly situated teachers and administrators equitably. 64

16. Student Growth Ratings 2017*
Core Curriculum Teachers
Non‐Core Curriculum Teachers
State** & Local Student Growth 25%
Local Student Growth 25%
Professional Practice per Evaluation Instrument 75%
Professional
Practice per Evaluation Instrument 75%
Local Student Growth:
Student Learning Objectives (SLOs)
Nationally normed or locally developed assessments aligned to state standards
Research-based growth measures
Other rigorous assessments that are comparable across the district
IEP goals (where applicable)
*Growth Ratings:
25% through 2017‐18; 40% 2018‐19 and after
**Use of State Assessment data optional until

17. Using the MAEIA Assessments to Demonstrate Educator Effectiveness
One of the MAEIA modules available on the MAEIA website

18. Purpose of this Module
Describe three ways that educators can use student performance on the MAEIA assessments to demonstrate their effectiveness Explain various ways of summarizing student results Methods to summarize student results to demonstrate “growth” are also covered Continued . . .
There are several purposes for this module. These include
Describe ways that educators can use student performance on the MAEIA assessments to demonstrate their effectiveness
Explain these different ways of summarizing student results, especially to show growth.

19. Purpose of this Module . . . Continued
Present an alternative to statistical procedures:
Documenting student work
Selecting student work for exhibition
Documenting instructional practice
Demonstrating educator effectiveness
In addition, this module presents different ways to show educator effectiveness, including
Documenting student work
Selecting student work for exhibition
Documenting instructional practice
Demonstrating educator effectiveness

20. Using MAEIA Assessments at the Classroom Level
Purpose of model performance assessments in the arts is to improve teaching and learning in the arts.
This slides serves as a reminder that the basic purpose of the MAEIA project and the MAEIA assessments in the arts is to improve teaching and learning in the arts.

21. Using MAEIA Assessments at the Classroom Level
When using the model MAEIA assessments:
Teachers are allowed to change the assessments to fit their classroom situations.
Teachers should document the processes and student products/performances needed for their professional collaboration and improvement (as part of a teacher demonstrating his or her effectiveness).
A couple of reminders about the MAEIA assessments:
Teachers are allowed to change the assessments to fit better fit their planned classroom sinstruction
Teachers should document the processes and student products/performances needed for their professional collaboration and improvement (as part of a teacher demonstrating his or her effectiveness).

22. MAEIA Educator Effectiveness Models
The MAEIA project has created three methods for educators to use in demonstrating their effectiveness. Two models use pre-post student data, while the third uses an array of student performances Each model recognizes that instruction and achievement in the arts is different than content areas such as mathematics or reading Each tries to characterize arts achievement in realistic terms, given limits of instructional time
The MAEIA project has created three methods for educators to use in demonstrating their effectiveness.
• Two models use pre-post student data, while the third uses a selection of student performances
• Each model recognizes that instruction and achievement in the arts is different than in content areas such as mathematics or reading
• Each tries to characterize arts achievement in realistic terms, given the limits of instructional time

23. MAEIA Educator Effectiveness Methods
Each MAEIA assessment is designated for use in one of these methods:
Method 1
Test-retest in the same school year
Method 2
Test-retest in adjacent school years
Method 3
Select examples of
student performance to show student achievement – the “new old-fashioned way” of demonstrating student proficiency
This slide shows a summary of the 3 methods of demonstrating educator effectiveness
Method 1 is a re-use of the same assessment within the same school year
Method 2 is the re-use of the same assessment in adjacent years
Method 3 is the selection of a range of student responses to the assessment to show levels of student achievement – the “new, old-fashioned way” of demonstrating student achievement
maeia-artsednetwork.org/educator-effectiveness-methods

24. Educator Effectiveness--Method 1
Test-retest in the same school year
A MAEIA assessment is given to students twice:
Fall and spring of the same school year (or start or end of a semester)
Before and after instruction on the content standards measured by the assessment
As this slide indicates, Method 1 is most suitable for assessments that do not take much time to administer. This is typically the performance events. In Method 1, the same assessment is given to students before and after instruction, which could be at the start and end of the school year, semester, marking period and so on.
Most suitable for assessments that can be given in a short period of time – such as MAEIA performance events.

25. Educator Effectiveness--Method 2
Test-retest in adjacent school years
A MAEIA assessment is given to students twice:
Fall (or spring) of adjacent school years
Pre-test and instruction in the first school year assessment, with post-test in second year
Most suitable for assessments that require longer periods of time to administer – such as MAEIA performance tasks
Feasible because the same arts educator might instruct the same students over multiple grade levels
Method 2 is similar to Method 1, except the pre-test and post-test occur in adjacent school years, which may be feasible because you have the same students for two or more grade levels.
Method 2 may be more suitable for lengthier assessments, such as the MAEIA performance tasks, but again, the assessment should occur before and after instruction has taken place.

26. Educator Effectiveness--Method 3
Select examples of student performance to show student achievement
Some MAEIA assessments are unique; doing them twice (as in Models 1 and 2) wouldn’t be useful or interesting to students nor informative to teachers.
Educators have typically demonstrated their effectiveness by selecting exemplars of student work for exhibition in their classrooms
This has been done traditionally, so we nick-named it the “new old-fashioned” method
Model 3 is suitable for any MAEIA assessment.
Method 3 is a more traditional method for showing what students know and/or are able to do. Hence, the nickname “the new old fashioned way” of showing student achievement.
This method has been assigned to assessments that are so unique that doing them twice wouldn’t be as useful or interesting to students nor informative to teachers
Traditionally, educators have demonstrated their effectiveness by selecting exemplars of student work for exhibition in their classrooms; that is what this method is at its heart
Note that Model 3 is suitable for any MAEIA assessment, even those labeled as Method 1 or 2

27. Educator Effectiveness Methods are included in the Online Assessment Catalogue
The MAEIA online catalog shows the suggested method for arts educators to use for each assessment to demonstrate their effectiveness

28. Hover on the number to see a description of the method.
As this slide shows, if you hover on the item number, a brief description of the method for demonstrating educator effectiveness will appear.

29. How to Use Methods 1 and 2
For methods 1 and 2, the teacher should first score each student’s responses, using the Teacher Scoring Rubrics found in the Teacher Booklet.
Now, I want to turn to how to actually produce numerical data for use in demonstrating your effectiveness
If using Methods 1 and 2, the teacher should first score each student’s responses, using each Teacher Scoring Rubric found in the Teacher Booklet

30. How to Use Methods 1 and 2
Next, fill out the MAEIA Classroom Score Summary page(s).
Please see Module 7 and the MAEIA Assessment Administration Manual for more information.
Next, fill out the MAEIA Classroom Score Summary page or pages with your scores for each student for each dimension of each Teacher Scoring Rubric
maeia-artsednetwork.org/model-assessments

31. How to Use Methods 1 and 2
Now, calculate a “total score” for each student by summing the scores on each dimension in the rubric.
Do this each time the assessment is used.
Subtract the Time 1 (pre-test) score from the Time 2 (post-test) score for each student; the result will usually be positive.
More detailed information is presented in:
Now, calculate a “total score” for each student across the different dimensions on each Teacher Scoring Rubric. Do this by summing the score across the dimensions in the rubric.
Do this each time the assessment is used – pre- and post-test
Subtract the Time 1 (pre-test) scores from the Time 2 (post-test) score for each student; the result will usually be positive.
More detailed information is presented in the Educator Effeciveness booklet available from this URL on the MAEIA website:
maeia.wpengine.com/wp-content/uploads/2016/02/Using-MAEIA-Asst-to-Demontrate-Educator-Effectiveness-v.-5.1.pdf
maeia.wpengine.com/wp-content/uploads/2016/02/Using-MAEIA-Asst-to-Demontrate-Educator-Effectiveness-v.-5.1.pdf

32. How to Use Methods 1 and 2
Choose a way to display your data:
Calculate a Mean Change Score for the classroom
Construct an Achievement Change Table
to show levels
of change.
There are two ways that you can show the data. The first is to calculate a mean change score for all students included in the group.
Alternatively, you could construct an achievement change table to show changes from the pre- to post-test for the group of students.

33. How to Use Methods 1 and 2
Calculate a Mean Change Score for the classroom
Add up the individual student change scores (sum of all individual T2-T1 scores)
Divide this number by the number of students (sum of T2-T1/N)
Do this for each MAEIA assessment used
This slide describes how to calculate the mean change score for a group of students.
Calculate a change score for each students – T2 score – T1 score
Next, add up the individual student change scores (sum of all individual T2-T1 scores)
Divide this number by the number of students (Sum of T2-T1/N)
Do this for each MAEIA assessment used

34. How to Use Methods 1 and 2 Ways to display the data:
Set up a table that shows several levels of change (gains or losses)
Ways to display the data:
Construct an Achievement Change Table to show levels of change for students in the classroom:
Up +20 points or more
Up +10 to +19 points
Up +1 to + 9 points
Unchanged
Down -1 to -9 points
Down -10 to -19 points
Down -20 points or more
A second method is to construct an Achievement Change Table to show levels of change for student in the classroom
Set up a table that shows several levels of change (gains or losses) using categories of change like those shown in the slide
Up +20 points or more
Up +10 to +19 points
Up +1 to + 9 points
Unchanged
Down -1 to -9 points
Down -10 to -19 points
Down -20 points or more
Report the number and percentage of students in each category
Report the number and percentage of students in each category

35. Example Achievement Change Table
–20 points or more
–10 to –19 points
–1 to – 9 points
No Change
+1 to +9 points
+10 to +19 points
+20 points or more
Number 1 4 6 8 7
Percent
3.3%
12.1%
20.0%
26.7%
23.3%
This slide illustrates what a Achievement Change Table to show levels of change might look like. From this table, you can see that two-thirds of the students (19 of 30) improved in performance, 6 of 30 remained the same, and 5 of 30 declined in performance.

36. How to Use Method 3
Select exemplars from student groups such as these to demonstrate overall achievement in your classroom:
Students who were already high achieving: How did they do on the assessments used? Did they improve?
Students who were initially struggling: Have they done well on the assessments used? Are they more confident learners?
Students who initially struggled to perform at all, who are now performing and perhaps doing much better.
This slide suggests some ways to select exemplars of student performance for Method 3, the “new, old-fashioned way” of showing educator effectiveness.
Select exemplars from student groups such as these to demonstrate overall achievement in your classroom:
Students who were already high achieving—How did they do on the assessments used? Did they improve?
Students who were initially struggling—Have they done well on the assessments used? Are they more confident learners?
Students who initially struggled to perform at all, who are now performing and perhaps doing much better.

37. Documenting Teacher Instruction is Essential!
Instructional information should be used along with student performance to demonstrate educator effectiveness.
As this slides notes, Instructional information should be used along with student performance to demonstrate educator effectiveness

38. Documenting Teacher Instruction
You may want to create a log of what you did instructionally on each content standard assessed.
A concise narrative summary of this for your supervisor would make it most useful.
The log and summary may be written, or you may use video of classroom instruction, student work, and student reflections on the assessment in the classroom.
You may want to create a log of what you did instructionally on each content standard assessed.
A concise narrative summary of this for your supervisor would make it most useful. Video excerpts of instruction can help, too
The log and summary may be written, or you may use video of student work in progress, student performances, and student reflections on the assessment in the classroom.

39. Documenting Teacher Instruction
You may want to create a log of what you did instructionally on each content standard assessed.
A concise narrative summary of this for your supervisor would make it most useful. Video excerpts of instruction can help, too
The log and summary may be written, or you may use video of student work in progress, student performances, and student reflections on the assessment in the classroom.

40. Teacher Reflections on the Assessment and Student Learning
Once the teacher has taught the content standards, documented instruction, and collected, scored, and analyzed student achievement, the teacher should prepare a concise reflection on what he or she learned.
Once all of the information has been collected, scored, and analyzed, the teacher should prepare a concise reflection on what he or she learned annotated with the examples of student work, videos of instruction or student work and performances, and their reflections

41. Teacher Reflections on the Assessment and Student Learning
What worked and what didn’t?
What formative information was collected during instruction or assessment and what changes in instruction did you make?
What did you learn about your students – their achievement and attitudes?
How did you use this information?
In reflecting on your work as an educator and the achievement of your students, reflect on the following questions
What worked and what didn’t? What would you do differently?
What formative information was collected during instruction or assessment and what changes in instruction did you make?
What did you learn about your students – their achievement and attitudes?
How did you use this information?

42. Putting it All Together
The teacher should prepare a portfolio of evidence for each assessment.
Statistical summaries of student achievement – Mean Change Score and/or Achievement Change Table
Samples of student work, both pre- and post-test, if used
Documentation (written or video) of the teacher’s instruction on the standards that were assessed
Student reflections on their learning, both written and video
Teacher’s reflective summary about instruction and assessment
To put all of this together, the teacher should prepare a portfolio of evidence for each assessment, with
Statistical summaries of student achievement – Mean Change Score and/or Achievement Change Table
Samples of student work, both pre- and post-test, if used
Documentation (written or video) of the teacher’s instruction on the standards that were assessed
Student reflections on their learning, both written and video
Teacher’s reflective summary about instruction and assessment

43. How to Use the MAEIA Results for Educator Effectiveness
Other sources of achievement and outcome data (e.g., other measures or indicators of achievement) should be used as well
Prepare corresponding narratives regarding teacher practices to accompany student assessment results
These achievement data, along with appropriate observational data, should be used in the overall evaluation of an educator
The goal of educator evaluation should primarily be improvement of educator practice
As this slide notes, think about demonstrating your effectiveness more broadly
Other sources of achievement and outcome data (e.g., other measures or indicators of achievement) should be used as well
Prepare corresponding narratives regarding teacher practices to accompany student assessment results
These achievement data, along with appropriate observational data, should be used in the overall evaluation of an educator
It is our belief that the main goal of educators demonstrating their effectiveness with educator evaluation system is primarily to improve educator practice

44. Reports of DEE Activities from DEE Project Participants
A review of the responses to the DEE survey sent out in advance of the May 24 meeting.

45. DEE Project Participant Feedback
DEE participant feedback on the demonstrating educator effectiveness process and outcomes

46. DEE Project Participant Feedback

47. Critical Response Process
Liz Lerman’s Critical Response Process (CRP) is a feedback system based on the principle that the best possible outcome from a response session is for the maker to go back to work
Consists of a series of protocols for effective conversation
Developed with the intention of giving and receiving better feedback
Created by Liz Lerman, founder of The Dance Exchange –
See

48. Critical Response Process
Involves people serving in three roles: Artist, Responder, Facilitator
Four stages of the process
Statements of Meaning
Artist as Questioner
Neutral Questions
Opinion Time

49. Applying the Critical Response Process
Statements of Meaning
Artist as Questioner
Neutral Questions from Responders
Opinion Time

50. Actual Educator Effectiveness Meeting
A DEE project participant – a teacher and a administrator/supervisor – conduct an actual meeting to review the effectiveness of the teacher.

51. Debrief of the Actual Meeting
For the two participants:
How well do you think the meeting went?
What might you do differently next time?
For all attendees:
What questions might you have?
What did you see that you would find useful to use in your own meetings?
What resources might have been helpful to promote deeper conversations about student achievement or instructional effectiveness?

52. Remaining Questions and Concerns
What questions and concerns do the meeting attendees have about the processes for demonstrating educator effectiveness?

53. Interest in Continued Work on DEE from DEE Project Participants
A discussion about whether there is value in continuing to work on the DEE during the school year.

54. Continued DEE Work Next Year?
Thinking about next year, what resources are you hoping would be available to you for you to use to demonstrate your effectiveness that are not currently available?
What questions remain for you?
What concerns do you have about the DEE process in the future?
Are you interested in continuing to work on the DEE next year?

55. MAEIA PMT Work for Next Year
Revise the Using the MAEIA Assessments in Educator Evaluation paper
Organize the DEE resources for ready access by all MAEIA teachers and administrators
Building out the DEE webpage on the MAEIA site
Preparing information for school administrators on understanding high quality arts instruction for arts educator observation

56. Contact Us MAEIA Project 517-816-4520 MAEIA Project Page
MAEIA Project Page
If you have any questions, contact us at this phone number or address.