1. Item Selection and Ordering Information September, 2014

2. MAEIA Project Management Team 2 Kathy Dewsbury-White MAEIA Project Director Ed Roeber MAEIA Assessment Director Ana Luisa Cardona MAEIA Arts Education Consultant Frank Stuart Philip MAEIA Assessment Consultant Cheryl Poole MAEIA Editor Kathy Humphrey MAEIA Field Test Coordinator and Project Management Support Barb Michelutti MAEIA PD and Project Support Director Jason O’Donnell MAEIA Administrative Assistant

3. What is the Michigan Arts Education Assessment and Instruction Project (MAEIA)? 3 https://www.dropbox.com/s/zng8w2bu9pg3fbe/MAEIA%20Voiceover.mov Open this link, download it to your desktop to see the 8.5 minute overview of the MAEIA Project. Be sure your sound is on. You’re part of something BIG!

4. 4

5. 5

6. 6  This presentation is intended for use by teachers who registered for the 2014-15 MAEIA Model High School Arts Assessments field test, but were unable to attend a Briefing Session.  If you have not yet registered to participate in the field test, please do so by visiting our webpage. http://www.michiganassessmentconsortium.org/maeia-high-school- assessment-field-test-resources  Please review this entire presentation BEFORE you order assessment items.  This presentation refers to resources and documents located on the webpage noted above.  Information on who to contact for assistance is included at the end of this presentation. Important!

7. Introduction to Assessment Standards Description of Assessment Types Samples of Teacher and Student Booklets Explanation of the Catalog of Assessments Selection of Items Ordering Before, During, and After Assessment Administration Return of Materials How to Use the Assessments Who to Contact with Questions Overview 7

8. Four disciplines—Dance, Music, Theatre, and Visual Arts Assessments can be used in several ways: To inform current instruction To improve student learning and achievement For future instructional program improvement As one portion of educator evaluation Intro to MAEIA Model Assessments 8

9. Based on MAEIA Performance Standards Assess important content, knowledge and skills in the Michigan Arts Education Standards and Benchmarks 2011 Organized as Create, Perform and Respond Performance Standards Items also are aligned to the VPAA Credit Guidelines Integral Role of Standards in the MAEIA Assessments 9

10. 10 MAEIA Performance Standards Michigan Content Standards and Benchmarks Michigan Grade Level Content Standards Layers of Standards

11. 11 The next six slides describes the types of assessments found in the catalogue of MAEIA Model Assessments. Coming up…..

12. 12 Performance Tasks Performance assessments carried out by individual students or small groups of students over time (days, weeks, months) Performance Events Performance assessments that are administered “on-demand,” without any or just a brief amount of rehearsal time. Types of Assessment Items Chapter 4 Nature of the Assessment Items

13. 13 Performance assessments carried out by individual students or small groups of students over time (days/weeks/months) Tasks may take place over an entire semester or trimester. Tasks are carried out in or out of class, and are very much related to instruction (e.g., class assignments) They measure essential outcomes in the content standards not easily measured in other ways Performances are judged using one or more scoring rubrics Performance Tasks

14. 14 Performance assessments that are administered “on- demand,” with limited or no practice or rehearsal time (i.e., no more than a few minutes) May be individual or small group assessments Test administrator presents items to one student or a small group of students, who respond in “real” time Performances are also judged using one or more scoring rubrics Performance Events

15. 15 Constructed Response Items in which students write a response to a prompt. Selected Response Items related to other items in which the student is given a prompt (a question or a statement) and answer choices Both C-Rs and S-Rs serve to illuminate declarative knowledge nested inside Tasks and Events. Types of Assessment Items cont’d Chapter 4 Nature of the Assessment Items

16. 16 Items in which students write a response to a prompt Usually can be administered to groups of students together Some type of stimulus (e.g., music selection, video, or picture) could be used Response may involve writing, sketching, constructing a table, as well as a written response Responses are judged using one or more scoring rubrics Constructed-Response Items

17. 17 Items in which the student is given a prompt (a question or a statement) and answer choices Student has to select either the correct answer or the most correct answer Multiple-choice questions are the most popular form of these items In MAEIA, these are used to tap the content knowledge or other procedural knowledge needed to respond to PT/PE/CR items Selected-Response Items

18. We follow Best Practice – by following the professional measurement standards that govern the development & use of assessments: Are they written so that teachers and students understand them? Do they yield assessable information? Do the items assess what they are intended to assess? Have the scoring guides been reviewed and revised (can they be strengthened)? Any assessment with stakes for teachers and/or students should be piloted or field tested before it is used to make important decisions. (AERA/APA/NCME Standards for Educational and Psychological Testing) Why do we field test assessments? 18

19. 19 The next four slides provides a look at the cover and the first three pages of a Teacher Booklet of a MAEIA Model Assessment. The Student Booklet is nearly identical to the Teacher Booklet but does not have the teacher-specific information or the answer keys for the selected- response questions. Coming up…..

20. 20 Samples of a Teacher Booklet- Cover Page

21. 21 M.E421 – This is the item label. It refers to Music.Event #421 Make Connections is the name of the assessment. High School Level 3 means it is a high school item written for the students who have had 3 years of work in the discipline. Under Teacher Booklet, you find a list of what is included in the booklet. Decoding the Cover Page

22. 22

23. 23 The table at the top shows the standards being assessed by this booklet. Following the table is an overview of the assessment activity. Page 2 also shows the suggested duration of this assessment, the list of materials and the set-up required. Decoding the 2 nd Page of the Booklet

24. 24

25. 25 This page shows the teacher directions. It also defines how many parts this assessment has and shows, in bold text, the beginning of the script written for the teacher to read to the students. Decoding the 3rd Page of the Booklet

26. 26 You will turn to the Assessment Administration Manual (AAM) to read the important considerations to be made when selecting which assessments fit into your curriculum and instruction. Assessment Administration Manual Read Chapter 4, pages 14-16, of the AAM. Now that you’ve seen a sample booklet…

27. 27 Selecting the Assessments to Administer Please read Chapter 4 of the Assessment Administration Manual 4.1–Catalog of Available Assessments14 4.2–Fitting the Assessments Into Your Instructional Program14 4.3–Selecting When to Use the Assessments15 4.4–Adding Additional Assessments15 4.5–Deleting Planned Assessments16 4.6–Questions or Concerns16

28. 28 COLUMN HEADINGS OF THE CATALOGUE … SEE SLIDE 29, 30, & 31 FOR DEFINITION OF EACH COLUMN

29. Discipline—Dance, Music, Theatre, or Visual Arts Item Type and Number—The code for each task (T) or event (E). Note: all High School assessments are the 400 series. This will distinguish them from grade K-2 assessments (100s), grade 3-5 assessments (200s), and grade 6-8 assessments (300s). Short Item Descriptor—A brief phrase or title of the assessment, intended to convey the essence of the item. Grade Range—This is always set at 4, i.e., high school. Levels 1, 2 and 3—This designates the level or levels for which the item was written. Performance Standard—Performance standards are discipline-specific and condense the Michigan Merit Curriculum’s five content standards into the three overarching performance standards of Create, Perform, and Respond. These performance standards were developed to serve as the basis for MAEIA model assessments that measure student proficiency in each arts discipline at each grade span - K-2, 3-5, 6-8, and 9-12. Introduction to the Catalogue of MAEIA Assessment Items 29

30. Content Standard and Benchmark—These are the Michigan State Board of Education-approved standards for arts education, taken from Michigan Arts Education Content Standards and Benchmarks for Dance, Music, Theatre and the Visual Arts (2011). VPAA Guideline—These are taken the Michigan State Board of Education-approved Michigan Merit Curriculum Credit Guidelines for the Visual, Performing, and Applied Arts (2006). Requires Recording—This is a signal to a teacher using this item about whether students’ responses will need to be electronically recorded. Total Time—This is an indication of the number of 50-minute class periods that the assessment requires. This number may not be precise for two reasons: Your class period length may differ The time it takes to record and download the performances of all students in larger classes may require an extra class period (or two) Introduction to the Catalogue of MAEIA Assessment Items 30

31. Because students participate in the arts for different reasons and durations, assessment tasks and events were developed for three different levels of HS arts students: Level 1—Students who fulfill their one credit visual, performing, or applied arts (VPAA) high school graduation requirement only, or who are in their first year of a multi-year VPAA program. Level 2—Students who have already completed their first year in an arts discipline course and are now in their second year of instruction in the same arts discipline. Level 3—Students who have already completed their first and second years in an arts discipline’s courses and are now in their third year or fourth year of instruction in the same arts discipline. If a student takes a year of instruction in one discipline (e.g., music) and another year of instruction in another discipline (e.g., theatre), they would participate in Level 1 assessments in each discipline. Teachers have the flexibility to adjust the assessments to match the instructional levels of the students being assessed. Assessment Levels 31

32. 32 Click on this link and scroll down to find the short description catalogue of items for your discipline: http://mi-arts.wikispaces.com/MAEIA+Project+Documents Then follow directions for Steps 1-3 on slides 34 -36. Coming up…..

33. Step 1 Look at the Catalog for your discipline and the Short Item Descriptor of the Tasks and Events. Make a notation of the Tasks or Events that interest or intrigue you, i.e., that you would like to know more about. Steps in Selecting Items 33

34. Step 2 Now, click on the hyperlinks for the Short Item Descriptors in the Catalogue for the items you selected in Step 1. Check the: Michigan Student Learning Standards Assessed, Level Overview and Outline Suggested Total Time. Select the Tasks and Events that logically fit into your existing curriculum and instructional plans. Make a notation of the ones that you confirm fit your instruction, curriculum and assessment plan. Steps in Selecting Items 34

35. Step 3 Enter the item type, number, the course in which you plan to use it and the number of students in that course. Check which instructional cycle you will be using for the assessment. Check to see that the items you have selected will result in a reasonable assessment plan for your courses This is the assessment field test plan. Keep this for your records. Steps in Selecting Items 35

36. 36 Read the review on slides 38 - 42. Just before ordering……

37. We are expecting teachers to select a small number of Events and Tasks to use throughout the school year (October–April). We are suggesting that items be used within three instructional cycles, to help you pace your use of the Events and Tasks throughout the school year: October 15 – December 15, 2014 (fall) January 7 – February 27, 2015 (winter) March 2 – April 17, 2015 (spring) We are suggesting 1-2 Tasks and 1-2 Events per Instructional Cycle. This is a recommendation, not a requirement; field testers may select fewer (or more) items. 37 In review…

38. 38 After using and scoring the assessments yourself, you will return student responses to us to aid in refining the items and enhancing the scoring rubrics. Your feedback on the uses and usefulness of the assessments is very important to us. We look forward to your feedback as you finish using the assessments during each instructional period and at the conclusion of the field test. There is a separate Teacher Booklet for each assessment. Each item is self-contained.

39. You will be sent a sufficient number of Student Booklets (plus a small overage) and couple of copies of the Teacher Booklet for each assessment item you select You will receive an Assessment Administration Manual (AAM) with overall field test directions in it. The AAM discusses general assessment topics, including actions you should take before, during, and after assessment. We encourage you to select a mix of Create, Perform and Respond Items. Combined, these strands are the creative process (in Visual Arts, we think of “Perform” as “Present.”) Any additional materials needed to administer an item is noted. Recording of student responses is also noted. 39

40. 40 Modifications are fair game. You must record any modifications that you make and return those notes to MAEIA for refining the assessments. Use the Item Review and Comment Form Examples of modifications might include: Changing genre of images, music, theatre and dance examples Adding criteria to scoring rubrics Revising to better fit your instruction Any modifications will be recorded on a form titled “Item Review and Comment Form” Modifications of Items

41. 41 Item Review and Comment Form

42. 42 Ready to order the items you’ve selected? Proceed to the next slide for the web address and directions for ordering.

43. 43 Ordering Your Items via an Online Order Form The next 5 slides will show you what you will see when you open the online order form.

44. 44 First, you will advance through two pages before you will enter contact and shipping information. This is very important! This screen continues in the actual order form.

45. 45 You’ll be asked for your district and building codes. If you do not have them, leave this blank.

46. 46 Ordering Your Items Next, you will select the discipline in which you will be field-testing assessment items.

47. 47 Ordering Your Items Select the tasks and events. Click on the little arrow that looks like a “v” to select number of students. Select instructional cycle(s).

48. 48 Ordering Your Items This is the last screen you will see before you submit your order. Please be sure your order is correct before you click SUBMIT. Thank you!

49. 49 Once you’ve submitted your order, a PDF of your order will appear on the screen. You can download it to your computer and save it.

50. 50 Before leaving this presentation you must refer again to the Assessment Administration Manual (AAM). Click on the link below. Assessment Administration Manual Assessment Administration Manual

51. CHAPTER 5 – Read pages 17-26 for the overview and before, during and after assessment for step by step instructions for Preparing for the assessment. Administering the assessment. Packing and returning student results, booklets and your comments and modifications. 51

52. 52 The next 15 slides will summarize the all-important Chapter 5 of the Assessment Administration Manual which provides the specifics of what you will do before, during and post administration of the assessments of your choice. Coming up….

53. Inventory the assessment materials you were sent. Do you have enough Student Booklets? Order extras from DRC is needed. Read through each assessment one or more times. Practice administering the assessment with a colleague or to yourself Assemble other materials needed (if any). Decide whether the assessment can be used as is or needs to be modified. You may modify them if necessary. Determine how students’ responses will be recorded and downloaded (if necessary) Determine when the assessment will be used Before Administering the Assessment 53

54. Observe students during the assessment and note any issues, confusion, or questions on their part on the Item Review and Comment Form sent with the shipment of assessment materials Collect the assessment materials between class periods and at the conclusion of the assessment Organize the assessment materials for return to DRC During the Assessment 54

55. 55 Assessments may require recording student responses in image, audio or video files. Set up recording devices when required. Record audio, images and/or video with smart phones or other recording devise; send digital files via email to get them to your computer. Then copy to the flash drive sent with your assessment. Be sure to record responses and save them during the assessment in preparation for submitting them at the conclusion of the assessment. One flash drive is sent for every 25 students are sent in the assessment shipments for this purpose. Recording Student Work

56. 56 See Attachment D on page 13 of the Assessment Administration Manual for instructions for recording student results. Recording Student Work

57. 57 Item Review and Comment Form

58. Score students’ responses using the Teacher Scoring Rubric(s) provided; note any issues with the rubrics Complete your feedback on the assessment using the Item Review and Comment Form Mark up a copy of the Teacher Booklet and/or Student Booklet with your comments Upload recorded student responses to the flash drive(s) provided (See Attachment D of AAM) These are secure assessment materials Put all responses to one assessment item on a flash drive (one flash drive has been sent to you for each 25 students indicated on your original order) Put the flash drive(s) in the envelope provided; fill out the label(s) provided and seal the envelope Post Assessment Activities 58

59. Place the loose ancillary student responses in the envelope provided and complete the Ancillary Student Materials Return Envelope label. Ancillary materials include the following: Assessment Questions pages that students are instructed to tear off and give to you at the start of many of the assessments Drawings/paintings done by students Photographs of student work in lieu of return of large, bulky responses and/or 3-D objects (e.g., cans or kites) Any other student responses not done in their Student Booklets Post Assessment Activities 59

60. 60 Use the appropriate, identified envelope when returning assessment results.

61. 61

62. 62

63. Place the Item Review and Comment Form and the marked up Teacher Booklet and/or Student Booklet with your comments in the envelope provided. Fill out the label to indicate the assessment Include a copy of any revised Teacher Booklet and/or Student Booklet that you have created in the same envelope Post Assessment Activities 63

64. Returning materials to DRC 64 Packaging the materials for return to DRC: Use sturdy clean boxes to ship the materials via UPS (e.g., the boxes the assessment materials were sent to you in) Send each assessment after you have given it and scored students’ responses Use the pre-paid labels to ship the materials to DRC at no cost to you or your school If the assessment return requires more than one box, label each “1 of X,” “2 of X,” etc., with X = total number of boxes Fill out a Materials Return Cover Sheet and place on top of the first box.

65. 65 Materials Return Cover Sheet

66. 66 Modifications of Items Describe in detail any modifications you make.

67. 67 Order of loading materials in boxes:

68. 68 The next 3 slides provide suggestions for and cautions about using the MAEIA Model Assessments for Educator Effectiveness. Coming Up… How the MAEIA Assessment Results Might Be Used for Educator Effectiveness

69. If you feel that the assessment represents a good assessment of students (that is, that students understood what to do and were able to respond to the assessment), then the assessment results may represent their current achievement Teachers can select students’ responses to represent their work with the class (to show what the class was able to do) as well as individual students (those whose performance was exceptional or students who struggled but were still able to complete the assessment) How to Use the Results for Educator Effectiveness 69

70. Teachers can also select examples of student work from each instructional period to illustrate changes in students’ work over the school year Students whose performance was marginal in the fall who are now doing exceptional work Students who struggled to perform at all who are now doing acceptable (or better) work Student who still need help and how you are working with them The teacher and the school administrator can use this work as one piece of evidence of student learning and achievement How to Use the Results for Educator Effectiveness 70

71. Other sources of achievement data (e.g., students’ responses to other MAEIA assessments as well as to other measures or indicators of achievement) should be used as well Prepare corresponding narratives regarding teacher practices to accompany student assessment results These achievement data along with appropriate observational data should be used in the overall evaluation of an educator The goal of such educator evaluation should be primarily improvement of educator practice How to Use the Results for Educator Effectiveness 71

72. 72 The next two slides provide final comments about appropriately using the results of the MAEIA Model Assessments. Comments and Cautions

73. Remember, this is the first time these assessments have been given to students. The assessments may have issues that prevent students from showing you (and us) what they have learned and can do. (The purpose of field testing is to uncover these issues and to correct them). The assessment items as written may not fit well with the instruction planned and implemented by the educator. Even if the assessment item worked well and fits with the educator’s instruction, no single data element should be used to make important decisions about educators or students. Cautions about Using the Results 73

74. 74 The MAEIA assessments are NOT designed to be used as pre and post assessments. Another Caution

75. 75 Know that you are serving an important role by field testing the MAEIA High School Items. You can expect your order of assessments within the next 7-10 days. Check your order right away to make sure that you’ve received the correct item(s). Retain the shipping boxes for returning the assessment results. Contact us right away if you have questions or corrections. Now that you’ve ordered…

76. MAEIA Field Test Resource Page http://www.michiganassessmentconsortium.org/maei a-high-school-assessment-field-test-resources http://www.michiganassessmentconsortium.org/maei a-high-school-assessment-field-test-resources Jason O’Donnell 517.327.9224 517.327.0779 (fax) jodonnell@michiganassessmentconsortium.org Jason will forward each message to the most appropriate person to answer your question(s). Thank you for volunteering to participate in the field test! Contact Us for Help: 76