Turn in Pink Management Rubric to Anne René Data Analysis Session 10 November 3, 2017 Turn in Pink Management Rubric to Anne René Sit with subject team with whom you have collected action research data

Assessment is integrally related to curriculum and instruction. Julie +2=2 Curriculum SLOs Instruction Assessment Quality teachers use evidence of learning (assessment) to inform what they teach (the curriculum) and how they teach (instruction) (SLO = Student Learning Outcomes/Objectives)

The outcomes process (SLO’s) Julie +2 = 4 Collect data DATA: diagnostic, formative, checking for understanding, feedback… What does it mean? Assessment Evaluation Inform Instruction, (summative) grades, etc. Decision making

WestEd.org Data Literacy for Teachers Julie +2=6 “What do we want students to know and be able to do???” Data literacy for teaching is the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data (assessment, school climate, behavioral, snapshot, longitudinal, moment-to-moment, etc.) to help determine instructional steps. It combines an understanding of data with standards, disciplinary knowledge and practices, curricular knowledge, pedagogical content knowledge, and an understanding of how children learn. Answers… “How will we know when they get there?”

When should we look at data? Anne +2=8 Formative and Progress Monitoring assessment informs both teachers and students about student understanding at a point when timely adjustments can be made. Helps teachers determine next steps during the learning process as the instruction approaches the summative assessment of student learning.

What is the basic process? Anne +1=9 Analyze data – look at multiple sources Examine possible relationships or causes Consider impact and consequences (intended and unintended)

And then…. Anne +1=10 Transform Information Into a Decision Determine next steps Make instructional adjustments

Let’s talk about it! Anne +10= 20 Turn to a partner, brainstorm examples of formative assessment that could be used to help you create next steps / inform instruction Read the article: 5 ways to use data to improve your teaching https://www.amplify.com/viewpoints/5-ways-to-use-data-to-improve-your- teaching Discuss: What are some key points?

“Music gives a soul to the universe, wings to the mind, flight to the imagination. – Plato Teri +1=11

Action Research Action research is “a disciplined process of inquiry conducted by and for those taking the action. The primary reason for engaging in action research is to assist the “actor” in improving and/or refining his or her actions.” (Richard Sagor) Teri +1=12

Steps of Action Research Selecting a focus Clarifying theories Identifying research questions Collecting data Analyzing data Reporting results Taking informed action Teri +1=13

Socio-Political History of Rock ‘n Roll Using era-specific music and lyrics to enhance student understanding of U.S. History. Research ”supports the use of music as a mnemonic device for the learning and recall of new information.” (Michelle Lazar) To what extent can using music as a vehicle for teaching increase student comprehension of U.S. History during the second half of the 20 century? Pre-Post Tests (Internal Assessment) Internal Assessment and External Assessment (AP Exams) results Preparing and recording charts that highlight results Modifying resources, adjusting topical coverage Teri +2=15

Considerations / Narrowing the Focus Have students taken U.S. History before? (11th or 12th grade)? AP U.S. History / College Prep U. S. History? What Middle School did they attend? Did they like History before signing up for the class? Are they currently and/or have they been involved in musical training? What is the first / primary language? Comparing results to the other school this is taught at (RBV) and are there similar / different results - checking District demographics between schools for further comparison. Did they develop a greater interest for history after completing the class? Teri +2=17

Student Ethnicity in VUSD The district serves a diverse population of students, including more than 3,000 in its special education programs and more than 6,000 who are learning English as a second language. Enrollment has declined over the last several years from about 26,000 students in 2000 to approx. 22,000 in 2012.   Student demographics include: 58% qualify for free lunch 4% African American 24% English learners 3% Asian 10% homeless 3% two or more races 60% Hispanic 1% Filipino 28% White 1% Pacific Islander Teri +2=19

Demographics for MVHS v. VHS Demographic Information for MVHS: Demographic Information for VHS GATE students: 22% GATE students: 17% English learners: 3% English learners: 18% Reclassified as English proficient: 17% Reclassified as English proficient: 32% Migrant education program participants: 1% Migrant education program participants: 5% Eligible for free/reduced price lunch: 29% Eligible for free/reduced price lunch: 61% Special education within past two years: 6% Special education within past two years: 16% Teri +2=21

Demographics – Parent Level of Education Parent's Education for MVHS: Parent’s Education for VHS Graduate School: 18% Graduate School: 8% College Graduate: 33% College Graduate: 19% Some College: 28% Some College: 22% High School Graduate: 13% High School Graduate: 17% Not a High School Graduate: 8% Not a High School Graduate: 35% Teri +2=23

Middle Schools Teri +1=24

Level of Accuracy on Pre and Post Tests (Internal Assessments) Teri +5=29 Table 1: Pre Test Results (out of 56) (Table 2: Post Test Results (out of 54) (Bold-Italicized = more than ½ answered correctly) Q# 1. 3 2. 8 3. 40 4. 11 5. 34 6. 32 7. 20 8. 9. 7 10. 21 11. 4 12. 10 13. 5 14. 31 15. 12 16. 14 17. 18. 19. 20. 21. 22. 23. 24. 19 25. 26. 27. 28. 29. 13 30. 18 Q# 1. 6 2. 27 3. 33 4. 20 5. 41 6. 38 7. 8. 12 9. 21 10. 11. 35 12. 22 13. 5 14.38 15. 16. 29 17. 18. 13 19. 23 20. 10 21. 28 22. 48 23. 14 24. 25. 2 26. 32 27. 34 28. 29. 30. 36

External Assessment (AP U.S. History Exam) Teri +5=29 All students Total # Students Average Score Score 5 4 3 2 1 # Students 12 19 32 23 88 2.295 % by score .02% 13% 22% 36% 26% AP U.S. History Elective Class Total # Students Average Score Score 5 4 3 2 1 # Students 12 2.833 % by score .08% 16% 33%

Results Teri +2=31 Results have been gathered and charted for 3 years and found: On average, there was 26% improvement on internal assessments; An average 10% increase on external assessments over students who had not taken the Rock ‘n Roll class; and A greater understanding and passion for history after taking the class, according to end of the semester essays.

Action Research Project Annette +2=33 2015-16 Academic Year (Spring 2016) 26 Teacher Candidates Conducted Action Research Projects in Advanced Clinical Practice Intent of the Action Research Project Teacher candidates and program faculty to assess the impact of using effective strategies for K-12 student learning, namely English learners Pre- and Post-test Action Research where students completed a writing assignment without effective strategies (pre-test) and then completed a second writing assignment (post-test) with multiple effective strategies

Action Research Project Effective Strategies Annette +2=33 Explicit Reading Supports: e.g., highlighting reading, chunked reading, different text for each reading level Explicit Vocabulary Supports: e.g., graphic organizer, bulletin board, tea party Explicit Group Supports: e.g., teacher chosen groups, assigned roles, tiered activities Explicit Writing Supports: e.g., Venn diagram, T-chart, sentence starters Rubric for the writing assignment aligned to content and language objectives (CCSS and ELD Standards)

Action Research Project Annette +2=33 Percentage of teacher candidates responding “yes” on the pre- / post-survey (N=26) Indicating how effective they felt strategies used in post- test were   Pre-Test Post-Test Did your English learner focus student meet the content objective? 31% 84% Did your English learner focus student meet the English language development objective? 32% 92% Did your focus student with an IEP meet the content objective? 80%

Action Research Project The following teacher candidates showed a significant growth in the proportion of students meeting the content and ELD standards after implementing language supports. (N=11 – 4 candidates highlighted in this presentation) 56% more students met the content objective after the language support

Action Research Project Teacher Candidate #1 Social Studies Reading supports provided: Chunked Reading, Visuals to accompany/replace text p-value 0.56 Approx 0 Vocabulary supports provided: Props, Graphic Organizer Group supports provided: Teacher Chosen Group, Cooperative Learning Elements, Active Involvement Techniques, Activity Choice, Multisensory Group Activity, Gradual Release of Responsibility p-value 0.56 Approx 0 Other supports provided: Sentence starters, paragraph template.

Action Research Project Teacher Candidate #1 Social Studies Response from students when asked about their preferences: A majority of the students cited the graphic organizer and paragraph template as useful. The student with the IEP missed most of the instructional part of the lesson, so did not cite a preference for supports. The English learner cited the hands-on group activity. The other student cited the hands-on group activity and the rubric. What the teacher candidate learned: That the various teaching supports made a significant difference, but many of the students need much more intensive help with their writing.

Action Research Project Teacher Candidate #2 Mathematics Reading supports provided: Highlighted Reading, Annotated Text p-value 0.4 .002 Vocabulary supports provided: 4 Square Graphic Organizer p-value 0.48 .0003 Group supports provided: Teacher Chosen Group, Cooperative Learning Elements, Gradual Release of Responsibility Other supports provided: T-Chart, Sentence Starters

Action Research Project Teacher Candidate #2 Mathematics Response from students when asked about their preferences: The whole class liked talking with their group members and comparing the evidence they put on their Pros and Cons list. The student with an IEP especially liked the group conversations so they could make sure that they were on the right track. The EL student liked the sentence starters because it helped them understand where to start with their response. The other students I identified like the bolded and underlined text in the reading because it made it easier for them to understand. What the teacher candidate learned: I learned that my students need all types of supports in one activity. Sometimes what works for most students doesn't work for a few students and they need a slightly different version of what the rest of the class has or they might need extra instruction. I also learned that my students can justify their thinking when given the proper resources to do so.

Action Research Project Teacher Candidate #3 Science Reading supports provided: Visuals to accompany/replace text p-value 0.94 Approx 0 Vocabulary supports provided: 4 Square Graphic Organizer p-value .57 .026 Group supports provided: Teacher Chosen Group, Gradual Release of Responsibility

Action Research Project Teacher Candidate #3 Science Response from students when asked about their preferences: 85% of students that performed the writing of the paragraph in the 3rd lesson stated that they thought the keywords provided were helpful in completing the paragraph. 75% of students responded that they thought the graphic organizer was helpful for defining scientific vocabulary keywords. The English learner, IEP student and a student who is easily distracted were with the majority - i.e. they thought the keywords provided for the paragraph writing was helpful and, with the exception of the EL student, responded that the graphic organizer was useful. What the teacher candidate learned: A writing exercise following a student-centric physical hands-on activity is more engaging to students than a writing assignment following a teacher-led demonstration.

Action Research Project Teacher Candidate #4 English Reading supports provided: Highlighted Reading, Chunked Reading, Visuals to accompany/replace text, Different Text for each reading level, Annotated Text p-value 0.85 Approx 0 Vocabulary supports provided: Other: ____________, Frayer Models p-value 0.85 Approx 0 Group supports provided: Teacher Chosen Group, Group Roles, Bloom's Taxonomy, Gradual Release of Responsibility Other supports provided: T-Chart, Sentence Starters, Template

Action Research Project Teacher Candidate #4 English Response from students when asked about their preferences: 100% of my students said that it was much easier to write the paragraph the second time around. The strategies that the students stated were most helpful included: academic vocabulary review, paragraph writing template, and reading supports. My ELD class as a whole appreciated the writing template the most. The focus English Language Learner (Level 1) thought that the adapted text was the most helpful. I created my own adapted text that was much shorter, included visuals, and was annotated as well. My student with an IEP believed that the most helpful support was the academic vocabulary activities and review. My other student, a Level 3, appreciated the writing template the most.

Action Research Project Teacher Candidate #4 English What the teacher candidate learned: Through this action research process, I truly understood the direct impact that carefully planned and intentional supports have on student success. There is so much that we can do as teachers simply by identifying the diverse/unique needs in the classroom and meeting those needs. I learned that providing supports for students in no way enables them, but just provides them with the differentiation that they need in order to make the content accessible to them. It was incredible to see the exponential growth between the two paragraphs.

Action Research Project Effective Strategies Annette +15= 71min What strategies have you used that you felt were effective for your students? English learners? Explicit Reading Supports: e.g., highlighting reading, chunked reading, different text for each reading level Explicit Vocabulary Supports: e.g., graphic organizer, bulletin board, tea party Explicit Group Supports: e.g., teacher chosen groups, assigned roles, tiered activities Explicit Writing Supports: e.g., Venn diagram, T-chart, sentence starters Rubric for the writing assignment aligned to content and language objectives (CCSS and ELD Standards)

Reading and Reflection Sit in Content Area Groups Julie + 10 = 81 Before analyzing your data, read the article: My Nine Truths about Data Analysis http://www.edweek.org/ew/articles/2011/06/15/35thomas.h30.html Discuss with your table group; What surprised you? Any a-has? What does the author mean by “knowledge-driven schools?”

Scoring Rubric Julie +1=82 Rating 4 3 2 1 Justifying/arguing (ELD 11.4) Justify opinions or persuade others by making connections and distinctions between ideas and texts and articulating sufficient, detailed, and relevant textual evidence or background knowledge, using the appropriate register. Justify opinions and positions or persuade others by making connections between ideas and articulating relevant textual evidence or background knowledge Justify opinions by articulating some relevant textual evidence or background knowledge. Only states opinion/s, no justification or evidence and references are absent or not relevant. Writing (content) Makes an assertion and explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows all writing conventions for sentence structure, spelling and topic-specific word choices and vocabulary with detailed sentences. Explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows some writing conventions writing, with few errors in sentences, spelling. Partially explains ideas, topics and theme of the prompt question, does not use grade level appropriate vocabulary, sentences short and not detailed. Does not explain topic or theme, does not refer to prompt question.

Formative assessment analysis: Justifying / Arguing and Writing Content Anthony +2=84 Written information from paragraphs - 3 ways to look at the data Number of responses at each level Descriptive statistics Student samples

Responses at each level Anthony +2=86 1 2 3 4 6 11 9 Look at the number of responses at each rubric level – what are the implications? (average is 2.76) Discuss – how might you respond? Make 2-3 statements about this data

Descriptive Statistics Anthony +2=88 What is the summary of your descriptive statistics? Justifying / Arguing Writing Content Discuss – what are the implications overall or for each area? Make 2-3 statements about this data

Calibration With Your Team Annette +20 = 108 min Divide up student samples Review scores for consistency Everyone reads the three focus students Agree on scores on focus students Rating 4 3 2 1 Justifying/arguing (ELD 11.4) Justify opinions or persuade others by making connections and distinctions between ideas and texts and articulating sufficient, detailed, and relevant textual evidence or background knowledge, using the appropriate register. Justify opinions and positions or persuade others by making connections between ideas and articulating relevant textual evidence or background knowledge Justify opinions by articulating some relevant textual evidence or background knowledge. Only states opinion/s, no justification or evidence and references are absent or not relevant. Writing (content) Makes an assertion and explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows all writing conventions for sentence structure, spelling and topic-specific word choices and vocabulary with detailed sentences. Explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows some writing conventions writing, with few errors in sentences, spelling. Partially explains ideas, topics and theme of the prompt question, does not use grade level appropriate vocabulary, sentences short and not detailed. Does not explain topic or theme, does not refer to prompt question.

Assessment Commentary Prompts Annette +12 = 1:20 A. Go to Taskstream and locate your EdTPA Handbook (Content Area) Read as a group B. Generic Assessment Commentary Prompts Identify Objectives, Standard, & Assessment Provide a graphic or narrative that summarizes student learning Describe the patterns of learning for the whole class and specific students Describe the evidence for the 3 focus students. Explain how feedback was provided to the 3 focus students How will you support students to apply feedback? Explain student struggling with language to develop content understanding. Based on student performance, what are your next steps of instruction for whole class and each of the 3 focus students? Justify next steps with research or theory

Assessment Data Analysis Drafts due Monday 6th on EDSS 511 Forum Bold prompts due to Teri Week 11 & All prompts due to Anne René Week 12 Provide data in a visual for # of students that met content objective Provide data in a visual - Median, mean, mode & range of scores on the content objective rubric  Provide data in a visual for # of students that met the language objective Provide data in a visual - Median, mean, mode & range of scores on the language objective rubric  Describe the patterns of learning for the whole class and specific students. Describe the evidence for the 3 focus students. Based on student performance, what are your next steps of instruction for whole class and each focus student? Use research and theory to back up your instructional strategies. What did your students prefer as support strategies? Share survey data in a visual (chart, graph,...). Provide data in a visual - Median, mean, mode & range of preferences for  What did you learn about the support strategies that you implemented based on the student surveys? Which will you use again? When? Why? What might you change and why? What did you learn about action research? How could you do this again? What might you change and how?

BREAK

Ticket Out The Door Refer to your Assessment Commentary Describe the patterns of learning for the whole class and specific students Describe the evidence for the 3 focus students Based on student performance, what are your next steps of instruction for whole class and each of the 3 focus students? Refer to your Assessment Commentary (Feedback to Guide Further Learning section)