1. Turn in Pink Management Rubric to Anne René
Data Analysis
Session 10 November 3, 2017
Turn in Pink Management Rubric to Anne René
Sit with subject team with whom you have collected action research data

2. Assessment is integrally related to curriculum and instruction.
Julie +2=2
Curriculum
SLOs
Instruction
Assessment
Quality teachers use evidence of learning (assessment)
to inform what they teach (the curriculum)
and how they teach (instruction)
(SLO = Student Learning Outcomes/Objectives)

3. The outcomes process (SLO’s) Julie +2 = 4
Collect data
DATA: diagnostic, formative, checking for understanding, feedback…
What does it mean?
Assessment
Evaluation
Inform Instruction, (summative) grades, etc.
Decision making

4. WestEd.org Data Literacy for Teachers Julie +2=6
“What do we want students to know and be able to do???”
Data literacy for teaching is the ability to transform information into actionable instructional knowledge and practices by collecting, analyzing, and interpreting all types of data (assessment, school climate, behavioral, snapshot, longitudinal, moment-to-moment, etc.) to help determine instructional steps.
It combines an understanding of data with standards, disciplinary knowledge and practices, curricular knowledge, pedagogical content knowledge, and an understanding of how children learn.
Answers…
“How will we know when they get there?”

5. When should we look at data? Anne +2=8
Formative and Progress Monitoring assessment informs both teachers and students about student understanding at a point when timely adjustments can be made.
Helps teachers determine next steps during the learning process as the instruction approaches the summative assessment of student learning.

6. What is the basic process? Anne +1=9
Analyze data – look at multiple sources Examine possible relationships or causes Consider impact and consequences (intended and unintended)

7. And then…. Anne +1=10 Transform Information Into a Decision
Determine next steps
Make instructional adjustments

8. Let’s talk about it! Anne +10= 20
Turn to a partner, brainstorm examples of formative assessment that could be used to help you create next steps / inform instruction
Read the article:
5 ways to use data to improve your teaching
teaching
Discuss: What are some key points?

9. “Music gives a soul to the universe, wings to the mind, flight to the imagination. – Plato Teri +1=11

10. Action Research
Action research is “a disciplined process of inquiry conducted by and for those taking the action. The primary reason for engaging in action research is to assist the “actor” in improving and/or refining his or her actions.” (Richard Sagor) Teri +1=12

11. Steps of Action Research
Selecting a focus
Clarifying theories
Identifying research questions
Collecting data
Analyzing data
Reporting results
Taking informed action
Teri +1=13

12. Socio-Political History of Rock ‘n Roll
Using era-specific music and lyrics to enhance student understanding of U.S. History.
Research ”supports the use of music as a mnemonic device for the learning and recall of new information.” (Michelle Lazar)
To what extent can using music as a vehicle for teaching increase student comprehension of U.S. History during the second half of the 20 century?
Pre-Post Tests (Internal Assessment)
Internal Assessment and External Assessment (AP Exams) results
Preparing and recording charts that highlight results
Modifying resources, adjusting topical coverage
Teri +2=15

13. Considerations / Narrowing the Focus
Have students taken U.S. History before? (11th or 12th grade)?
AP U.S. History / College Prep U. S. History?
What Middle School did they attend?
Did they like History before signing up for the class?
Are they currently and/or have they been involved in musical training?
What is the first / primary language?
Comparing results to the other school this is taught at (RBV) and are there similar / different results - checking District demographics between schools for further comparison.
Did they develop a greater interest for history after completing the class?
Teri +2=17

14. Student Ethnicity in VUSD
The district serves a diverse population of students, including more than 3,000 in its special education programs and more than 6,000 who are learning English as a second language. Enrollment has declined over the last several years from about 26,000 students in 2000 to approx. 22,000 in 2012.
Student demographics include:
58% qualify for free lunch 4% African American
24% English learners 3% Asian
10% homeless 3% two or more races
60% Hispanic 1% Filipino
28% White 1% Pacific Islander
Teri +2=19

15. Demographics for MVHS v. VHS
Demographic Information for MVHS: Demographic Information for VHS GATE students: 22% GATE students: 17% English learners: 3% English learners: 18% Reclassified as English proficient: 17% Reclassified as English proficient: 32% Migrant education program participants: 1% Migrant education program participants: 5% Eligible for free/reduced price lunch: 29% Eligible for free/reduced price lunch: 61% Special education within past two years: 6% Special education within past two years: 16% Teri +2=21

16. Demographics – Parent Level of Education
Parent's Education for MVHS: Parent’s Education for VHS
Graduate School: 18% Graduate School: 8%
College Graduate: 33% College Graduate: 19%
Some College: 28% Some College: 22%
High School Graduate: 13% High School Graduate: 17%
Not a High School Graduate: 8% Not a High School Graduate: 35%
Teri +2=23

17. Middle Schools Teri +1=24

18. Level of Accuracy on Pre and Post Tests (Internal Assessments) Teri +5=29 Table 1: Pre Test Results (out of 56) (Table 2: Post Test Results (out of 54) (Bold-Italicized = more than ½ answered correctly) Q# 1. 3 2. 8 3. 40 4. 11 5. 34 6. 32 7. 20 8. 9. 7
10. 21
11. 4
12. 10
13. 5
14. 31
15. 12
16. 14
17.
18.
19.
20.
21.
22.
23.
24. 19
25.
26.
27.
28.
29. 13
30. 18 Q# 1. 6 2. 27 3. 33 4. 20 5. 41 6. 38 7. 8. 12 9. 21
10.
11. 35
12. 22
13. 5
14.38
15.
16. 29
17.
18. 13
19. 23
20. 10
21. 28
22. 48
23. 14
24.
25. 2
26. 32
27. 34
28.
29.
30. 36

19. External Assessment (AP U.S. History Exam) Teri +5=29
All students
Total # Students
Average Score
Score 5 4 3 2 1
# Students 12 19 32 23 88
2.295
% by score
.02%
13%
22%
36%
26%
AP U.S. History
Elective Class
Total # Students
Average Score
Score 5 4 3 2 1
# Students 12
2.833
% by score
.08%
16%
33%

20. Results Teri +2=31
Results have been gathered and charted for 3 years and found:
On average, there was 26% improvement on internal assessments;
An average 10% increase on external assessments over students who had not taken the Rock ‘n Roll class; and
A greater understanding and passion for history after taking the class, according to end of the semester essays.

21. Action Research Project Annette +2=33
Academic Year (Spring 2016)
26 Teacher Candidates Conducted Action Research Projects in Advanced Clinical Practice
Intent of the Action Research Project
Teacher candidates and program faculty to assess the impact of using effective strategies for K-12 student learning, namely English learners
Pre- and Post-test Action Research where students completed a writing assignment without effective strategies (pre-test) and then completed a second writing assignment (post-test) with multiple effective strategies

22. Action Research Project
Effective Strategies Annette +2=33
Explicit Reading Supports: e.g., highlighting reading, chunked reading, different text for each reading level
Explicit Vocabulary Supports: e.g., graphic organizer, bulletin board, tea party
Explicit Group Supports: e.g., teacher chosen groups, assigned roles, tiered activities
Explicit Writing Supports: e.g., Venn diagram, T-chart, sentence starters
Rubric for the writing assignment aligned to content and language objectives (CCSS and ELD Standards)

23. Action Research Project Annette +2=33
Percentage of teacher candidates responding “yes” on the pre- / post-survey (N=26)
Indicating how effective they felt strategies used in post- test were
Pre-Test
Post-Test
Did your English learner focus student meet the content objective?
31%
84%
Did your English learner focus student meet the English language development objective?
32%
92%
Did your focus student with an IEP meet the content objective?
80%

24. Action Research Project
The following teacher candidates showed a significant growth in the proportion of students meeting the content and ELD standards after implementing language supports. (N=11 – 4 candidates highlighted in this presentation)
56% more students met the content objective after the language support

25. Action Research Project Teacher Candidate #1 Social Studies
Reading supports provided:
Chunked Reading, Visuals to accompany/replace text
p-value
0.56
Approx 0
Vocabulary supports provided:
Props, Graphic Organizer
Group supports provided:
Teacher Chosen Group, Cooperative Learning Elements, Active Involvement Techniques, Activity Choice, Multisensory Group Activity, Gradual Release of Responsibility
p-value
0.56
Approx 0
Other supports provided:
Sentence starters, paragraph template.

26. Action Research Project Teacher Candidate #1 Social Studies
Response from students when asked about their preferences:
A majority of the students cited the graphic organizer and paragraph template as useful. The student with the IEP missed most of the instructional part of the lesson, so did not cite a preference for supports. The English learner cited the hands-on group activity. The other student cited the hands-on group activity and the rubric.
What the teacher candidate learned:
That the various teaching supports made a significant difference, but many of the students need much more intensive help with their writing.

27. Action Research Project Teacher Candidate #2 Mathematics
Reading supports provided:
Highlighted Reading, Annotated Text
p-value
0.4
.002
Vocabulary supports provided:
4 Square Graphic Organizer
p-value
0.48
.0003
Group supports provided:
Teacher Chosen Group, Cooperative Learning Elements, Gradual Release of Responsibility
Other supports provided:
T-Chart, Sentence Starters

28. Action Research Project Teacher Candidate #2 Mathematics
Response from students when asked about their preferences:
The whole class liked talking with their group members and comparing the evidence they put on their Pros and Cons list. The student with an IEP especially liked the group conversations so they could make sure that they were on the right track. The EL student liked the sentence starters because it helped them understand where to start with their response. The other students I identified like the bolded and underlined text in the reading because it made it easier for them to understand.
What the teacher candidate learned:
I learned that my students need all types of supports in one activity. Sometimes what works for most students doesn't work for a few students and they need a slightly different version of what the rest of the class has or they might need extra instruction. I also learned that my students can justify their thinking when given the proper resources to do so.

29. Action Research Project Teacher Candidate #3 Science
Reading supports provided:
Visuals to accompany/replace text
p-value
0.94
Approx 0
Vocabulary supports provided:
4 Square Graphic Organizer
p-value
.57
.026
Group supports provided:
Teacher Chosen Group, Gradual Release of Responsibility

30. Action Research Project Teacher Candidate #3 Science
Response from students when asked about their preferences:
85% of students that performed the writing of the paragraph in the 3rd lesson stated that they thought the keywords provided were helpful in completing the paragraph. 75% of students responded that they thought the graphic organizer was helpful for defining scientific vocabulary keywords. The English learner, IEP student and a student who is easily distracted were with the majority - i.e. they thought the keywords provided for the paragraph writing was helpful and, with the exception of the EL student, responded that the graphic organizer was useful.
What the teacher candidate learned:
A writing exercise following a student-centric physical hands-on activity is more engaging to students than a writing assignment following a teacher-led demonstration.

31. Action Research Project Teacher Candidate #4 English
Reading supports provided:
Highlighted Reading, Chunked Reading, Visuals to accompany/replace text, Different Text for each reading level, Annotated Text
p-value
0.85
Approx 0
Vocabulary supports provided:
Other: ____________, Frayer Models
p-value
0.85
Approx 0
Group supports provided:
Teacher Chosen Group, Group Roles, Bloom's Taxonomy, Gradual Release of Responsibility
Other supports provided:
T-Chart, Sentence Starters, Template

32. Action Research Project Teacher Candidate #4 English
Response from students when asked about their preferences:
100% of my students said that it was much easier to write the paragraph the second time around. The strategies that the students stated were most helpful included: academic vocabulary review, paragraph writing template, and reading supports. My ELD class as a whole appreciated the writing template the most. The focus English Language Learner (Level 1) thought that the adapted text was the most helpful. I created my own adapted text that was much shorter, included visuals, and was annotated as well. My student with an IEP believed that the most helpful support was the academic vocabulary activities and review. My other student, a Level 3, appreciated the writing template the most.

33. Action Research Project Teacher Candidate #4 English
What the teacher candidate learned:
Through this action research process, I truly understood the direct impact that carefully planned and intentional supports have on student success. There is so much that we can do as teachers simply by identifying the diverse/unique needs in the classroom and meeting those needs. I learned that providing supports for students in no way enables them, but just provides them with the differentiation that they need in order to make the content accessible to them. It was incredible to see the exponential growth between the two paragraphs.

34. Action Research Project
Effective Strategies Annette +15= 71min
What strategies have you used that you felt were effective for your students? English learners?
Explicit Reading Supports: e.g., highlighting reading, chunked reading, different text for each reading level
Explicit Vocabulary Supports: e.g., graphic organizer, bulletin board, tea party
Explicit Group Supports: e.g., teacher chosen groups, assigned roles, tiered activities
Explicit Writing Supports: e.g., Venn diagram, T-chart, sentence starters
Rubric for the writing assignment aligned to content and language objectives (CCSS and ELD Standards)

35. Reading and Reflection Sit in Content Area Groups Julie + 10 = 81
Before analyzing your data, read the article: My Nine Truths about Data Analysis
Discuss with your table group;
What surprised you?
Any a-has?
What does the author mean by “knowledge-driven
schools?”

36. Scoring Rubric Julie +1=82
Rating 4 3 2 1
Justifying/arguing (ELD 11.4)
Justify opinions or persuade others
by making connections and distinctions
between ideas and texts and articulating
sufficient, detailed, and relevant textual
evidence or background knowledge, using the
appropriate register.
Justify opinions and positions or persuade others by making connections between ideas and articulating relevant textual
evidence or background knowledge
Justify opinions by articulating some
relevant textual evidence or background
knowledge.
Only states opinion/s, no justification or evidence and references are absent or not relevant.
Writing (content)
Makes an assertion and explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows all writing conventions for sentence structure, spelling and topic-specific word choices and vocabulary with detailed sentences.
Explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows some writing conventions writing, with few errors in sentences, spelling.
Partially explains ideas, topics and theme of the prompt question, does not use grade level appropriate vocabulary, sentences short and not detailed.
Does not explain topic or theme, does not refer to prompt question.

37. Formative assessment analysis: Justifying / Arguing and Writing Content Anthony +2=84
Written information from paragraphs - 3 ways to look at the data
Number of responses at each level
Descriptive statistics
Student samples

38. Responses at each level Anthony +2=861 2 3 4 6 11 9
Look at the number of responses at each rubric level – what are the implications?
(average is 2.76)
Discuss – how might you respond?
Make 2-3 statements about this data

39. Descriptive Statistics Anthony +2=88
What is the summary of your descriptive statistics?
Justifying / Arguing
Writing Content
Discuss – what are the implications overall or for each area?
Make 2-3 statements about this data

40. Calibration With Your Team Annette +20 = 108 min
Divide up student samples
Review scores for consistency
Everyone reads the three focus students
Agree on scores on focus students
Rating 4 3 2 1
Justifying/arguing (ELD 11.4)
Justify opinions or persuade others
by making connections and distinctions
between ideas and texts and articulating
sufficient, detailed, and relevant textual
evidence or background knowledge, using the
appropriate register.
Justify opinions and positions or persuade others by making connections between ideas and articulating relevant textual
evidence or background knowledge
Justify opinions by articulating some
relevant textual evidence or background
knowledge.
Only states opinion/s, no justification or evidence and references are absent or not relevant.
Writing (content)
Makes an assertion and explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows all writing conventions for sentence structure, spelling and topic-specific word choices and vocabulary with detailed sentences.
Explains ideas, topics and theme of the prompt question, using grade-level appropriate language and follows some writing conventions writing, with few errors in sentences, spelling.
Partially explains ideas, topics and theme of the prompt question, does not use grade level appropriate vocabulary, sentences short and not detailed.
Does not explain topic or theme, does not refer to prompt question.

41. Assessment Commentary Prompts Annette +12 = 1:20
A. Go to Taskstream and locate your EdTPA Handbook (Content Area)
Read as a group
B. Generic Assessment Commentary Prompts
Identify Objectives, Standard, & Assessment
Provide a graphic or narrative that summarizes student learning
Describe the patterns of learning for the whole class and specific students
Describe the evidence for the 3 focus students.
Explain how feedback was provided to the 3 focus students
How will you support students to apply feedback?
Explain student struggling with language to develop content understanding.
Based on student performance, what are your next steps of instruction for whole class and each of the 3 focus students?
Justify next steps with research or theory

42. Assessment Data Analysis Drafts due Monday 6th on EDSS 511 Forum Bold prompts due to Teri Week 11 & All prompts due to Anne René Week 12
Provide data in a visual for # of students that met content objective
Provide data in a visual - Median, mean, mode & range of scores on the content objective rubric 
Provide data in a visual for # of students that met the language objective
Provide data in a visual - Median, mean, mode & range of scores on the language objective rubric 
Describe the patterns of learning for the whole class and specific students.
Describe the evidence for the 3 focus students.
Based on student performance, what are your next steps of instruction for whole class and each focus student? Use research and theory to back up your instructional strategies.
What did your students prefer as support strategies? Share survey data in a visual (chart, graph,...).
Provide data in a visual - Median, mean, mode & range of preferences for 
What did you learn about the support strategies that you implemented based on the student surveys? Which will you use again? When? Why? What might you change and why?
What did you learn about action research? How could you do this again? What might you change and how?

43. BREAK

44. Ticket Out The Door Refer to your Assessment Commentary
Describe the patterns of learning for the whole class and specific students
Describe the evidence for the 3 focus students
Based on student performance, what are your next steps of instruction for whole class and each of the 3 focus students?
Refer to your Assessment Commentary
(Feedback to Guide Further Learning section)