1 What Are We Doing Here Anyway? Vision for our Work: Effective Science Learning Experiences Dave Weaver RMC Research Corp.

Statewide Science Achievement 2

By the 2010 Numbers 26,211 Grade 5 students met science standard 41,184 Grade 8 students met science standard 50,816 Grade 5 students did not meet science standard 34,328 Grade 8 students did not meet science standard 3

4 Conclusion From 2009 Evaluation The infrastructure to support the use of a core curriculum of inquiry-based science instructional modules is in place and is functioning adequately in the schools visited. Although these conditions are necessary for the implementation of inquiry-based science instruction, they are not sufficient to raise student achievement as measured by the science WASL/MSP.

Lack of Vision & Purpose “In most instances, principals, lead teachers, and system-level administrators are trying to improve the performance of their schools without knowing what the actual practice would have to look like to get the results they want at the classroom level.” (City, 2009, p 32) There is often a “lack of an agreed-upon definition of what high-quality instruction looks like.” 5

Washington State LASER Responded In early 2009 LASER set out to develop a logic model and theory of action Draft shared with science leadership in Spring 2009 Incorporated into the Science Partnership Academy Meetings Subcommittee formed to revise contents of Box 3 (Theory of Action) Current version completed in July

Logic Model Review Committee Joanne Johnson Anne Kennedy Kim Klinke Mark Lewis Kirk Robbins Adrienne Somera Dave Weaver 7

8 Logic Model

9 LASER PD Theory of Action If all science professional development is focused on helping teachers use research- based instructional practices, materials, and assessments so that each student: –Reveals preconceptions, initial reasoning, and beliefs; –Is intellectually engaged; –Uses evidence to generate explanations; –Communicates and critiques their scientific ideas and the ideas of others; –Makes sense of the learning experience and draws appropriate understandings; –Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways; and –Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes; Then (we believe): –Student science achievement would increase; –More students would enroll and successfully complete challenging and advanced science courses at the high school level; –More students would seek further studies beyond high school in STEM content and would seek STEM careers; –More students will seek to be engaged in STEM-related activities in out-of-school time, and; –More students would have the capacity to make informed decisions based on scientific information and scientific ways of knowing.

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 10

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 11

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 12

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 13

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 14

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 15

Comparing Old and New Box 3 Teachers use research-based instructional practices, materials, and assessments so that students: 1.Draw upon a deep foundation of usable knowledge within the context of a conceptual framework 2.Are intellectually engaged and motivated 3.Reveal preconceptions and reasoning 4.Use evidence to generate explanations 5.Communicate and critique their scientific ideas and the ideas of others 6.Reflect on how personal understanding has changed over time Teachers use research-based instructional practices, materials, and assessments so that each student: 1.Reveals preconceptions, initial reasoning, and beliefs 2.Is intellectually engaged 3.Uses evidence to generate explanations 4.Communicates and critiques their scientific ideas and the ideas of others 5.Makes sense of the learning experience and draws appropriate understandings 6.Makes connections between new and existing scientific concepts by understanding and organizing facts and information in new ways 7.Reflect on how personal understanding has changed over time and recognize cognitive processes that lead to changes 16

WHAT EVIDENCE DO WE HAVE THIS MIGHT WORK? Results from a quasi-experimental research study of the schools participating in the Observing for Evidence of Learning project The following material is based upon work sponsored by the National Science Foundation under Grant No. ESI to the Institute for Systems Biology. 17

Observing for Evidence of Learning 18 Project Director: Caroline Kiehle, M.Ed. Senior Researcher: David Weaver, M.S. Funding support by:  National Science Foundation  Department of Education through Washington State Office of the Superintendent of Public Instruction  The Seattle Foundation

OEL Theory of Action (Purpose) Student science achievement will increase if science teachers use instructional strategies and instructional materials to: –Elicit students’ initial ideas, –Engage students intellectually with important science content, –Provide opportunities for students to confront their ideas with evidence, –Help students formulate new ideas based on that evidence, and –Encourage students to reflect upon how their ideas have evolved, 19

OEL Essential Elements (Process) OEL Cycle 20

Quasi-Experimental Research School was the unit of change Matched each of the 21 participating schools with demographically similar non-participating schools –Matched on grade range, enrollment, FRL, ELL, ethnicity, community type All parameters agreed within 3% except Asian (difference 7%) Trends between 2004 & 2010 with baseline in 2006 No need for statistical adjustments 21

OEL Schools Compared to a Matched Set of Nonparticipating Schools 22

OEL Schools Compared to a Matched Set of Nonparticipating Schools 23

OEL Schools Compared to a Matched Set of Nonparticipating Schools 24 Significant Gap

OEL Schools Compared to a Matched Set of Nonparticipating Schools 25 Significant Gap

Comparison of Low Socioeconomic Schools 26

Comparison of Low Socioeconomic Schools 27

Comparison of Low Socioeconomic Schools 28

Comparison of Low Socioeconomic Schools 29

Observing for Evidence of Learning A clear process applicable to unique needs of teachers and students –OEL Essential Elements –School-based collaboration A clear purpose –OEL Theory of Action –Provided the context for the collaboration Deprivatizes practice through collaboration 30

In Your Teams! Where does your current professional development offerings support the new Box 3 of the Logic Model? What does your team need to do to develop an "agreed-upon definition of the high-quality instruction" that is described in box 3? 31

32 ?? Questions ?? Dave Weaver RMC Research Corp., Portland, Oregon