Sentinel Site Visitor Panel Discussion Laura Krebs Dave Weaver RMC Research Corporation 111 SW Columbia, Suite 1200 Portland, Oregon (800) 788–1887

2 Agenda Introduction of Panel Members WebEx Protocols Background Discussion Questions Questions and Answers

3 Panel Members Don Blagsvedt Virginia (Ginni) Erion Karen Madsen Joyce Nishimura John Parker

4 WebEx Protocol This Session Is Being Recorded –Please keep your phone on mute unless you have a question * 6 to mute or unmute –Will be available at Meeting Logistics –Use chat box to submit questions –Raising your hand

5 Background Sentinel Site Selection: –Schools with significant LASER participation during the 3 years prior to the site visit –Schools visited 34 schools during school year 30 schools during school year –Used standard protocol and rubrics –Defined 2 groups of schools based on science WASL change for 2 years prior to site visit Demonstrated Significant Positive Gains Demonstrated Little, No, or Negative Gains

6 Sentinel Site Visits Mostly outside consultants Conducted web-based training sessions Each site visit: 1 ½ to 2 days –Principal interview –Teacher surveys –At least 3 interviews –At least 3 classroom observations

7 Question 1 As a result of your experience visiting schools and observing science classes what was one thing that you discovered about the teaching and learning of science?

8 Question 2 What were some of the characteristics of the schools that you felt had strong science instruction?

9 Question 3 What were some of the characteristics of the schools that you felt were struggling with their science instruction?

10 Question 4 What would be the best thing Washington State LASER could do to help teachers use the modules to effectively improve student learning?

11 Questions For the Panel Don Blagsvedt Virginia (Ginni) Erion Karen Madsen Joyce Nishimura John Parker Sentinel Site Data Collection Instruments:

12 Thank You! Further Questions? Contact: Dave Weaver RMC Research Corporation 111 SW Columbia, Suite 1200 Portland, Oregon (800) 788–1887

13 Sentinel Site Finding 1 There were a number of traits that site visitors consistently rated high, many of which were a direct outcome of Washington State LASER.

14 What Was Going Well At The School Level Materials Support System –The system for maintaining, storing, and refurbishing the instructional modules was effective and well organized. Condition of Modules –Teachers always received modules that were complete and ready for classroom use. Inquiry-Based Materials –The school was implementing 3 or more modules per grade level as the core science curriculum materials. Ninety-four percent of the teachers used the modules as the core science curriculum. Administrative Support –Strong evidence indicated that the school administrators were very supportive of inquiry- based science instruction. District Support –Strong evidence indicated that the district administrators were very supportive of inquiry- based science instruction. Sequence –All science teachers used the modules according to a sequence clearly prescribed by the district. Critical Mass –Most (80% or more) teachers in the school had attended the initial use training for all of the modules they used.

15 School Level Areas Receiving Low Scores Summative District Assessments –Very few school had districtwide or schoolwide summative assessments in science that were administered to students annually Formative Assessments –A few teachers (25% or less) had adopted a standard formative assessment strategy for science. Instructional Time –Science instructional time varied considerably amount teachers at the elementary level and few schools required a minimum amount of instructional time for science. Professional Development Time for Teachers –Teachers rarely had scheduled time during normal work hours to participate in school-based professional development in science. Partnership With Business, Informal Science, or Higher Education –A few teachers had a tentative partnership with a business, an informal science organization, or an IHE around science education.

16 Classroom Observation: High Scoring Alignment of Lesson Activities –Lesson activities addressed the stated learning objectives but there was some question about how the lesson activities would lead to a deeper student understanding of the learning objectives. Motivation –The lesson provided mostly extrinsic and some intrinsic motivation. The intrinsic motivation was truncated by the lesson structure and was relatively short lived. Understanding of Purpose –Throughout the lesson, many students understood why they were doing each activity but the purpose of activities could have been more explicit. Classroom Discourse –For the most part, students and teachers support and encourage respectful and constructive discourse around important science concepts, however, only some students feel comfortable asking questions, backing up their own claims, and/or critiquing claims made by others.

17 Classroom Observation: Low Scoring Lesson Closure –By the end of the lesson, the teacher provided a brief review, but students did not have an opportunity to fully make sense out of how the lesson related to science concepts. Application of Science –A few students applied something they learned in the lesson to a new context. Reflection and Meta-cognition –By the end of the lesson, students had some opportunity to reflect on their thinking but students were not asked to identify ways in which their thinking about the science concepts had changed.

18 Analysis of Gains Divided the schools into 2 groups –Based on: Percent of students who met the science standard Gains calculated over 2 years prior to the site visit –Group definition Schools that demonstrated an increase in student achievement Schools that had no change or decreasing student achievement The 2 groups of schools had significant demographic differences Percent of students who qualified for free or reduced price lunch Percent of Asian students

19 Finding 2 Although schools that demonstrated increasing student science achievement were significantly different demographically from those that did not, there were also significant differences in the instructional practices of the teachers observed by the site visitors.

20 Differences Between Gain Groups Science classes in schools that demonstrated an increase in student achievement were more likely to: –Have clear lesson objectives –Involve activities that clearly address the learning objective –Have students who understand the purpose of the lesson –Have students that are more intellectually engaged in the science content –Have students applying science content to new contexts –Have students engaged in science discourse –Motivate students intrinsically

21 Analysis by Achievement Ranking Divided the schools into 2 groups –Based on percent of students who met the science standard the year of the site visit –Group definition Schools at or above the state average Schools below the state average Methods –Regression analysis to determine which variables were significant predictors of the achievement ranking –Controlling for Student achievement the previous year Percent of students who qualify for free or reduced price lunch

22 Finding 3 Although schools that demonstrated above average student science achievement were significantly different demographically from those that were below average, there were also significant differences in the characteristics of the schools.

23 Characteristics of Above Average Performing Schools Instructional Time Allocated –The elementary school has a designated amount of instructional time allocated for science. Integration of Literacy –Many teachers integrated science and literacy through the use of supplementary reading materials and science notebooks. Professional Development Time for Teachers –Teachers occasionally had scheduled time during normal work hours to participate in school-based professional development in science. District Support –Evidence indicated that the district administrators were somewhat supportive of inquiry-based science instruction. Parent and Community Support –Evidence indicated that the parents/community were somewhat supportive of inquiry-based science instruction.

24 Principal Survey Online survey that closely paralleled data collected during sentinel site visits Selected schools with 15 hours or more of science professional development per teacher over a 5-year period prior to March 31, schools invited Survey administered between May 22 and July 31, principals completed the survey (19.4% return rate)

25 Principal Survey Analysis Regression analysis to determine which survey items were significant predictors of student achievement on the 2009 science WASL Controlling for the percent of students who qualified for free or reduced price lunch (FRL)

26 Finding 4 There were several items on the principal survey that were significant predictors of student performance on the science WASL above and beyond what could be attributed to FRL.

27 Predictors of Student Performance Schools that made an organized effort to identify instructional materials to fill the gaps Schools where the principals observe student using evidence to engage in discourse about science. Schools that provide time during the normal work day for and how often teachers participate in school- based science professional development. Schools that support professional learning communities that focus on improving science teaching and learning.

28 Regression Analysis Results Survey Item Grade Tested Adjuste d R 2 Change in R 2 pBeta Has your school or district made an organized effort to identify instructional materials to fill the gaps? <.001 *.352 Principal observation of classes: Students had opportunities to make claims, and/or use evidence to back up their claims or critique claims made by others. The lesson reinforced the notion that science is a process by which knowledge is constructed <.001 *.350 Is time scheduled during normal work hours for teachers in this school to participate in organized, school-based professional development specifically for science? *.048 How often do teachers participate in school-based professional development specifically for science? <.001 *.153 Has any of the PLC activities focused on improving science teaching and learning? <.001 *.211 Approximately what percentage of the PLC is devoted to science teaching and learning? *.047

29 Finding 5 There are themes among the various findings that further confirm and support recommendations and conclusion from last year and the planned shifts in Washington State LASER.

30 Significant School-Level Factors PD time for teachers during the school day Allocated science instructional time Professional learning communities Filling curriculum gaps Integration of literacy District, parent, and community support

31 Significant Instructional Factors Science discourse using evidence Purposeful instruction Intellectual engagement Intrinsic motivation Application of science skills

32 Conclusion The infrastructure to support the use of a core curriculum of inquiry-based science instructional modules is in place and is functioning adequately in the schools visited. Although these conditions are necessary for the implementation of inquiry-based science instruction, they are not sufficient to raise student achievement as measured by the science WASL.

33 Recommendation 1 Ensure that the professional development on research-based instructional practices is consistent and explicit across all of the LASER Alliances –Help teachers understand the elements of effective science instruction and use the modules as a means of carrying out the element with their students.

34 Recommendation 2 Increase support for school-based professional development that helps teachers: –Assume accountability for student learning that results from the use of the modules, and –Collaboratively implement the elements of effective science instruction. –Ample structure and leadership for success