Balancing the Call for Evidence- Based Research Designs With Formative Evaluation to Improve Implementation of Inquiry-Based Science Teaching Ken Wareham Lewis-Clark State College Catherine Callow-Heusser  ndVision Research & Evaluation, LLC

Math/Science Partnership (MSP) The intent of this program is to encourage institutions of higher education, local school districts, elementary schools, and secondary schools to participate in professional development activities that increase the subject matter knowledge and teaching skills of mathematics and science teachers.

Math/Science Partnership (MSP) Professional development activities must be sustained, intensive, classroom focused, and aligned with state and local standards and mathematics and science curricula.

Math/Science Partnership (MSP) These activities must result in a demonstrable and measurable improvement in student academic achievement in mathematics and science.

State 4 th Grade Math Exams: Percent of student at or above proficient.

State vs NAEP 4 th Grade Math Exams: Percent of student at or above proficient.

Identifying the Need. Data indicates the factor that is putting the district in AYP jail is Language Arts. Research review results in finding the Valle Imperial Project in Science, headed by Supt. Michael Klentschy. A partnership is formed with Klentschy to replicate his program. Teaching for Excellence in Science and Literacy Achievement (TESLA) is awarded spring 2007.

Objectives Increase teacher’s understanding of current research- based best practices related to teaching science using kit-based science modules (e.g. FOSS).

Objectives Increase teachers’ pedagogical skills in science by demonstrating and modeling evidence- based best practices in relation to using kit-based science modules.

Objectives Provide teachers with an in-depth knowledge of scientific concepts through intense study, skill enhancement, application, and demonstration.

Objectives Help teachers understand the interconnectedness of the scientific methods, literacy, and mathematics when learning science and incorporate these connections in the teaching process.

Objectives Provide long-term support to teachers via in-class mentoring and skill development.

20 Evaluation Design Formative evaluation to guide program improvement  Surveys  College faculty observation of classroom implementations with feedback  Student assessment data Summative evaluation  Comparison group design proposed but not feasible  Pre/posttest design based on SAT-9T gains in grade equivalent scores 20

21 Post-Workshop Survey (Fall 2009)

22 Post-Workshop Survey Responses

23 Post-Workshop Survey Responses: Variables

24 Assessment of Teacher Knowledge Pretest-Post: Variables 24 Variables Pretest Posttest

25 Assessment of Teacher Knowledge Pretest-Post: Inquiry 25 Inquiry Pretest Inquiry Posttest

26 Student Outcomes Pretest-Posttest (fall, spring) Stanford Achievement Test version 9 form T (SAT-9T)  Designed to reduce student reliance on memorized factual information and emphasize the unifying concepts and themes in science as outlined in the AAAS publication Science for All Americans (1990)  Subtests for science and language arts 26

27 SAT-9T Outcomes (Spring 2009) Grade Science Grade Equivalent Change Language Arts Grade Equivalent Change 4 th th th

28 Additional SAT-9T Outcomes Participants were 4 th, 5 th, and 6 th grade students Science  41% who were enrolled in participating classrooms for two years were at a 12 th grade equivalent  89% had scores that were at or above grade level Language Arts  45% were at at a 9 th grade or higher grade level equivalent  75% who were enrolled in participating classrooms for two years were above grade level 28

29 Contextual Factors Influencing Project Implementation and Evaluation External evaluator in another state College faculty local to project  Observe classrooms to improve FOSS implementation Varied academic preparation and experience in evaluation, K- 12 teaching, coaching/mentoring Which resulted in limited use of observation data for summative evaluation (due to reliability)  Unmeasured side effect: TESLA built capacity of faculty with regard to inquiry teaching, coaching/mentoring Teachers who volunteered to participate District/school administrators with varying levels of support and involvement 29

30 TESLA Idaho MSP Principal Investigator Kenneth L. Wareham, Ph.D., Associate Professor Lewis-Clark State College Division of Education th Avenue Lewiston, ID (office) (fax) Evaluator Catherine Callow-Heusser, President EndVision Research and Evaluation, LLC 41 E University Blvd #321 Logan, UT (mobile) (office) (fax) 30