Jay Dubner Columbia University 2008 AGU Fall Meeting December 16, 2008 Teacher Research Programs: An Effective Form of Professional Development to Increase Student Achievement & Benefit the Economy

Practice What You Teach Columbia University’s Summer Research Program for Science Teachers was established in 1990 by Dr. Samuel Silverstein to contribute to the improvement of science achievement of students by providing New York metropolitan area middle and high school teachers with experiences in the practice of science. Teachers become members of research teams for 2 consecutive summers.

middle and high school science teachers 88% public schools 55% women 46% minorities 7

- Program participation open to New York metropolitan area public, private & parochial science teachers - Program receives 5 to 7 applications for each available Fellowship - 12 new participants are accepted each summer, joined by 12 returning participants from previous summer (peer coaches) Program Background

Two Summer Commitment - 8 Weeks immersion in each of two consecutive summers - 4 days per week in research lab - 1 day per week in pedagogical meeting Weekly summer seminar series serves several purposes: -Broaden Science Knowledge: Columbia’s faculty or a speaker from outside the University gives a talk on a topic of scientific interest. -Develop a Professional Learning Community: Meetings provide protected time for teachers’ energy to be directed towards solving persistent and common problems encountered by students. Teachers lead hands-on demonstrations, engage in discussions of common classroom problems, and exchange ideas on solutions that work. -Peer Coaching: Second-year participants provide guidance and encouragement to first-year participants -Program Performance: Meetings enable the members of the Advisory Committee to become acquainted with the teachers personally, and to learn about their professional needs and concerns. SRP culminates with a three-day End-of- Summer Symposium at which time the program’s Advisory Committee is able to assess each teacher's grasp of the principles on which his/her work is based. Program Summary

Stipend - $6,000 each summer Classroom Enhancement Funds - $1,000 following each of the two summers Research Group Funds – $1,000 in each of the two summers Travel to Professional Conference – Science education and/or scientific society conferences in each of the school years following participation in the program International Program – Following completion of program, 2 teachers are selected to participate in an exchange of science teachers with Singapore’s Ministry of Education and 2 teachers are selected to participate in a 3rd summer (winter?) of research in Australia Program ‘Perks’

School Year Classroom Support Teachers are afforded the opportunity to have the graduate student with whom they worked alongside in the lab, visit their classrooms one day a month.

Evaluation A unique aspect of Columbia University’s Summer Research Program is its emphasis on evaluation. Evaluation has informed and guided the Program’s evolution from its inception, and enabled it to respond to the changing needs of teachers

Student Outcomes Studies 1.Columbia’s Summer Research Program (1993-present) –Collected data from the NYC public high schools, the program’s largest cohort, with the assistance of the school science department chairs and the NYC Dept. of Ed.’s Division of Assessment & Accountability 2.NSF Multi-site Study ( ) –8 science teacher research programs –Arkansas, California, Georgia, Idaho, New York, Oregon, Texas, & Washington State

Instruments Administered NSF Multi-site Study Pre-program survey* Post-program survey* Mentor survey* Student Attitudinal Survey* Student Cognitive Tests –Biology & Chemistry *Surveys available at (Revised versions of program and mentor surveys available at

Comparison of Study and Control Science Teachers (1) (1) Data for 32 Study teachers and 32 Control teachers Study Teachers Comparison Teachers Undergraduate or graduate major or minor in biology, chemistry physics, earth science or math 94%87% Advanced degree63%62.5% Hours course credit beyond highest degree 34 hours54 hours Years of teaching experience Years of science teaching experience Hours/week spent outside of class preparing for teaching 9.7 hours9.25 hours Assigned textbook covered in science course 60%59% Used textbook publisher’s tests rarely or never 68% Used textbook publisher’s tests sometimes 18%26% Used textbook publisher’s tests frequently 14%6%

Self-reported Changes in Attitudes and Classroom Practices Scale Study teachersComparison Teachers Pre- teaching survey Post- teaching survey Change Pre- teaching survey Post- teaching survey Change Traditional Goals & Objectives Inquiry Goals & Objectives * Traditional Teaching Methods * Traditional Student Activities Inquiry Student Activities * Teacher Efficacy Number of teachers =58 * Difference in two change scores is significant at the p<0.05 level The items that constitute each scale were identified through a confirmatory factor analysis and the scale scores are standardized factor scores with a mean of 250 and a standard error of 50. Information about item wording and factor loadings can be found at These comparisons are based on 58 teachers and comparison teachers who completed all of the items comprising all of the scales. Five teachers failed to complete all of the items in one or more of the scales.

Student Assessments of Classroom Practices StudyComparison Number of Students Mean Number of Students Mean Reflected on course material by writing in a notebook. 2 2, , * Used primary sources such as journals. 1 2, ,4691.8* Listened to guest speakers or went on field trips. 1 2, , * Explored career opportunities in math or technology. 1 2, , * Teacher has encouraged me to think about math/ science career. 2 2, , * Difference between Study and Comparison teachers’ students was statistically significant at the p<0.05 level 1 Measured on a 5 point scale where 1 = never, 2 = 1-2 times a month, 3 = 1-2 times a week, 4 = almost every class, 5 = every class in response to the question “Approximately how often did you engage in the following learning activities in the mathematics/science class that you are currently taking?” 2 Measured on a 5 point scale from strongly disagree (1) to strongly agree (5)

Changes in Student Achievement in Science Subject Study Teachers’ StudentsComparison Teachers’ Students Pre-course Post- course ChangePre-course Post- course Change Second year Biology * * Chemistry * * * Difference in two change scores is significant at the.05 level The test scores were derived through IRT methods and converted into a scale score with a mean of 250 and a standard error of 50. The student achievement scores are derived from tests constructed by Westat, and the Westat tests were built on sets of items that were used previously in the National Assessment of Educational Progress (NAEP), the Third International Mathematics and Science Study (TIMSS), and the Longitudinal Study of American Youth (LSAY).

2008 Multi-Program Surveys (5 programs participated)

Summer Research Program 1993 – 2008 Pre-program survey Post-program survey Mentor survey Spring implementation survey Student participation in Science Clubs and Intel-type competitions Scores on NYS standardized science exams (Regents)

2008 Teacher Survey Data 100% reported developing new or revised content to lessons and/or labs* *341 standards-based lesson plans are on the program’s website: 91% reported increasing hands-on activities in their classrooms and/or new laboratory exercises in response to their experiences at Columbia 78% reported including lessons on science careers and related job requirements with their students 74% reported introducing new technologies in their classroom instruction (e.g.; chromatography, pipetting, PowerPoint) 61% reported reading scientific journals more frequently 52% reported increased requirements for formal written reports and/or oral presentation requirements 48% reported assuming new leadership roles/responsibilities in their school/district/region

Columbia’s Summer Research Program collects quantitative student data from NYC public high schools - Study Group -- Students in science classes of program teachers - Control Group -- Students in science classes of non- participating teachers from the same schools

Intel Science Talent Competition

(20 datasets)

NYC Economic Impact Regents Diploma – Must pass 5 Regents exams 1 (includes 1 Science Exam) NYC spends $12,930 per public school student 2 Student takes 5 courses/year = $2,586 per course 20 more students passing Regents over 2 years 20 students X $2,586 = $51,720 each teacher/year 1 New York State Education Department, New York State Total Public Report Card, The New York Sun, New York Outpacing Other States in School Spending, April 4, 2006

National Economic Impacts High School dropouts earn $392,000 less in lifetime earnings than a high school graduate thereby paying less taxes 1. At a tax rate of 20%, loss of $78,400 in local and federal revenues. High School graduates are less likely to commit crimes. Increasing HS completion of males by just 1% would reduce criminal justice system costs by $1.4 billion per year High School graduates receive higher salaries thereby not needing food stamps, housing assistance, etc. Estimated annual savings are $8 billion per year 1 Fogg, N., Harrington, P., & Khatiwada, I., The Longterm Labor Market Consequences of Dropping out of High School in Pennsylvania, Center for Market Labor Studies, Northeastern University, October 2007

Publishing Our Findings “Scientific Work Experience Programs for Teachers: A multi- site evaluation of impact on teachers and on student achievement” –Status: Very final draft form. Expected to be submitted in the next few weeks for peer review: Educational Evaluation and Policy Analysis “High School Science Teacher Participation in Columbia University’s Summer Research Program Improves Student Interest and Achievement in Science” –Status: Readying for final draft. To be submitted for publication in the next 2 months. Publication to be determined.

Current Funding Sources Ambrose Monell Foundation Bay and Paul Foundations Hebrew Technical Institute Mellam Family Foundation National Institutes of Health – SEPA Program National Science Foundation New York Times Company Foundation

Columbia University’s Summer Research Program for Science Teachers