AFT 7/12/04 Marywood University Using Data for Decision Support and Planning

AFT 7/12/04 Marywood University Presenters Dr. Marie Loftus Dr. Barbara R. Sadowski Marywood University

AFT 7/12/04 Marywood University Session Overview  Group activity followed by discussion  Starting the research process  Break  Group activity – questionnaire analysis and discussion  Research results  Question and answer

AFT 7/12/04 Marywood University Group Task 1  Develop 2-3 questions of interest  For each question list: – a) what data could be used –b) factors that might affect the process of obtaining data _ c) potential problems with existing data – d) environmental factors that might bias the data or interpretation of the results –e) relevant existing data sources

AFT 7/12/04 Marywood University  To determine if a teacher’s Thinking Math training has an effect on student achievement in Mathematics. Research Purpose

AFT 7/12/04 Marywood University Questions Considered  Does the mathematics training of teachers have an effect on the math achievement of students they teach?  Does professional development training of teachers translate into higher student achievement?

AFT 7/12/04 Marywood University Practicality of Research Options  Question 1: – Difficult to get data on teacher’s background in mathematics –Measurement difficulty in defining college course work with in-service professional development  Question 2: -Teacher training data available -Non-transient teacher population -Well-documented training program -Student achievement data available

AFT 7/12/04 Marywood University Other Variables Considered  What constituted teacher training? How many TM workshops?  Did TM trained teachers actually implement the principles? Which ones were implemented? How confident did the teachers feel about their ability to implement?  Was years of teaching experience a factor in how well TM was implemented?

AFT 7/12/04 Marywood University Other Variables Considered  Was number of years teaching at grade level a factor?  Was the math background of the teacher a factor in how well TM was implemented?  How much of the time did the teacher actually implement TM when teaching math?

AFT 7/12/04 Marywood University Other Variables Considered  What student variables might affect the score on the achievement test? Reading level? SES? Title 1 support? The number of TM trained teachers they had before reaching fifth grade?  Would school climate affect how the teachers felt about implementing TM math principles? Did the principal support their use of TM?  Ability grouping in the assignment of students

AFT 7/12/04 Marywood University Research Question 1 Is the number of years students have a TM trained math teacher a predictor of how they will perform on a math achievement test?

AFT 7/12/04 Marywood University  Total enrollment 8,673 students  Economically disadvantaged54%  Mobility25.2%  Schools18 –13 elementary (Pre K-5)  Teachers712 –strong collective bargaining history with the Scranton Federation of Teachers Scranton PA School District

AFT 7/12/04 Marywood University Data Collection  Survey instrument developed collaboratively by Thinking Math trainers and researchers  Information session held for teachers and administrators  Surveys administered to TM teachers  Student data collected at school sites

AFT 7/12/04 Marywood University Sample The study sample was teachers and 5 th grade students in:  4 of 13 elementary schools –3 schools with TM trained teachers at the 5 th grade level. –1 school with TM trained teachers, none at the 5 th grade level

AFT 7/12/04 Marywood University Sample Schools  All offer Title 1 reading, two also offer math  Fifth grade enrollments are 70, 55, 44, and 34

AFT 7/12/04 Marywood University Sample Schools  Percentages of low income students: 28, 36, 46, and 77% (PA Profiles, )  Class organization –heterogeneous and homogeneous –departmentalized and self-contained

AFT 7/12/04 Marywood University Group Task 2  Review/analyze teacher questionnaire results  Consider possible reasons for the results of the scale item means  Are the scales correlated? Which ones?  What other variables are related to the scales?  What changes would you make to improve the instrument?

AFT 7/12/04 Marywood University Teacher Survey Results

AFT 7/12/04 Marywood University Teachers Surveyed  33 of 37 are female  Pre-K - 5, Title 1, Learning Support  10% have < ten years teaching experience  70% have twenty or more years of experience  average of 1.5 ER&D courses beyond TM

AFT 7/12/04 Marywood University NCTM 4/ 10/ 03 Scranton SD - Marywood University Teacher Survey Scales  Confidence Scale ( 7 items) Rate yourself on your confidence in... e.g., accepting multiple solutions. 1 = Not confident 10 = Very confident  Scale Mean = 8.57 s.d. = 0.80

AFT 7/12/04 Marywood University Teacher Survey Scales  Implementation Scale (7 items) To what extent do you... e.g., have your students explain and justify their solutions to problems? 1= Not at all 10 = To a great extent  Scale Mean = 8.1 s.d. = 0.89

AFT 7/12/04 Marywood University Influence of TM Scale Items 1. How frequently do you incorporate the TM principles in math instruction? 2. To what degree has your delivery of math instruction changed as a result of TM training? 3. How has the depth of the math content you are teaching changed since your TM training? 4. How much has TM research added to your content knowledge of mathematics?  Scale Mean = 8.19 s.d. = 0.97

AFT 7/12/04 Marywood University Correlations Confidence Implementation Influence of TM.395 *.429 ** Confidence.877 ** * p<.05 ** p <.01 n = 37, two-tailed

AFT 7/12/04 Marywood University Regression  Implementation as the dependent variable  Influence of TM as independent variable  Confidence in TM Principles as independent variable  89% of the variance in implementation is accounted for by the two independent variables

AFT 7/12/04 Marywood University Experience and Implementation Teacher Experience N Mean s.d. Less than 20 years More than 20 years

AFT 7/12/04 Marywood University Teacher Comments on TM  “I now change problems to reflect my class, linking school math to real world experiences.”  “I now use more manipulatives.”  “It has given me new insight for teaching multiple strategies and meeting individual needs.”

AFT 7/12/04 Marywood University Student Achievement Results

AFT 7/12/04 Marywood University Correlations: SAT – PSSA Math

AFT 7/12/04 Marywood University Correlations: Reading - Math

AFT 7/12/04 Marywood University Research Hypotheses  Students with 5 th grade TM trained teachers will perform significantly better on PSSA 5 th grade Math tests compared to the control group.  Students with 5 th grade TM trained teachers will perform significantly better on PSSA 5 th grade Math tests than the district and state.

AFT 7/12/04 Marywood University Math Performance Levels ***_ 2 = 14.6; df =3, p <.005

AFT 7/12/04 Marywood University Math Performance Levels Chi-square is not significant.

AFT 7/12/04 Marywood University Mean Comparisons * t = 2.40 ; df = 39, p <.05 ** t = 2.74 ; df = 195, p <.01 *** t = 4.0 ; df = 41, p <.000

AFT 7/12/04 Marywood University Mean Comparisons ** t = 2.91 ; df = 166, p <.005 *** t > 20 ; df = 167, p <.000

AFT 7/12/04 Marywood University Mean Comparisons Mean difference is not significant.

AFT 7/12/04 Marywood University Teacher-Student Results

AFT 7/12/04 Marywood University Mean Comparisons * t = ; df = 197, p =.081 ** t = ; df = 197 p =.081 *** t = -2.04; df = 197, p <.05

AFT 7/12/04 Marywood University Research Question Can math achievement be predicted from teacher experience, professional development and implementation of the TM principles?

AFT 7/12/04 Marywood University Analysis  A multiple regression was run with SAT math procedures as the criterion variable, and three independent variables: years of experience at grade level, professional development and scale score on implementation of TM principles.  A second multiple regression used SAT problem solving as the dependent variable.

AFT 7/12/04 Marywood University Math Procedures Regression * p =.066 *** p <.001

AFT 7/12/04 Marywood University Math Problem Solving Regression ** p <.05

AFT 7/12/04 Marywood University Reading Comprehension Regression * * p =.023 *** p <.000

AFT 7/12/04 Marywood University Conclusions  Teacher confidence in TM principles and perception of the importance of TM are predictors of implementation which is in turn a predictor of student mathematics achievement.  The number of years a student has a TM trained teacher is a predictor of student achievement in mathematics.

AFT 7/12/04 Marywood University Conclusions  Students who have a 5 th grade TM trained teacher performed significantly better on the PSSA 5 Th grade Math Test than the control group and the district.

AFT 7/12/04 Marywood University Research Purpose  The research results suggest that TM training does have a significant effect on student achievement.

AFT 7/12/04 Marywood University Topics for Further Research  Does TM training affect student achievement in Reading Comprehension?  What is the relationship between TM training, student reading comprehension and student mathematics problem-solving achievement?

AFT 7/12/04 Marywood University Questions?