2. RENEW Evaluation Sarah Hough, Ph. D. & Niki Sandoval, M. A. University of California, Santa Barbara

3. Purpose of RENEW Evaluation Are implementation goals met? Are project activities moving participants toward the specific pedagogical objectives of –Increasing participants’ understanding of mathematics Does the project help to retain teachers in the profession? –Increasing participants’ ability to implement national standards in mathematics –Increasing participants’ understanding of equity issues and their ability to work with English Learners

4. RENEW Grows Year 1 76 preceptees 30 preceptors 42 schools 8 districts Year 2 107 preceptees 125 b. teachers 29 preceptors 33 schools 8 districts Year 3 169 preceptees 56 preceptors 55 schools 9 districts

5. Current Participants 136 preceptees 110 new teachers attended summer institutes 36 preceptors 57 schools 9 districts

6. Activity Increases

7. What New Teachers Say Every time I attend a math institute I become more confident and empowered in math. I feel like a better teacher every second I am present in the institute. summer institute participant I feel more confident when analyzing statistical data now based on the activities we did as a group. summer institute participant Cross institute feedback in terms of growth in leadership What beginning teachers said they gained from the institutes. How many of the BTs signed up for participation in RENEW How many returning preceptees from previous years

8. Juggling Pedagogical Content Knowledge Student Diversity Subject Matter Beliefs About Math Instructional Strategies Beliefs About Teaching Efficacy Differences in Student Thinking Curriculum CONTENT PEDAGOGY STUDENT

9. Collecting Data About Beliefs  Each year renew beginning teachers fill out a questionnaire that includes:  Two graded response scales that measure beliefs about mathematics and beliefs about pedagogy

10. Beliefs About Mathematics and it’s Teaching Before Participation Math as procedures Math as investigation Teaching as “telling” In line with PSSM Not in line with PSSM

11. Change in Preceptee Beliefs Math as procedures Teaching as “telling” Math as investigation ß

12. Collecting Data About Learning Content Pre and post “statistics” concept maps from 22 beginning teacher participants at the UCSB institute were analyzed for change. Content Analysis--themes of institute were: measures of central tendency, graphical representations, probability, sampling and bias.

13. Content Analysis of Maps Graphs Venn Diagrams Bar charts T-charts Tables Statistic s Representing Pie-charts concept Chunk

14. Math Content Analysis of Maps

15. Example 3-pre map Statistics Median Baseball players Mean Average Middle When do we really use this? Histogram Bar graphs Mode Most Frequency Races

16. Example 3-post map Statistics Median Not always useful Mean Average Middle Box and whisker Bar graphs Mode Big 3 Most frequent Stem and leaf Reinforces results Graphs Venn Diagrams Pie-charts Bar charts T-charts Analyze Data

17. Changes in Subject Matter Structure Mean Pre Mean Post Range Pre Range Post #Concepts 24357-3913-58 Width+depth 15218-2912-37 #Chunks 460-100-18 #Links 010-10-5

18. Participant view of their own learning When participants were asked to reflect on their learning using the pre and post concept maps the majority of them agreed that they had added more concepts to their maps “ I ran out of room for all of my thoughts on my 2 nd concept map! What a great visual assessment of my learning. I hadn’t realized just how much I really did learn this week until I saw the difference between Monday’s and Friday’s maps. ”

19. Participant view of their own learning And, that their maps were more sophisticated and connected “ My first graph does not have as many connections with the sub categories as the second one. I have a deeper understanding of topics now, for instance, that the mean could get affected by a very big number, while the median could stay the same. ”

20. Change in PCK--Vertical Curriculum Knowledge “Prior to this week’s activities, I attributed statistics more to a college level or advanced math level. However, now I understand and am excited to see how statistics applies to the elementary level of teaching and learning.”

21. Change in PCK--Horizontal Curriculum Knowledge “What I have come away with is not only a better understanding of the content of statistics but more importantly, how to integrate it into my classroom. I can honestly say statistics was not a topic I covered in my first year of teaching but I feel I now have many ideas/topics that would be relevant to my grade level as well as have student interest. am very excited and plan to use Math By All Means –Probability Grades 1-2. this year I plan to use the textbook less and use Math By All Means to supplement the lesson ideas. ””

22. How RENEW Fosters PCK Math Activities Classroom connections (e.g. Good Questions for Teaching Mathematics) Understanding English Learners (e.g. Language oppression and implications for learning of math)

23. Change in PCK-- Understanding of Student Diversity “Bias is a concept on my second map, but not on my first. I had heard bout statistics being “used” to support an opinion but did not truly understand how the same information could say such different things. In the activity looking at the work force comparing gender and race, though are groups were given the same data, the results were quite different. Looking at sampling size, locations, etc. makes a huge impact on what the statistics on a given topic may say. “

24. Change in PCK-- Understanding of Student Diversity “ One thing that was illustrated a lot was how differently people think about the same thing, how we each approach and solve problems in various ways. Its great to learn different perspectives by working with others. The language and statistics activity was great for emphasizing how much more I need to be conscious of possible language barriers in my classroom as opposed to the students not understanding the material. I’m really glad that you incorporate social issues like race/ethnicity and sexism.”

25. Collecting Data About Efficacy Each year renew beginning teachers fill out a questionnaire that includes:  Six graded response questions about efficacy

26. Efficacy to implement effective teaching practices Manage a class of students engaged in hands-on project work Not adequately prepared Very well prepared Somewhat prepared Fairly Well prepared

27. Efficacy to implement effective teaching practices Help students take responsibility for their own learning Not adequately prepared Very well prepared Somewhat prepared Fairly Well prepared

28. Efficacy to implement effective teaching practices Recognize and respond to student diversity Not adequately prepared Very well prepared Somewhat prepared Fairly Well prepared

29. Efficacy to implement effective teaching practices Use strategies that specifically encourage minorities and females Not adequately prepared Very well prepared Somewhat prepared Fairly Well prepared

30. Efficacy to implement effective teaching practices Lead a class of students using investigative strategies Not adequately prepared Very well prepared Somewhat prepared Fairly Well prepared

31. Efficacy to implement effective teaching practices Encourage student interest in mathematics Not adequately prepared Very well prepared Somewhat prepared Fairly Well prepared

32. Retention 93% of RENEW beginning teachers are still participating in RENEW after 1 or more years in the project.

33. How is RENEW developing PCK in BTs By providing intellectual and emotional support to the Preceptor cadres with whom they work By providing professional development that focuses on leadership to these experienced teachers. The project is collecting data using multiple methods to trace the development of leadership capacity in preceptors

34. RENEW Evaluation on the web http://renew.education.ucsb.edu Research and Evaluation Papers and Presentations, Reports