1. CaMSP—A Retrospective
CaMSP Network Meeting
February 27 and 28, 2012, Sacramento, CA
Mikala L. Rahn, PhD, Public Works

2. Session Overview CaMSP--then and now
Evolving issues and policy priorities in mathematics and science education
Implementation and outcomes--what has worked and is working well? What is a work in progress?
Discussion time--looking back and positioning the state and partnerships for the future

3. Statewide Evaluation Research Questions
How have the Partnerships ensured that all students have access to, are prepared for, and are encouraged to participate and succeed in challenging and advanced mathematics and science courses?
How have the Partnerships enhanced the quality of the mathematics and science teacher workforce?
What evidence-based outcomes from the Partnerships contribute to our understanding of how students effectively learn mathematics and science?

4. Evaluation Needs in California
Statewide Evaluation
Qualitative assessment of implementation measured through site visits, observation of professional development and surveys of teachers and partners.
Quantitative measurement of student outcomes through a matched comparison study of California Standards Test results in mathematics and science.
Local Evaluation
Measures teacher content knowledge.
Designed to answer local questions about professional development models and student impact.
Federal Reporting and Annual Performance Report 4

5. Key Features of CaMSP Features of the programs reviewed based on
the legislation:
Partnership driven
Teacher quality
Challenging courses and curricula
Evidence-based design and outcomes
Institutional change and sustainability

6. Where we started…pre-NCLB Snapshot 2000-02
Historical context of NCLB
Uneven quality of state standards and assessments
Nothing comparable from state to state
Federal vs state vs local control
Centralized states model for NCLB legislation--not a good fit in all states and no consensus on “national curriculum”
Economic imperative for a focus on mathematics & science; poor US performance on international assessments
Professional development under previous federal efforts such as Eisenhower and others lacked measurement and rigorous research

7. Where we started…California Context Snapshot 2000-02
California developing a rigorous standards-based accountability system, curriculum frameworks, and textbook alignment
STAR based on a “growth” model for improvement and the Academic Performance Index released to the public
Gradual movement from norm-referenced tests to California Standards Tests (CST) aligned to SBE standards in core content areas
Forward thinking professional development through Subject Matter Projects, Eisenhower grants, and efforts to reform teacher certification and preparation, but not always aligned to standards and accountability 7

8. NCLB arrives in California
Introduces new accountability provisions based on absolute targets for performance on reading and mathematics assessments
Measures all students and specified subgroups
Absolute targets quickly overshadowed STAR growth model for improvement
Like most states, California moved to a more centralized, standardized direction and narrowing of the curriculum--SAIT, DAIT, textbook training, etc. 8

9. CaMSP The Bright Spot in NCLB for California
Title II included the Improving Teacher Quality Act (ITQ) grant programs
Scientifically-based professional development--we call it research first, PD second
One of two competitive grant programs in California (one under CDE funded through LEAs, the other under CPEC funded through IHEs now housed in CDE)
Funded professional development for high need LEAs
Became an opportunity for professional development aligned to local needs but with parameters, goals, and targets 9

10. CaMSP in California
To date, nine separate cohorts of partnerships have been authorized by the California Department of Education involving 127 partnerships, 485 districts and 13,558 teachers.
Two new cohorts include three Cohort 9 partnerships and four Demonstration sites.
Partnerships focus on mathematics or science in grades 3 through 8 or Algebra I.
PD models currently incorporate significant hours of training for three funding cycles--less attrition as implementation requirements tightened in California (60 hours of Intensive hours and 24 hours of Classroom follow-up support).
Same cohort of teachers participate over three years in each local partnership. We have data to measure based on consistent dosage of professional development and rules for implementation. 10

11. What are we learning from Mathematics and Science Partnerships in California?
Strengths
A more structured approach for IHEs is building closer relationships to LEA partners, resulting in better alignment of training to state adopted curricula, local benchmark testing systems and district policies and initiatives.
LEA’s are getting used to their role in building partnerships with individuals from IHE’s and building capacity for managing high quality PD, especially in medium size districts.
Structure and personnel in the classroom follow-up piece is essential to successful implementation and has improved; adaptation to teacher needs is important but fidelity to follow-up model supports institutionalization.
Areas to Focus
Difficulty in maintaining involvement and input from site administrators in planning and PD continues as an area of concern in some partnerships.
Visibility of and continued focus on local evaluation is beginning to lead to higher quality instrumentation and documentation but we must continue to build and share what we are learning across the state.
Rural multi-district partnerships value the professional development training but are particularly challenged by three year teacher participation rules in California.

12. Statewide Outcome Study Results
Study involved large numbers of teachers and students. In : 284,538 treatment and comparison students/1,581 treatment teachers; In and : 165,209 treatment and comparison students/1,594 treatment teachers
A positive and significant—though small—effect on overall mathematics CST scores in 2008 and Also showed a significant, and more dramatic, effect on Algebra I test scores in 2008 and However, not in 2010.
Science partnerships appeared to have a slight, though positive, impact on science learning in California in 2008, 2009, and This is mostly attributable to 8th grade performance. The science comparison group in California is limited to the 5th and 8th grades where the science CST is administered.
A more in-depth analysis in the and study of partnerships by dosage/funding cycles completed reveal the diversity of mathematics and science achievement among partnerships. Some partnerships showed marked improvement, while others were little different from the comparison groups. 12

13. Where are we going and how can CaMSP inform us?
We have learned a lot about lesson study and coaching and models for professional development that span the whole school year--a better connection to best practices in the research
District leadership has really stepped up--recruiting teachers and teacher support hard to do but has helped embed PD
Continuous reinforcement of ‘research first, PD second’ might mean we are the only state with any outcome data; centralized teacher database
Local evaluator capacity built over time
Science received attention, training and classroom time
Including 3rd grade helped reach more teachers and students

14. What else does California need to learn to move forward?
How can non-participating LEAs adopt best practices in PD (geographic coverage limited)?
Have we learned anything from teacher content assessment? What do we need to measure this aspect of PD?
What can be communicated to others that has been learned to help students prepare for Algebra I?
What lessons can be learned about partnering with IHEs--new relationships formed but lack of institutional change?
Are trained teachers enough to sustain the efforts? How can we build in coaching and collaboration in other ways? 14

15. Discussion & Wrap-up
What has been most successful in moving our partnership forward?
How do these efforts contribute to a competitive position for California in mathematics and science education?
What is your sixty second elevator speech to promote and expand the work of your partnership? 15