Arkansas MSP Grants: Statewide Evaluation Plan Judy Trowell Arkansas Department of Education Charles Stegman Calli Holaway-Johnson University of Arkansas National Office for Research on Measurement and Evaluation Systems (NORMES) Laura Kent University of Arkansas Department of Curriculum & Instruction

Statewide Evaluation Team  Charles Stegman, Ph.D.  Professor of educational statistics and research design  Laura Kent, Ph.D.  Associate professor of mathematics education  Calli Holaway-Johnson, Ph.D.  Research assistant with background in math education

NORMES  Housed at the University of Arkansas within the College of Education and Health Professions  Staff  University of Arkansas Educational Statistics faculty  Education and technology professionals  Graduate students from education, policy, statistics, and computer engineering  Funded through federal grants, state contracts, and foundations

Evaluation Plan  Goal of evaluation is to aggregate results across all MSP grants in the state  Use qualitative and quantitative techniques to determine:  quality of grant implementation  enactment of professional development in the classroom  impact of content-based training on teacher content knowledge  Examine state assessment data to determine impact on student achievement

Evaluation Plan, Current Grants  Arkansas currently has 22 MSP grants  Impact of current MSP grants will be evaluated on:  alignment of professional development with stated program goals  results of program assessments or surveys  quality of implementation as described in yearly reports

Evaluation Plan, Future Grants  8 MSP grants to begin Summer 2007  Impact of future MSP grants will be evaluated using:  teacher-level data of all participants  student-level data for all participating teachers  program assessment data or survey results  direct observation studies

Role of NORMES  Developed standardized evaluation component for RFP (January 2007)  Conduct focus meetings with project directors and evaluators  Develop standardized evaluator and program director questionnaire  Analyze program data to determine the overall effectiveness of programs

Role of NORMES  Assist new grantees and their evaluators in developing appropriate evaluation techniques specific to their program  Assist new grantees and their evaluators in developing or identifying a measure of teacher content knowledge  Disseminate results of evaluation  Focus meetings with project directors  NORMES website using GIS technology

Contact Information Judy Trowell Charles Stegman Laura Kent Calli Holaway-Johnson