Leigh M. Tolley Syracuse University Paper presented at the American Evaluation Association Annual Meeting, Anaheim, California, November 2011

Overview Defining formative assessment and context Review of factors presented in the literature that are described as impacting formative assessment Implications for formative evaluation, program evaluation, and future research

Formative Assessment Formative assessment involves teachers using evaluative skills and strategies to guide and improve student learning through timely feedback, and is akin to program evaluators offering recommendations for improvement. (Ayala & Brandon, 2008)

Context Context (of an evaluation) : The combination of the factors accompanying the evaluation that may have influenced its processes [emphasis added] or findings. These factors include the geographic location; its timing; the political, social, cultural, and economic influences in the region at that time; the other relevant professional activities that were in progress; and/or any other pertinent factors. Yarbrough, Shulha, Hopson, Caruthers & the Joint Committee on Standards for Educational Evaluation (2011), p. 285.

Existing Versus Emerging Existing Context In place before evaluation occurs Fixed, more expected Emerging Context Changes as evaluation progresses Evolves slowly or shifts rapidly

National Contexts Described Black & Wiliam (2005): England, France, Germany, and the United States Crossouard (2011): Scotland Frey & Fisher (2009): United States Nelson & Eddy (2008): United States Yin et al. (2008): United States

Levels of Context Broad contextual factors District- and Administrative-level factors School-level factors

Levels of Context (continued) Teacher-level factors In the classroom Outside of the classroom Student-level factors

Importance of Contextual Factors Rich school environment Existing and emerging contexts are important to consider Preference for focusing more on existing or emerging? Why?

Next Steps Implications for program evaluation Evaluation capacity building Consistent implementation Documentation Implications for formative evaluation Process improvement Involving teachers in assessment process teaches evaluative skills

Next Steps (continued) Future research suggestions More exploration into impact of context on formative assessment Development of checklist of contextual considerations Evaluation capacity building Formative assessment’s role in the evaluation planning process How does formative assessment impact teachers’ professional development and their instructional decisions?

References Ayala, C. C., & Brandon, P. R. (2008). Building evaluation recommendations for improvement: Insights from student formative assessments. In N. L. Smith & P. R. Brandon (Eds.), Fundamental issues in evaluation (pp ). New York, NY: The Guilford Press. Black, P., & Wiliam, D. (1998a). Assessment and classroom learning. Assessment in Education, 5 (1), Black, P., & Wiliam, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80 (2), Black, P., & Wiliam, D. (2005). Lessons from around the world: How policies, politics and cultures constrain and afford assessment practices. Curriculum Journal, 16 (2), Crossouard, B. (2011). Using formative assessment to support complex learning in conditions of social adversity. Assessment in Education: Principles, Policy & Practice, 18 (1), Frey, N., & Fisher, D. (2009). Using common formative assessments as a source of professional development in an urban American elementary school. Teaching and Teacher Education, 25 (5),

References (continued) Nelson, M., & Eddy, R. M. (2008). Evaluative thinking and action in the classroom. New Directions for Evaluation, 117, Tolley, L. M. (2011, May). When the stars align: The fundamental issue of context and its role in exemplary evaluations. Paper presented at the Eastern Evaluation Research Society (EERS) 34 th Annual Conference, Galloway, New Jersey. U. S. Department of Education. No Child Left Behind legislation and policies. Retrieved from Wiliam, D. (2006). Formative assessment: Getting the focus right. Educational Assessment, 11 (3&4), Yarbrough, D. B., Shulha, L. M., Hopson, R. K., Caruthers, F. A., & the Joint Committee on Standards for Educational Evaluation (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Los Angeles, CA: Sage. Yin, Y., Shavelson, R. J., Ayala, C. C., Ruiz-Primo, M. A., Brandon, P. R., Furtak, E. M., Tomita, M. K., & Young, D. B. (2008). On the impact of formative assessment on student motivation, achievement, and conceptual change. Applied Measurement in Education, 21 (4),

Thank You! Feedback, thoughts, and further suggestions for study are much appreciated. Contact: