Ali Alkhalaf ALM Zarudeen Michelle Corby Yu Kyoung Park WF ED 597 March 23, 2012

Developmental Evaluation

Definition of developmental evaluation

Nonlinearity: Sensitivity to initial conditions in which small actions can stimulate large reactions, thus the butterfly wings metaphor (Gleick, 1987). Emergence : Innovators can’t determine in advance what will happen, so evaluators can’t determine in advance what to measure. Adaptation : Interacting elements and agents respond and adapt to each other, and to their environment. Uncertainty : Under conditions of complexity, processes and outcomes are unpredictable, uncontrollable, and unknowable in advance. Coevolution : As interacting and adaptive agents self-organize, ongoing connections emerge that become evolutionary as the agents evolve together (coevolve) within and as part of the whole system, over time (Philip, 1999).

Summative EvaluationFormative Evaluation  Overall judgment of merit or worth about a stable and fixed program intervention based on explicit criteria like effectiveness, efficiency, relevance, and sustainability.  Improve the program. Fine- tune the model, clarifying key elements and linkages from inputs to activities and processes to outputs, outcomes, and impacts.  Determine the future of the program and model, including especially whether it should be disseminated as an exemplar and taken to scale.  Determine efficacy and effectiveness at a pilot level to establish readiness for summative evaluation. Source: Scriven (1991). Beyond formative and summative evaluation.

Summative EvaluationFormative Evaluation  Well-defined intervention model supported by an explicit and testable theory of change.  Clear, specific, measurable, attainable, and time-bound outcomes.  Draft program model to be fine-tuned.  Establish criteria for quality implementation to guide and focus process improvement.  Processes and instruments for getting participant feedback.  Is this an especially effective practice that should be funded and disseminated as an model program, a best practice?  Is the program model ready for summative evaluation? Source: Scriven (1991). Beyond formative and summative evaluation.

Support adaptation in complex, uncertain, and dynamic conditions. Identify emergent processes and outcomes that accompany innovation, and support making sense of their implications. Support ongoing development and adaptation to changing conditions. Determine when and if an innovation is ready for formative evaluation as a pilot intervention. Help social innovators explore possibilities for addressing major problems and needs, and identify innovative approaches and solutions. Source: Patton (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use.

TraditionalComplexity-sensitive  Formative-summative distinction dominant: formative improves; summative tests, proves, and validates program models; accountability.  Finding out if a program model works; focus on effectiveness, efficiency, impact, and scalability.  Outcome-driven; systems viewed as context.  Supports development of innovations and adaptation of interventions in dynamic environments.  Exploring possibilities; generating ideas and trying them out; preformal model, so preformative; nonsummative in that ongoing innovation and development is expected, never arriving at a fixed intervention.  System-change-driven; specific outcomes emergent, dynamic. Source: Patton (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use.

7 Scriven, M. (1991). Beyond formative and summative evaluation. In M. W. McLaughlin & D. C. Phillips (Eds.) Evaluation and Education: At Quarter Century, pp Chicago, IL: The University of Chicago Press. Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: NY: The Guilford Press. Gleick, J. (1987). Chaos: Making a new science. New York: Penguin Books. Philip, A. (1999). Complexity theory and organization science. Organization Science, 10(3),

Scene 1: CEO Scene 2: