EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn & Carl D. Westine October 7, 2010

Stage 2 Theories Stage 2 theories can be characterized as a period of “enlightenment” in the evolution of evaluation theory –Theorists argued that rigorous causal knowledge was not always useful for improving programs –Theorists emphasized different users of evaluation and their roles in “valuing” social programs –Theorists focused on the complex nature of social programs –Despite a number of commonalities, the three major theorists of this period differed in more and less significant ways

Stage 2 Theories Criterion 1: Theory of Social Programming –Stage 1 theorists operated on “naive” assumptions (i.e., that radical changes would be made as a result of evaluation –Stage 2 theories and theorists recognized the “political” aspects of evaluation and directed efforts toward “incremental change” (Weiss, in her later work, reverted to more radical ideologies) –Related to theories of use, Stage 2 theories of social programming sought to determine “who” could make changes The resulting assumptions were complex and produced no definitive solutions

Stage 2 Theories Criterion 2: Theory of Use –In Stage 2, theories of use became the dominant concern of theories and theorists Who would use? For what? –These concerns were reflected in a view of wider types of uses than those anticipated by Scriven and Campbell –They also reflected concerted efforts to identify users Ultimately, this became Patton’s ideological perspective (i.e., intended users for specific uses) Even so, users and uses differed between the major theorists of the time

Stage 2 Theories Criterion 3: Theory of Knowledge Construction –Priority shifted from “truth” to both “truthful and useful” –During this period theories and theorists took a “pragmatic” stance Willingness to sacrifice valid knowledge for useful knowledge –This pragmatic position was reflected in various ways, but predominately as “methodological pluralism” (with the exception of Stake) –This position also included a more thorough understanding of social programs “in context” Central focus was no longer effects, but included a variety of more “discovery” related concerns

Stage 2 Theories Criterion 4: Theory of Valuing –Theorists at this stage predominately advocated descriptive values –This view reflected the notion of “pluralistic” sources of values as valuing takes place in a political context of social programs with multiple stakeholders and constituents –Here, the three dominant theorists of the period prefer to let intended users determine value Valuing is not a task for evaluators given the complexities involved in valuing as well as the complexities involved in decisions about social programs

Stage 2 Theories Criterion 5: Theory of Practice –Although methodological pluralism was prominant during this period, Wholey and Stake assigned central methods to practice –Weiss never claimed that particular methods were to be preferred over other Her choice of method was one related to types of knowledge to be generated, intended users of that knowledge, and the questions to be answered through evaluation In her later work, she (like Cronbach) became more concerned with issues about generalization and explaining how programs work

Carol Weiss Weiss gave priority to state and federal decision makers in her thoery of evaluation Originally premised on “truth” (similar to Stage 1 theories) she later developed a contingency-based theory –Who? –For what purpose? Later in her career she became more concerned with “enlightenment”

Joseph Wholey Wholey was predominately concerned with management issues and decisions reflecting their concenrs Like Weiss, Wholey began under the auspises of “truth” seeking His later work emphasized “instrumental use” of evaluation—immediate decisions and actions Overall, his central concern was producing actionaable information aimed at those who are in positions to take decisions Like Weiss, his choice of method was quantitative in orientation

Robert Stake Stake, unlike prior (and later) theorists, developed his theory on the premise of “discovery” By discovery, Stake implied that the only realities (i.e., truth) are those that are constructed He also claimed that valuing is socially constructed –The value of a program is best determined by those who experience it –Emphasis was on local stakeholders values –Premised on the assumption that most decision making takes place locally, and that values of local constituents often compete with others

Contrasting Stage 1 and 2 Work with the person sitting to your right, select one of the criteria for evaluation theory, and critically compare and contrast (using example if possible) Stage 1 and Stage 2 theorists’ position on the selected criterion 1.Theories of Social Programming 2.Theories of Use 3.Theories of Knowledge Construction 4.Theories of Valuing 5.Theories of Practice

Further Discussion of the Readings What specific questions were raised as you read the assigned readings for this week (or past weeks)? Is there something specific that you didn’t understand? What problems remain unresolved regarding Stage 2 theories? What other issues emerged from the readings that need further exploration?

Encyclopedia Entries for this Week Conceptual use Context Countenance model of evaluation Cronbach, Lee J. Developmental evaluation Evaluability assessment Evaluation use Goal Implementation Intended users Intended uses Objectives Performance indicator Process use Responsive evaluation Rossi, Peter H. Setting Stake, Robert E. Thick description Utilization of evaluation Weiss, Carol H. Wholey, Joseph S.