1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2011

2. Agenda American Evaluation Association conference reflections and experiences Stage Three theories – General characteristics – Lee Cronbach Questions and discussion Encyclopedia of Evaluation entries

3. General Characteristics of Stage Three Theories Synthesis of Stage One and Stage Two theories Circumstantial, tailored, situational approach to evaluation Theories that are contingent – Denial that there exist only one right way to do evaluation

4. Stage Three Theory of Social Programming Programs are politically affected Programs change gradually Improving existing programs offers the best chance to contribute to short-term social change Abrupt and novel changes do sometimes occur

5. Stage Three Theory of Knowledge Construction Recognized that no single paradigm has sufficient empirical or theoretical support to dominate Evaluation should embrace multiple epistemologies, methods, and priorities for the types of knowledge that are important Dependent upon circumstances

6. Stage Three Theory of Valuing Reliance on stakeholder values complemented by independent sources of values Use of multiple sources of relevant values so that important values are not overlooked Separate summative conclusions by each criterion rather than over all criteria

7. Stage Three Theory of Knowledge Use Emphasis on enlightenment as well as instrumental use Enlightenment is more likely to be the focus of those who are not under pressure to respond to requests for specific information Instrumental use is more likely to be the focus of those who are placed to facilitate such uses

8. Stage Three Theory of Evaluation Practice Specified contingencies under which different evaluation practices should be used and make sense Often centered around the various stages of program lifecycles Contingencies

9. “It is self-defeating to aspire to an evaluative conclusion as precise and…as beyond dispute…as from the laboratory” — Lee J. Cronbach

10. Biographical Sketch Born in 1916 in Fresno, California Died in 2001 in Palo Alto, California Ph.D. in Education Psychology, University of Chicago Scored 200 on Stanford Binet at age 5 (by some accounts at age 4) Completed high school at age 14 Graduated from Fresno State College at age 18 with degrees in mathematics and chemistry

11. Cronbach’s View of Evaluation Mainly a critic of prior intellectual traditions and offered recommendations for improvement Recognition of the lack of strong substantive theories and powerful methods of inquiry Greater attention to external validity and/or generalizability

12. Cronbach’s Influence Far reaching Social science (social inquiry) methodology (at least equal to, and in some respects greater than, Campbell) – Measurement theory – Individual differences – Program evaluation – Epistemology

13. Cronbach’s Major Contributions Social inquiry Political accommodation Generalizability theory Generalization through explanation Aptitude-treatment interactions Bandwidth and fidelity utos, UTOS, *UTOS, sub-utos

14. Cronbach’s Theory of Social Programming Rejection of naïve normative expectations (e.g., rational decision making) The core of social programing and policy making is among conflicting stakeholders Assumes four stages: (1) breadboard; (2) superrealization; (3) prototype; and (4) ongoing program

15. Cronbach’s Theory of Knowledge Construction Knowledge is predicated on assumptions regarding determinants (i.e., sources of variance) of performance (and, therefore, knowledge claims) No clear epistemological position, though multiple methods are clearly advocated

16. Cronbach’s Theory of Valuing Prescriptive valuing plays only a small role Descriptive valuing is central Favored a democratic control that assigned open debate as central to value claims

17. Cronbach’s Theory of Knowledge Use Enlightenment rather than instrumental use is preferable Use occurs not only from evaluation findings, but also through prior knowledge and from basic research about substantive theories It is unrealistic to expect that knowledge will dictate decisions Long-term view to influence future programs

18. Cronbach’s Theory of Evaluation Practice Constructed around assumptions about types of questions worth asking and the degree of certainty worth attaining Eclectic in choice of methods, avoiding adherence to any particular methods Educator rather judge, philosopher- king, or servant to a particular stakeholder group

19. Encyclopedia Entries Cousins, J. Bradley Logic Model Patton, Michael Quinn Participatory Evaluation Program Logic Program Theory Rossi, Peter H. Utilization-Focused Evaluation