Download presentation
Presentation is loading. Please wait.
Published byRudy Longyear Modified over 9 years ago
1
EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Kristin A. Hobson Fall 2012
2
Agenda Stage One theories – General characteristics – Michael Scriven Questions and discussion Encyclopedia of Evaluation entries
3
General Characteristics of Stage One Theories Stage One theories can be characterized as a period of social problem solving Emphasis was on scientific rigor and valid causal knowledge of the effects of social programs Reflects the tenor of the 1960s, a period of rapid development in the social sciences
4
Stage One Theory of Social Programming Programs largely viewed as immediately implementable solutions to social problems Rested on assumptions that A.Implementation and evaluation would be unambiguous B.“Successes” would be adopted C.A and B combined, would lead to ameliorating social problems Concerned primarily with innovations/pilots/demonstrations
5
Stage One Theory of Knowledge Construction Priority was given to “truth” The dominant view was that an external reality probably exists, though it may never be perfectly known Valid knowledge could be constructed (more or less) Emphasis on sources of bias and the best methods for reducing them
6
Stage One Theory of Valuing Theorists at this stage agree that valuable social programs solve important social problems They disagree about how value claims are made and whose values should be given priority – Prescriptive versus descriptive valuing
7
Stage One Theory of Knowledge Use Dominant assumption was “naïve instrumentalism” Effective programs would be maintained or scaled-up and ineffective ones would be eliminated (i.e., a rationalist view) Use would occur almost naturally – Evaluation findings, being scientifically rigorous, would inform decision making
8
Stage One Theory of Evaluation Practice Emphasis on assessing a program’s effectiveness at solving important social problems Primary task is summative claims or judgments – Methods of arriving at summative statements were given priority Distance is crucial to scientific rigor and integrity
9
“Bad is bad and good is good and it is the job of evaluators to decide which is which” — Michael Scriven
10
Biographical Sketch Born in 1928, in Bealieu, Hampshire, England Ph.D. in Philosophy, Oxford University M.A. in Mathematics and Philosophy, Melbourne University Former President of AERA and AEA Author or more than 300 publications Recipient of numerous awards and prizes
11
Scriven’s View of Evaluation Scriven argues that a singular statement of value is the sine qua non of evaluation and is the primary task of evaluators Evaluation is an autonomous discipline, and a vital component of all other disciplines
12
Scriven’s Influence Scriven is widely viewed as the first major evaluation theorist Whether one agrees or disagrees with his views, Scriven has directly and indirectly influenced nearly every other theorist and theory
13
“Not to know Scriven is to be illiterate in evaluation” — Thomas D. Cook
14
Scriven’s Major Contributions Definition of evaluation Valuing theory Formative and summative distinction Goal-free evaluation Causal logic and cause-probing methods Logic of evaluation Theory-free evaluation Metaevaluation Key Evaluation Checklist
15
Goal-Free Evaluation Premised on examining what effects a program is actually having rather than intended effects Aimed at bias reduction through avoiding knowledge of program goals Relies on needs assessment to determine the extent to which a program meet needs
16
Causal Logic and Reasoning Modus operandi method (MOM) and general elimination method (GEM) for causal analysis Causal claims ‘beyond a reasonable doubt’ should be the standard within and outside of science, as opposed to those supported by ‘the balance of the evidence’ Overdetemination problem
17
Theory-Free Evaluation The role of evaluators is to determine only whether programs work, not to explain how they work To attempt explanations is beyond the capability of most evaluators Requires the correct (not just the believed) logic or theory of the program
18
Metaevaluation An evaluation of an evaluation Internal and external quality control Identify strengths, weaknesses, and uses Appropriate criteria – Validity – Credibility – Utility – Cost-effectiveness – Ethicality
19
Key Evaluation Checklist Organized into four parts with 15 checkpoints – Part A: Preliminaries – Part B: Foundations – Part C: Subevaluations – Part D: Conclusions & Implications
20
Scriven’s Theory of Social Programming Social programs exist to meet important societal needs – Consumer needs should be given the same primacy in an evaluation Good programs should be expanded and poor programs should be abandoned
21
“If I had asked people what they wanted, they would have said faster horses” — Henry Ford
22
Scriven’s Theory of Knowledge Construction Realist philosophy and ideology Valid knowledge is possible, but nearly always incomplete Practical inference rather than the development of scientific theories about social programs Socio-organizational controls to reduce bias in knowledge claims
23
Scriven’s Theory of Valuing Prescriptive theory premised on some values having higher priority than others Valuing is no less scientific in its validity than “value-free” science Relative valuing, rather than absolute valuing, is preferable
24
Scriven’s Theory of Knowledge Use Use is “self-evident” “Naïve instrumentalism” Primarily concerned with summative statements emphasizing short-term instrumental use
25
“Evaluation is what it is, the determination of merit and worth, and what it is used for is another matter” — Michael Scriven
26
Scriven’s Theory of Evaluation Practice Methodologically pluralistic Audience of evaluation is predominately “consumers” Evaluations should serve the “public good” Emphasizes bias minimization through socio-organizational controls
27
Encyclopedia Entries Bias Causation Checklists Chelimsky, Eleanor Conflict of Interest Countenance Model of Evaluation Critical Theory Evaluation Effectiveness Efficiency Empiricism Independence Evaluability Assessment Evaluation Use Fournier, Deborah Positivism Relativism Responsive evaluation Stake, Robert Thick Description Utilization of Evaluation Weiss, Carol Wholey, Joseph
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.