EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Nick Saxton Fall 2014

Agenda Course overview Introductions Questions and discussion Activity 1 & 2

Course description This course is designed to provide an overview of the theory, method, and practice of evaluation – Comparative study of theory, research, and practice perspectives – Analysis of core concepts and definitions, rationale and uses, the field’s history and standards, alternative models and approaches, and emerging and enduring issues

Course website The website for this course is located at eval-6000-foundations-of-evaluation/ From this site you can access – The course syllabus – Required and supplementary readings – Weekly lecture notes – Other materials related to the course

Required textbooks

Learning objectives 1.A deep understanding of a wide array of evaluation theory and practice perspectives 2.An in-depth understanding of the origins and history of evaluation as well as its evolution toward an independent discipline 3.A clear understanding of key evaluation concepts/vocabulary/terminology 4.A clear understanding of the nature and purpose of evaluation, and the distinctions between evaluation, basic and applied research, and related terms such as assessment and diagnosis

Learning objectives 5.An ability to describe, distinguish among, and critically evaluate the usefulness and validity of selected models and approaches to evaluation, and to identify the conditions under which each should be used 6.A firm grasp of the fundamental logic and methodology of evaluation 7.A basic understanding of how to integrate traditional methodologies with evaluation- specific methodologies 8.An evaluative and critical thinking mindset, in general

By the end of this course you should have a clear understanding of evaluation’s… history, standards, theories, methods, & practices

Secondary learning objectives 1.Conveying constructive criticism in a professional, balanced, and tactful manner 2.Facilitating discussion to engage others in dialogue about evaluation theory, method, and practice 3.Writing clearly and concisely for both academic and non-academic audiences 4.Giving high quality, professional oral presentations for both academic and non-academic audiences

Course components Attendance & class participation 10% Critical readings papers 30% Application paper 20% Thought paper & presentation 20% Final examination 20%

A note on course assignments All course assignments are due by 5:00 PM on the date indicated in the course schedule All assignments should be ed to the instructor and teaching assistant with “EVAL 6000” in the subject line NO late assignments will be accepted, will not be graded, and will be assigned a grade of “0” (“F”)

Schedule of topics Fundamentals of evaluation (two parts) Evaluation approaches & models (three parts) Evaluation tasks, procedures, & tools (three parts) Metaevaluation & institutionalizing evaluation

Course structure If necessary, a question-and-answer session for prior week’s material Lecture – At the end of each lecture is a list of entries from the Encyclopedia of Evaluation that you are expected to study (many will appear in the final examination) Discussion Activity

Who are you? Why are you here? What do you expect to learn? What prior experiences do you have with evaluation? Research? Design? Measurement? Statistics (or analysis in general)?

Who are we?

An abbreviated life history… that has significantly influenced my world view and evaluation practice

“…bad is bad and good is good and it is the job of evaluators to decide which is which” — Scriven Major influence on my thinking about evaluation

Other influences Davidson Cook Shadish Patton Cronbach Stufflebeam Chelimsky Hattie

Most profound influence on my evaluation practice

Who are we?

An abbreviated life history… that has significantly influenced my world view and evaluation practice

Major Influence on My Thinking About Evaluation “A system of morality […] based on relative emotional values is […] a thoroughly vulgar conception, which has nothing sound in it and nothing true” — Socrates

Activity 1 Draw the first image that comes to mind when you hear the word ‘evaluation’ In small groups share your images and identify similar and dissimilar themes Share your group’s thematic analysis

Activity 2 Carefully review the ‘definitions’ of evaluation enumerated by major evaluation theorists on the following slides In small groups, discuss what you see as potential factors influencing these theorists’ definitions of evaluation and how their definitions might influence evaluation practice Share your group’s analysis

“…the act or process of determining the merit, worth, or significance of something or the product of that process” — Scriven

“…the use of evaluation concepts and techniques to foster self- determination” — Fetterman

“…the use of social science research procedures to systematically investigate the effectiveness of social intervention programs” — Rossi, Lipsey, & Freeman

“…[to] describe and assess what was intended (goals and objectives), what happened that was unintended, what was actually implemented, and what outcomes and results were achieved” — Patton

“…[to explain] how and why programs work, for whom, and under what conditions” — Chen

Encyclopedia entries Assessment Accountability Auditing Campbell, Donald T. Cook, Thomas D. Criteria Evaluand Evaluation Evaluation Theory External Evaluation Formative Evaluation History of Evaluation Independence Logic of Evaluation Objectivity Scriven, Michael Shadish, William R. Standards Summative Evaluation Value-free Inquiry Value Judgment Values

“…just because you have a library card doesn’t make you Yoda” — Pitt