1. EVAL 6000: Foundations of Evaluation Dr. Chris L. S. Coryn Nick Saxton Fall 2014

2. Agenda Course overview Introductions Questions and discussion Activity 1 & 2

3. Course description This course is designed to provide an overview of the theory, method, and practice of evaluation – Comparative study of theory, research, and practice perspectives – Analysis of core concepts and definitions, rationale and uses, the field’s history and standards, alternative models and approaches, and emerging and enduring issues

4. Course website The website for this course is located at http://www.wmich.edu/evalphd/courses/ eval-6000-foundations-of-evaluation/ From this site you can access – The course syllabus – Required and supplementary readings – Weekly lecture notes – Other materials related to the course

5. Required textbooks

6. Learning objectives 1.A deep understanding of a wide array of evaluation theory and practice perspectives 2.An in-depth understanding of the origins and history of evaluation as well as its evolution toward an independent discipline 3.A clear understanding of key evaluation concepts/vocabulary/terminology 4.A clear understanding of the nature and purpose of evaluation, and the distinctions between evaluation, basic and applied research, and related terms such as assessment and diagnosis

7. Learning objectives 5.An ability to describe, distinguish among, and critically evaluate the usefulness and validity of selected models and approaches to evaluation, and to identify the conditions under which each should be used 6.A firm grasp of the fundamental logic and methodology of evaluation 7.A basic understanding of how to integrate traditional methodologies with evaluation- specific methodologies 8.An evaluative and critical thinking mindset, in general

8. By the end of this course you should have a clear understanding of evaluation’s… history, standards, theories, methods, & practices

9. Secondary learning objectives 1.Conveying constructive criticism in a professional, balanced, and tactful manner 2.Facilitating discussion to engage others in dialogue about evaluation theory, method, and practice 3.Writing clearly and concisely for both academic and non-academic audiences 4.Giving high quality, professional oral presentations for both academic and non-academic audiences

10. Course components Attendance & class participation 10% Critical readings papers 30% Application paper 20% Thought paper & presentation 20% Final examination 20%

11. A note on course assignments All course assignments are due by 5:00 PM on the date indicated in the course schedule All assignments should be e-mailed to the instructor and teaching assistant with “EVAL 6000” in the subject line NO late assignments will be accepted, will not be graded, and will be assigned a grade of “0” (“F”)

12. Schedule of topics Fundamentals of evaluation (two parts) Evaluation approaches & models (three parts) Evaluation tasks, procedures, & tools (three parts) Metaevaluation & institutionalizing evaluation

13. Course structure If necessary, a question-and-answer session for prior week’s material Lecture – At the end of each lecture is a list of entries from the Encyclopedia of Evaluation that you are expected to study (many will appear in the final examination) Discussion Activity

14. Who are you? Why are you here? What do you expect to learn? What prior experiences do you have with evaluation? Research? Design? Measurement? Statistics (or analysis in general)?

15. Who are we?

16. An abbreviated life history… that has significantly influenced my world view and evaluation practice

17. “…bad is bad and good is good and it is the job of evaluators to decide which is which” — Scriven Major influence on my thinking about evaluation

18. Other influences Davidson Cook Shadish Patton Cronbach Stufflebeam Chelimsky Hattie

19. Most profound influence on my evaluation practice

20. Who are we?

21. An abbreviated life history… that has significantly influenced my world view and evaluation practice

22. Major Influence on My Thinking About Evaluation “A system of morality […] based on relative emotional values is […] a thoroughly vulgar conception, which has nothing sound in it and nothing true” — Socrates

23. Activity 1 Draw the first image that comes to mind when you hear the word ‘evaluation’ In small groups share your images and identify similar and dissimilar themes Share your group’s thematic analysis

24. Activity 2 Carefully review the ‘definitions’ of evaluation enumerated by major evaluation theorists on the following slides In small groups, discuss what you see as potential factors influencing these theorists’ definitions of evaluation and how their definitions might influence evaluation practice Share your group’s analysis

25. “…the act or process of determining the merit, worth, or significance of something or the product of that process” — Scriven

26. “…the use of evaluation concepts and techniques to foster self- determination” — Fetterman

27. “…the use of social science research procedures to systematically investigate the effectiveness of social intervention programs” — Rossi, Lipsey, & Freeman

28. “…[to] describe and assess what was intended (goals and objectives), what happened that was unintended, what was actually implemented, and what outcomes and results were achieved” — Patton

29. “…[to explain] how and why programs work, for whom, and under what conditions” — Chen

30. Encyclopedia entries Assessment Accountability Auditing Campbell, Donald T. Cook, Thomas D. Criteria Evaluand Evaluation Evaluation Theory External Evaluation Formative Evaluation History of Evaluation Independence Logic of Evaluation Objectivity Scriven, Michael Shadish, William R. Standards Summative Evaluation Value-free Inquiry Value Judgment Values

31. “…just because you have a library card doesn’t make you Yoda” — Pitt