PP450: Program Evaluation Faculty Sheila Toppin, DPA (abd), MPA

Welcome to the Class Greetings Availability ◦ Office hours: Monday & Thursday 6:00 - 8:00 pm EST ◦ AIM ID: Sheila Toppin ◦ Telephone ◦ Appointments: send an message Faculty & Student Introductions ◦ Personal Triumphs

Course & Syllabus Info Units (10 weeks), Wednesday to Tuesday Discussion Board – 1 to 2 discussions per Unit, Wednesday to Tuesday MST Seminar meets every Monday at 8 PM EST Assignments Announcements: Informational & Reminders Holiday Break: Independence Day, July 4th Last day to request an Incomplete: 8/6/12 Last day of class: Tuesday, 8/14/12

Course & Syllabus Info Assignments: ◦ Unit 3, Evaluation Model ◦ Unit 5, Evaluation Process ◦ Unit 7, Needs Assessment ◦ Unit 9, Final Project Discussion Board (25 pts ea) Seminars (10 pts ea)

Unit 3 Assignment: Evaluation Models Great Start Case Study You are a member of the state commission charged with undertaking the evaluation of the Great Start Program. Write a memo to the State Secretary of Education in which you: Recommend four basic evaluation models that could be used for this task and evaluate their appropriateness. Be sure to recommend both qualitative and quantitative models. For each program evaluation model you recommend using, what key guiding program evaluation principles are particularly noteworthy and applicable? Research the guiding principles of program evaluation as outlined by the American Evaluation Association on its website: Respond to assignment questions.

Unit 5 Assignment: Evaluation Process Great Start Case Study To prepare the commission to conduct a meaningful study, write a report in which you analyze the processes which are involved in the evaluation. What key elements and/or considerations would be important in the process of conducting an evaluation of the Great Start program? Why did you choose the specific processes you did? How would they be applied to a program evaluation of the Great Start Program? Review Posavac & Carey, Program Evaluation: Methods and Case Studies, pages for a basic review of the processes involved in program evaluation. Having completed your planning, state the steps the Commission needs to undertake if you were to actually conduct the study.

Unit 7 Assignment: Needs Assessment Doc on Wheels Case Study Develop a program logic model for the “Doc on Wheels” program. Use the background material given for the “Doc on Wheels” program and design a flow chart to illustrate a program logic model. Include program inputs, activities, outputs and outcomes (initial, intermediate and long-term). Analyze the elements of the flow chart: How does the logic model inform contribute to a needs assessment of the “Doc on Wheels” program? What are the specific inputs, activities and outputs of the program? In what ways can this logic model assist the Hospital? If the Hospital chooses to share it with the City Council, in what ways can it assist the local government? What is the intended casual linkages of the “Doc on Wheels” program as illustrated by your program logic model.

Unit 9 Final Project Drug Court Case Study You are to design an evaluation plan to address the following: Qualitative concerns: program effectiveness, appropriate allocation of resources, necessary implementation improvements Quantitative concerns: impact on public safety, impact on recidivism, impact on employment outcomes. The design of your evaluation plan should have at a minimum these three parts: a clear articulation of the question(s) the evaluation seeks to answer, a section on the methods you propose to get the answers to the questions, a section that clearly outlines what you would propose to do with the answers to the questions. Your evaluation plan should include baseline information about the effectiveness of drug court programs in other localities, graphical data, and prediction of the outcomes of your evaluation and the possible policy changes that would result from your evaluation.

Course & Syllabus Info Con’t. Plan B Seminar: Live (Chat) in classroom Reading assignments are VITAL (eBook Chapters & Web Articles located via Reading Icon or DocSharing tab in classroom.) Assignments: Upload to DropBox in classroom Assignment Feedback: Grading Rubrics located in DocSharing tab in classroom. Late Policy (assignments are accepted up to 3 weeks from the due date.)

Course & Syllabus Info Con’t. Print a hardcopy of syllabus for easy access. Course Resources: eBook, library, & websites. Research: Scholarly work for Discussion Board & written assignments. Netiquette: ◦ Respect the opinion of others ◦ Dialogue honestly, but tactfully ◦ Politically divisive topics require civility Questions???

Kaplan’s Expectations Timeliness in handing in assignments. Achieve academic excellence by thoroughly understanding of major assignments. Academic integrity: ◦ Students should do their own work. Per the catalog: “Kaplan University considers academic honesty to be one of its highest values” (p. 34). ◦ Plagiarism is to be avoided at all costs.

Kaplan’s Expectations What is plagiarism? Briefly, it is the “theft of someone else’s ideas and work” (p. 34). Plagiarism at Kaplan includes: ◦ copying verbatim from another source, whether that source is from the Internet, from another student, or from one’s own previous work. ◦ This includes recycling our own papers from one course to another.

Kaplan Expectations As Diane Hacker writes in Rules for Writers (6 th edition): “Three different acts are considered plagiarism: ◦ (1) failing to cite quotations and borrowed ideas, ◦ (2) failing to enclose borrowed language in quotation marks, and ◦ (3) failing to put summaries and paraphrases in your own words” (p. 480). Questions???

Class Expectations Respect others’ opinion All communications between students and/or the instructor be conducted at the highest degree of professionalism Respond to all Discussion Questions in a substantive manner to include citing sources to support statements Students should aspire to post a minimum of 3 messages to the Discussion Board each week. This includes responding to the initial post and other students’ posting. Students should aim to meet this requirement by Tuesday of each unit week Participate in weekly Seminars to receive credit. An alternative assignment is listed in syllabus under “Seminar Grading Criteria”

Course Objectives This course is intended to enable students… to study the methods and techniques used to assess effectiveness and monitor the performance of programs. to understand theories, research, and practice related to program evaluation. to develop skills that will assist the practitioner in determining the effectiveness of new or existing programs in public organizations.

Course Outcomes By completing this course, you will be able to: Evaluate the basic models and principles of program evaluation Analyze the processes involved in conducting program evaluation Apply research principles and methods to needs assessment and program evaluation Design a program evaluation plan Demonstrate college-level communication through the composition of original materials in Standard American English Demonstrate knowledge of various methods of research design.

Unit 1 To-Do List Introductions Readings (text & web sources) Discussion (2 Topics) Seminar (See the To-Do Checklist PDF that is available in the classroom to keep track each Unit) Questions??

What is Program Evaluation?

Key Concepts Program: a group of related purposive activities that is intended to achieve one or several related objectives. Program evaluation: a systematic process for gathering and interpreting information intended to answer questions about a program. Program effectiveness: the extent to which a programs achieve its intended outcomes. Performance measurement: the process of designing and implementing quantitative and qualitative measures of program results, including outputs and outcomes.

Key Concepts Cost effectiveness: the ratio of program inputs (expressed in monetary units) to program outcomes Cost-benefit analysis: an evaluation of the costs and benefits of a policy, program, or project wherein all the current and future costs and benefits are converted to current dollars Program logic model: a way of representing a program as an open system that categorizes program activities and outlines the intended flow of activities from outputs to outcomes. Logic models are visual representations of programs that show how resources for a program are converted into activities, and hence into intended results

Program Logic Models Logic models… specify relationships among program goals, objectives, activities, outputs, and outcomes. use graphics or schematics to show theoretical connections among program components, goals, and outcomes. show measures that will be used to determine if activities were carried out as planned (output measures) and if the program's objectives have been met (outcome measures).

Program Logic Models They serve as a format for clarifying what the program hopes to achieve; They are an effective way to monitor program activities; They can be used for either performance measurement or evaluation; They help programs stay on track as well as plan for the future; and They are an excellent way to document what a program intends to do and what it is actually doing (BJA 2012).

Program Logic Models Components: Goal (intent) Objectives (expected achievements) Resources (means) Activities (program efforts) Program Measures/Outputs (data) Outcomes (results)

Program Logic Models Common Problems When Developing Logic Models Links among elements (e.g., objectives, activities, outcome measures) of the logic model are unclear or missing. Too much (or too little) information is provided on the logic model. Objectives are confused with activities. Objectives are not measurable. Questions???

Class Activity – Offender Reentry Program Logic Model corrections/reentry-logic-model.pdf 1. What is the goal? 2. What are the objectives? 3. What are the activities? 4. What are the outputs/Process Measures? 5. What are the outcome measures?

Unit 1 Seminar Summary Reviewed Class & Syllabus Expectations Reviewed Course Objectives & Outcomes Defined Program Evaluation & key concepts for Unit 1 Identified the purpose & challenges in creating program Logic Models Applied Activity – Developed an Offender Reentry Logic Model Final Questions??? Dismissal