An Overview of Evaluation Research Unit 2 Yvonne Bustamante, MS, FDC

Evaluation Research 1. Evaluation research refers to: a. Clinical case discussions between professionals when the client’s progress on target behavior is the focus of the dialogue. b. A systematic collection and analysis of data regarding a specific program or intervention. c. Both of the above. d. None of the above.

Answer # 1 b. A systematic collection and analysis of data regarding a specific program or intervention. Focuses on an intervention Is not concerned with the testing of general theories of social behavior or their development We use it to determine worth or value The two major purposes of evaluation are to determine the extent to which objectives have been achieved and to discover ideas for improvement.

Components of evaluative research 2. Which of the following statements is/are true? a. It is a reasonable expectation that the scientific method will eliminate all subjectivity that is inherent in the judgment of worth. b. In evaluation research, comparison is essential. You must have a basis for comparing the present state of affairs, as reflected in data, with something else, such as past data, recognized standards, and so forth c. Both of the above. d. None of the above.

Answer # 2 b. In evaluation research, comparison is essential. You must have a basis for comparing the present state of affairs, as reflected in data, with something else, such as past data, recognized standards, and so forth (we are making a judgment) The scientific method does not eliminate subjectivity, but instead offers a systematic method for collection and analysis of data that controls for subjectivity

Types of Evaluation 3. The evaluation of the extent to which an agency is adhering to good standards (e.g., employing only licensed practitioners) would be an example of what kind of evaluation? a. An evaluation of service process b. An evaluation of service output c. An evaluation of service outcome d. All of the above

Answer # 3 a) An evaluation of service process Evaluation of Input (social problems and social needs) Evaluation of process (things done on behalf of the program to meet goals, standards, quality insurance) Evaluation of output (a study of efficiency, in human services efficiency id the units of service delivered) #1 Define the unit of service for a given program # 2 Compute the total costs of delivering the program’s service units # 3 Compute the cost per unit of service Evaluation of outcome (were the program goals achieved) Did clients achieve a gain on the target behavior? Was the client growth due to the intervention?

Models of evaluation 4. Which of the following models of evaluation has the greatest clarity of method for collecting and analyzing data? a. The experiment b. The case study c. The utilization-focused evaluation d. The primary evaluation

Answer # 4 a. The experiment The experiment (uses the best principles of the scientific method) Systematic Introduce and intervention and measure its effects on behavior Measuring behavior before and after the intervention Random assignment of participants to two groups (reduces or eliminates meaningful differences between groups and people, extraneous factors) Large samples are best for experiments

The Case Study Appropriate when you wish to investigate a single individual, event, or social setting to gain an in-depth understanding of how it operates and functions Best suited for (complicated) questions how and why versus who, what, and where The five major components of the Case Study: 1. The development of the study’s questions 2. The development of study propositions 3. The determination of the unit of analysis 4. The development of the logic that links selected data to the propositions 5. The development of the criteria for interpreting the findings

The Utilization-Focused Evaluation A model of evaluation that focuses on “intended use by intended-users” (think of calling in an expert for consulting) Step One: Selecting Intended Users from Identified Stakeholders. Step Two: Identifying Intended Uses of Evaluation. Step Three: Identifying Evaluation Questions. Step Four: Designing the Means for Collecting Data. Step Five: Collecting and Analyzing Data. Step Six: Facilitating Intended Use.

HAPPY HOLIDAYS Break begins December 24th and ends on January 2nd Unit 3 begins December 22nd and ends on January 4th Unit 4 starts January 5th