CHAPTER 12 Descriptive, Program Evaluation, and Advanced Methods

Descriptive Research Descriptive research designs typically involve the researcher observing and systematically recording some particular behavior(s) for the purpose of better describing the behavior in question. Some common descriptive designs include the following: –Large-Scale Surveys –Archival Data Methods –Naturalistic Observation –Structured Test Methods

Large-Scale Surveys Large-scale surveys are conducted to describe the behaviors or characteristics of some population. –They basically involves lists of questions that participants are asked to answer. –Because of its purpose large-scale surveys must have representative samples. Why? Advantages: –Easy to conduct –Generate lots of data in a short period of time –Can be on virtually any topic Disadvantages: –Nonresponse bias – many participants do not respond. –Social Desirability bias: participants answer questions in ways they think are socially acceptable. The data collected can be analyzed by a variety of methods, but most often simple descriptive statistics are used (e.g., percentages, frequencies).

Archival Research Archival records are documents that can be typically found in university libraries and government buildings. Some common types of archival information that psychologists may be interested in include: –Data on birth weights and rates –Death rates and causes of death –Weather reports –Crime rates –Sales of certain products –And many more! Archival research often involves content analysis – the objective description, classification and evaluation of previous research. Thus, archival research is considered descriptive in nature.

Naturalistic Observation Naturalistic observation is another descriptive method where a researcher observes subjects’ behavior in a “natural” or normal setting without trying to influence or interfere with the behavior. Advantages: –Behavior is observed in a natural setting –Free from reactivity effects Disadvantages: –Can take a very long time –No knowledge or control of events that cannot be seen. –Experimenter bias

Structured test methods are descriptive methods that involve setting up or structuring some test situation to determine how individuals behave in that situation. –The structured test situation is designed to either increase or decrease the likelihood of a certain behavior. –Also structured tests help to increase the frequency of the behavior of interest – making it easier to study. Structured Test Methods

We learned in chapter 11 that case studies were considered intrasubject designs because they were typically performed on one or a few participants. Case studies can also be considered descriptive research since case reports involve careful systematic observation of the participant. Case Study

Program Evaluation Program evaluation is a type of research that typically involves assessing the need for and success of a particular program. The programs we are talking about here are generally large scale – involving institutions like school boards or governments. The outcomes of these programs often have significant implications to various groups of people. The goals of program evaluation are to provide administrators unbiased information and feedback pertaining to the proposed or current program. –This information can be used to begin a new program, cancel an old program, or to restructure a current program to make it more efficient.

Program Evaluation: Needs Analysis A needs analysis is a study that attempts to identify: –If a problem exists –If so, how many people (or animals) are affected, –What factors will help reduce of eliminate the problem –And how many people will use the program aimed to fix the problem. A needs analysis is almost exclusively a descriptive research strategy. Methods typically employed involve: –Archival data (census, public records) –Surveys

Program Evaluation: Process Analysis A process analysis examines whether a currently operating program is: –Known to exist and being utilized by the target population –Being implemented and structured properly Data are typically gathered through surveys, staff records, and observation of program in action. –Thus process analysis is descriptive in nature.

Program Evaluation: Outcome Analysis An outcome analysis assesses the impact or results of a program. –Is the program achieving its goal? The success of a program can be accomplished by various means: –Descriptive strategies would base evaluations on frequencies and percentages acquired before and after the program. –Quasi-experimental strategies would use t-tests and ANOVA to compared means before and after the program. –Experimental strategies would also use t-tests and ANOVA but would have a control group, and equivalent control and experimental groups.

Program Evaluation: Efficiency Analysis If the outcome analysis shows that a program works, administrators now want to how efficient the program is. Efficiency analysis is usually deals with the monetary aspect of the program and evaluates the programs costs relative to its benefits. The data generated here is often in ratio form, e.g., cost-benefit ratio.

Advanced Methods: Meta-Analysis –A meta-analysis is a statistical method that combines the results of previous studies (on any given topic) to provide an integrated overview of those previous findings. –Two common approaches used in meta- analysis include: Combined probability Combined estimation

Advanced Methods: Multivariate Analyses –Multivariate analyses are a group of statistical procedures that typically involves either: Two or more dependent variables Advanced correlational methods with one dependent variable and multiple independent variables. Some examples of multivariate analysis: –Multivariate Analysis of Variance (MANOVA) –Structural Modeling and Path Analysis