Fitting Research Design to Research Purpose

The overall purpose of most research is to investigate a predicted relationship between the occurance of some variation of one variable, A, and the occurance of variations of another variable, B, in the same setting.

Variables may be states of the physical or social environment (e. g Variables may be states of the physical or social environment (e.g., weather conditions, the number of people present in the situation), properties of a stimulus (e.g., the facial expression in a photograph, the content of a message), or characteristics of a person or a person’s behavior (e.g., mood state, degree of aggression). Relationships can be between two environmental variables (e.g., the relationship between variations in the coldness of the weather on the number of people who are in an outdoor setting), between an environmental or stimulus variable and an individual characteristic or trait (e.g., the relationship between the state of the weather and the average mood of people exposed to it), or between two characteristics of an individual (e.g., the relationship between mood and aggressiveness). To say that there is a relationship between two such variables means that if the state of one variable differs or changes, we can expect that the state of the other will also change or differ. So, for example, if we measure people’s mood on a sunny day and then again on a cloudy day and there is a difference in mood such that mood is more negative on the second occasion, then we can say we have shown a relationship between the state of the weather and individuals’ moods.

The nature of the relationship may be specified in terms of the form it will take, that is, what kind of changes in B will accompany particular changes in A and what the causal direction of the relationship will be. Directionality may be differentiated into three types. Unidirectional causation, in which changes in A are predicted to produce subsequent changes in B, but changes in B are not expected to influence A (e.g., increases in the temperature-humidity index are accompanied by an increase in aggressive responses of rats, but the degree of aggressiveness of rats does not affect weather conditions).

Bidirectional causation, in which changes in A lead to changes in B and, in addition, changing B produces changes in A (e.g., perceiving threat produces feelings of anxiety, and increasing anxiety enhances the perception of threat).

Noncausal covariation (or third-variable causation), in which changes in A are indirectly accompanied by changes in B because both A and B are determined by changes in a third variable, C (e.g., birth rate and consumption of beef steak rise or fall with increases or decreases in the cost of living index).

Moderators and Mediators In addition to specifying the nature and direction of a causal relationship under study, it also is important to distinguish between two different types of “third variables” that can influence causal relationships—moderators and mediators

Sometimes causal relationships can be either augmented or blocked by the presence or absence of factors that serve as moderator variables. To take another weather-related illustration, consider the causal relationship between exposure to sun and sunburn. Although there is a well-established cause–effect link here, it can be moderated by a number of factors. For instance, the relationship is much stronger for fair-skinned individuals than for dark-skinned persons. Thus, fair skin is a moderator variable that enhances the causal relationship between sun exposure and burning. However, this does not mean that the sun–sunburn relationship is spurious.

The moderator variable (skin pigmentation) does not cause the effect in the absence of the independent variable (sun exposure). Other moderator variables can reduce or block a causal sequence. For instance, the use of effective suntan lotions literally “blocks” (or at least retards) the causal link between the sun’s ultraviolet rays and burning. Thus, a researcher who assesses the correlation between sun exposure and sunburn among a sample of fair-skinned people who never venture outdoors without a thick coat of 30 SPF sunblock would be ill-advised to conclude that the absence of correlation implied the absence of causation.

Moderator relationships can be represented notationally as follows:

It is important here to distinguish between third variables that serve as moderators and those that serve as mediators of a cause–effect relationship. With moderator effects, the causal link is actually between X and Y, but the observed relationship between these two variables is qualified by levels of variable C, which either enhances or blocks the causal process. A mediational relation, on the other hand, is represented as follows:

In this case, the presence of C is necessary to complete the causal process that links X and Y. In effect, varying X causes variations in C, which, in turn, causes changes in Y.

To return to our weather examples, the effect of rain (X) on depression (Y) may be mediated by social factors (C). Rain (X) causes people to stay indoors or to hide behind big umbrellas, hence reducing social contact (C). Social isolation (C) may, in turn, produce depression (Y). However, rain may not be the only cause of social isolation. In this case, rain as an independent variable is a sufficient, but not necessary, cause in its link to depression. To demonstrate that X causes Y only if C occurs does not invalidate the claim that X and Y have a causal relationship; it only explicates the causal chain involved.

FORMS OF VALIDITY The research strategy should be be guided by considerations of two types of validity—internal and external validity Internal validity has to do with the certainty with which one can attribute a research outcome to the application of a treatment or manipulation that is under the rigid control of the researcher. Internal validity is about the extent to which causal inferences can legitimately be made about the nature of the relationship between the treatment and the outcome.

Just as the choice of research method must be conditioned on considerations of the nature of the phenomenon of interest, so too must the role of statistical techniques be evaluated with respect to the general goal of eliminating or reducing the plausibility of rival alternative hypotheses for the events under investigation.

One potential rival explanation that plagues social research at all stages of investigation is the operation of “chance.” The phenomena of interest to the social sciences are generally subject to considerable nonsystematic variation, that is, variations from individual to individual and, within individuals, from time to time. The purpose of most inferential statistical tests is to assess the validity of this rival explanation of results in terms of the probability, or likelihood, that the obtained data pattern could have occurred by chance.

The results of a statistical inference test tell us the probability of a Type I error of inference—the likelihood that a result would be obtained when the null hypothesis (no true relationship between the independent and dependent variable) is actually valid. Statistical significance is achieved when this probability is so low as to render the chance explanation implausible.

External validity is concerned with the issue of generalizability External validity is concerned with the issue of generalizability. Assuming that a research finding is internally valid, external validity has to do with the extent that it can be generalized to other respondent groups, to other settings, and to different ways of operationalizing the conceptual variables.

Even when internal validity is high, however, there may arise questions about the validity of interpretations of causal effects obtained in any given study, particularly their applicability or generalizability outside of the experimental setting. These concerns constitute questions of external validity, which can be further divided into questions of (1) generalizability of operationalizations and (2) generalizability of results to other places and participant populations

Validity of Operationalizations corresponds to misusing a statistical technique such as scale vs. technique conflict or failure of assumptions that the technique rests on.

Once a research study has been completed, the investigator is usually interested in reaching conclusions that are generalizable across people and across settings. Threats to this form of external validity arise from possible interaction effects between the treatment variable of interest and the context in which it is delivered, or the type of participant population involved. (a’ka random sampling) An experimental finding lacks external validity if the nature of the effect of the independent variable would be reduced or altered if the setting or the participant population were changed.

Need to cover basic concepts in probability and statististics for reviewing reliability.