Chapter 23 Nonexperimental Research

Definition Nonexperimental research is systematic empirical inquiry in which the scientist does not have direct control of independent variables because their manifestations have already occurred or because they are inherently not manipulable. Inferences about relations among variables are made, without direct intervention, from concomitant variation of independent and dependent variables.

Basic Difference Between Experimental and Nonexperimental Reseach In nonexperimental research, y is observed, and an x, or several xs, are also observed. They are observed either before, after, or concomitant to the observation of y. The basic purpose of both is the same: to establish the empirical validity of so-called conditional statements of the form: If p, then q. The essential difference is direct control of p, the independent variable. In experimental research, p can be manipulated, which is rather direct “control.”

Basic Difference Between Experimental and Nonexperimental Reseach In nonexperimental research, direct control is not possible: neither experimental manipulation nor random assignment can be used. These are two essential differences between experimental and nonexperimental approaches. The most important difference between experimental research and nonexperimental research, then, is control. In experiments, investigators at least have manipulative control: they have at least one active variable.

Self-Selection and Nonexperimental Research It is possible to draw participants at random in both experimental and nonexperimental research. But it is not possible, in nonexperimental research, to assign participants to groups at random or to assign treatments to groups at random. Participants can “assign themselves” to groups.

Self-Selection and Nonexperimental Research Self-selection occurs when the members of the groups being studied are in the groups, in part, because they differentially possess traits or characteristics extraneous to the research problem. Note that careful scientific investigators will usually not use the word cause unless the study was performed under the strictest conditions.

Self-Selection and Nonexperimental Research Self-selection can be a subtle business. There are two types: (1) self-selection into samples and (2) self-selection into comparison groups. The latter occurs when participants are selected because they are in one group or another: cancer and no cancer, college and no college, underachievement and no underachievement.

Self-Selection and Nonexperimental Research For example, we may select college freshmen at random and then follow them to determine the relation between intelligence and success in college. The students selected themselves into college, so to speak. One or more of the characteristics they bring with them to college, other than intelligence—socioeconomic level, motivation, family background—may be the principal determinants of college success.

Large-Scale Nonexperimental Research Nonexperimental behavioral research often focuses on large problems of social and human importance: social class, political processes, segregation and desegregation, public attitudes, school achievement, for example.

Large-Scale Nonexperimental Research Determinants of School Achievement Do American schools offer equal educational opportunity to all children? Equally important, however, was the question of the relation between student achievement and the kinds of schools students attend. Most achievement variance was accounted for by what the children brought with them to school.

Large-Scale Nonexperimental Research Response Style Differences between East Asian and North American Students The independent variable in this study was culture and it was not manipulated. Regardless of the sampling in this study, the study is nonexperimental. Hence, one cannot really state explicitly that if you are from this culture, you will respond on rating scales in such-and-such way.

Smaller Scale Nonexperimental Research Cochran and Mays (1990): Sex, Lies, and HIV Even though the data points strongly toward men’s willingness to lie in order to obtain sexual favors, one cannot automatically assume that all men will lie about their sexual history.

Smaller Scale Nonexperimental Research Elbert (1993): Impaired Reading and Written Language in Attention Deficit Children. The nonexperimental nature of this study resulted in very different-sized groups. Elbert did however, perform numerous analyses to check on the equality of the groups on the variables age, grade level, mother’s level of education, and IQ. Statistical tests between ADD+H and ADD-H on these variables were not significant. Note the nonexperimental nature of Elbert’s study. No manipulated independent variable and no randomization.

Testing Alternative Hypotheses In nonexperimental studies, although one cannot have the confidence in the “truth” of an “If x, then y” statement that one can have in the experiments, it is possible to set up and test alternative or “control” hypotheses. (Of course, alternative hypotheses can also be and are tested in experimental studies.)

Testing Alternative Hypotheses Similarly, we can test alternative dependent variables, which also imply alternative hypotheses. Alper, Blane, and Abrams (1955) asked: Do social class differences in child-training practices result in class differences in personality? The results have differences when fingerpaints and crayons were used in measuring dependent variables.

Testing Alternative Hypotheses The method of testing alternative hypotheses, though important in all research, is particularly important in nonexperimental studies, because it is one of the only ways to control the independent variables of such research. Lacking the possibility of randomization and manipulation, nonexperimental researchers, perhaps more so than experiments, must be very sensitive to alternative hypothesis-testing possibilities.

Evaluation of Nonexperimental Research The limitations of nonexperimental Interpretation Nonexperimental research has three major weaknesses, two of which have already been discussed in detail: (1) the inability to manipulate independent variables, (2) the lack of power to randomize, and (3) the risk of improper interpretation. In other words, compared to experimental research, other things being equal, nonexperimental research lacks control; this lack is the basis of the third weakness.

Evaluation of Nonexperimental Research The Value of Nonexperimental Research Despite its weaknesses, much nonexperimental research must be done in psychology, sociology, and education simply because many research problems do not lend themselves to experimental inquiry.

Conclusions Replication is always desirable, even necessary. An important point being made is that replication of research does not only mean repetition of the same studies in the same settings. It can and should mean testing empirical implications of the theory—interpreting ”theory” broadly—in similar and dissimilar situations and experimentally and nonexperimentally.