Experimental Method and Statistical Reasoning in Psychology

The Scientific Method The four basic goals of psychology are to (1) describe, (2) explain, (3) predict, and (4) control or influence behavior and mental processes. To achieve these goals, psychologists rely on the scientific method. The scientific method refers to a set of assumptions, attitudes, and procedures that guide researchers in creating questions to investigate, in generating evidence, and in drawing conclusions. Like all scientists, psychologists are guided by the basic scientific assumption that events are lawful. When this scientific assumption is applied to psychology, it means that psychologists assume that behavior and mental processes follow consistent patterns. Psychologists are also guided by the assumption that events are explainable. Thus, psychologists assume that behavior and mental processes have a cause or causes that can be understood through careful, systematic study.

The Scientific Method In striving to discover and understand consistent patterns of behavior, psychologists are open-minded. They are willing to consider new or alternative explanations of behavior and mental processes. However, their open-minded attitude is tempered by a healthy sense of scientific skepticism. That is, psychologists critically evaluate the evidence for new findings, especially those that seem contrary to established knowledge. And, in promoting new ideas and findings, psychologists are cautious in the claims they make.

The Steps in the Scientific Method Like any science, psychology is based on empirical evidence—evidence that is the result of objective observation, measurement, and experimentation. As part of the overall process of producing empirical evidence, psychologists follow the four basic steps of the scientific method. In a nutshell, these steps are: Formulate a specific question that can be tested. Design a study to collect relevant data. Analyze the data to arrive at conclusions. Report the results.

Step 1. Formulate a Hypothesis That Can Be Tested Empirically Once a researcher has identified a question or an issue to investigate, he or she must formulate a hypothesis that can be tested empirically. Formally, a hypothesis is a tentative statement that describes the relationship between two or more variables. A hypothesis is often stated as a specific prediction that can be empirically tested, such as “psychological stress increases the likelihood of physical illness.” The variables contained in any given hypothesis are simply the factors that can vary, or change. These changes must be capable of being observed, measured, and verified. The psychologist must provide an operational definition of each variable to be investigated. An operational definition defines the variable in terms of how it is to be measured, manipulated, or changed. Operational definitions are important because many of the concepts that psychologists investigate—such as memory, happiness, or stress—can be measured in more than one way. In providing operational definitions of the variables in the study, the researcher spells out in very concrete and precise terms how the variables will be manipulated or measured. In this way, other researchers can understand exactly how the variables were measured or manipulated in a particular study.

Step 2. Design the Study and Collect the Data Descriptive methods are research strategies for observing and describing behavior, including identifying the factors that seem to be associated with a particular phenomenon. Descriptive methods answer the who, what, where, and when kinds of questions about behavior. Who engages in a particular behavior? What factors or events seem to be associated with the behavior? Where does the behavior occur? When does the behavior occur? How often? The experimental method is used to show that one variable causes change in a second variable. In an experiment, the researcher deliberately varies one factor, then measures the changes produced in a second factor. Ideally, all experimental conditions are kept as constant as possible except for the factor that the researcher systematically varies. Then, if changes occur in the second factor, those changes can be attributed to the variations in the first factor.

Step 3. Analyze the Data and Draw Conclusions Once observations have been made and measurements have been collected, the raw data need to be summarized and analyzed. Researchers use the methods of a branch of mathematics known as statistics to summarize, analyze, and draw conclusions about the data they have collected. Researchers rely on statistics to determine whether their results support their hypotheses. They also use statistics to determine whether their findings are statistically significant. If a finding is statistically significant, it means that the results are not very likely to have occurred by chance. As a rule, statistically significant results confirm the hypothesis. Appendix A provides a more detailed discussion of the use of statistics in psychology research.

Meta-analysis A statistical technique called meta-analysis is increasingly being used in psychology to analyze the results of many research studies on a specific topic. Basically, meta-analysis involves pooling the results of several studies into a single analysis. By creating one large pool of data to be analyzed, metaanalysis can sometimes reveal overall trends that may not be evident in individual studies. Meta-analysis is especially useful when a particular issue has generated a large number of studies, some of which have produced weak or contradictory results.

Step 4. Report the Findings For advances to be made in any scientific discipline, researchers must publish or share their findings with other scientists. In addition to reporting their results, psychologists provide a detailed description of the study itself, including the following: Who participated in the study How participants were selected How variables were operationally defined What procedures or methods were used How the data were analyzed What the results seem to suggest Describing the precise details of the study makes it possible for other investigators to replicate, or repeat, the study. Replication is an important part of the scientific process. When a study is replicated and the same basic results are obtained again, scientific confidence that the results are accurate is increased. Conversely, if the replication of a study fails to produce the same basic findings, confidence in the original findings is reduced.

Naturalistic Observation When psychologists systematically observe and record behaviors as they occur in their natural settings, they are using the descriptive method called naturalistic observation. Usually, researchers engaged in naturalistic observation try to avoid being detected by their subjects, whether people or nonhuman animals. The basic goal of naturalistic observation is to detect the behavior patterns that exist naturally—patterns that might not be apparent in a laboratory or if the subjects knew they were being watched.

A Case Study A case study is an intensive, in-depth investigation of an individual or a small group of individuals. Case studies involve compiling a great deal of information, often from a variety of different sources, to construct a detailed picture of the person. The subject may be intensively interviewed, and his or her friends, family, and co-workers may be interviewed as well. Psychological records, medical records, and even school records may be examined. Other sources of information can include extensive psychological testing and observations of the person’s behavior. Clinical psychologists and other mental health specialists routinely use case studies to develop a complete profile of a psychotherapy client. Case studies are also used to investigate rare, unusual, or extreme conditions.

Surveys A direct way to find out about the behavior, attitudes, and opinions of people is simply to ask them. In a survey, people respond to a structured set of questions about their experiences, beliefs, behaviors, or attitudes. One key advantage offered by survey research is that information can be gathered from a much larger group of people than is possible with other research methods. Typically, surveys involve a carefully designed questionnaire in a paper-and-pencil format that is mailed to a select group of people. Computer-based or Internet-based surveys have become increasingly more common. And, surveys are still often conducted over the telephone or in person, with the interviewer recording the person’s responses.

Surveys As with paper-and-pencil surveys, the interviewer usually asks a structured set of questions in a predetermined order. Surveys are seldom administered to everyone within the particular group or population under investigation. Instead, researchers usually select a sample—a segment of the larger group or population. Selecting a sample that is representative of the larger group is the key to getting accurate survey results. A representative sample very closely parallels, or matches, the larger group on relevant characteristics, such as age, sex, race, marital status, and educational level.

The Correlational Study A correlational study examines how strongly two variables are related to, or associated with, each other. Correlations can be used to analyze the data gathered by any type of descriptive method. The participants also filled out a questionnaire on past delinquent behavior and completed tests designed to measure different personality characteristics. Finally, each respondent’s cumulative grade should point average. Once the data were collected from their survey participants, psychologists used statistical procedures to calculate a figure called a correlation coefficient. A correlation coefficient is a numerical indicator of the strength of the relationship between two factors.

The Correlational Study A positive correlation is one in which the two factors vary in the same direction. That is, the two factors increase or decrease together. For example, Anderson and Dill found that there was a positive correlation of .22 between the amount of time spent playing violent video games and aggressive personality characteristics. That is, as the amount of time spent playing violent video games increased, aggression scores on personality tests increased. In contrast, a negative correlation is one in which the two variables move in opposite directions: As one factor decreases, the other increases. For example, Anderson and Dill found that there was a negative correlation of .20 between the amount of time spent playing video games and academic achievement, as measured by cumulative college grade-point average. As the amount of time spent playing video games increased, college grade-point average decreased.

Experimental method The experimental method is a research method used to demonstrate a cause-and-effect relationship between changes in one variable and the effect that is produced on another variable. Conducting an experiment involves deliberately varying one factor, which is called the independent variable. The researcher then measures the changes, if any, that are produced in a second factor, called the dependent variable. The dependent variable is so named because changes in it depend on variations in the independent variable. To the greatest degree possible, all other conditions in the experiment are held constant. Thus, when the data are analyzed, any changes that occur in the dependent variable can be attributed to the deliberate variations of the independent variable. In this way, an experiment can demonstrate a cause-and-effect relationship between the independent and dependent variables.

Limitations of Experiments The strength of a well-designed experiment is that it can provide convincing evidence of a cause-and-effect relationship between the independent and dependent variables. Experiments do have limitations, however. Because experiments are often conducted in highly controlled laboratory situations, they are frequently criticized for having little to do with actual behavior. That is, the artificial conditions of some experiments may produce results that do not generalize well, meaning that the results cannot be applied to real situations or to a more general population beyond the participants in the study.

Limitations of Experiments In order to make experimental conditions less artificial, experiments are sometimes conducted in a natural setting rather than in a laboratory. Another limitation of the experimental method is that even when it is possible to create the conditions that the researchers want to study, it may be unethical to do so.