Research Overview Research Basics

Steps in the Research Process Develop research Idea Search literature & develop hypothesis Select a research design & method Nonexperimental or experimental Manipulations and measurement Sampling and recruitment plan Conduct study Evaluate data Report results Refine/Retest Here is a big picture view of the research process. I’ve listed 7 steps here, but some of these steps could be separate and/or combined. You’ll notice that some of the steps correspond to specific topics that we have already addressed and some that we will discuss in more depth later. Two steps that we have already discussed include developing a research idea, searching the literature, and developing a hypothesis. Today, we’ll mostly be focused on step three – selecting a research design and method- specifically, differences between nonexperimental and experimental research.

Designing a Study Which strategy, design, and/or method should be used to test the hypothesis? Decision based on many factors Advantages & limitations of the strategy/design/method State of current knowledge on topic Practical issues – money, time, resources Ethical issues After you have a general idea of what your hypothesis is, then you have to figure out how you will test this hypothesis. There are several factors that play into the researcher’s decision about how to test her or his hypothesis. These include the advantages and limitations of the each of the strategy/design/method. So, with some strategies we might only be able to describe a variable and other strategies we might be able to make cause-and-effect statements. It also depends on the state of the current knowledge on the topic. What is know and unknown? How do other researchers typically try to answer questions similar to mine? Additionally, there are all sorts of practical issues to consider – money, time, resources We have a bunch of ethical issues to consider as well.

Strategy, Design, & Method Research Strategy General approach to research determined by the kind of question posed by the researcher Nonexperimental, Quasi-experimental or Experimental? Research Design Plan for implementing research strategy Groups versus individuals Same individuals versus different individuals Number of variables manipulated and measured Research Method How are the data being collected? Some researchers use these terms (i.e., research strategy, design, and method) interchangeably. However, there are some differences between them. Research strategy – this is considered a general approach to research that is determined by the kinds of questions posed by the researcher. Typically, the strategy falls into the nonexperimental quasi-experimental, or experimental category. We’ll talk about each of these in this lecture. Research design – refers to the the researcher’s plan for implementing the research strategy. This is often related to the data analytic plan (i.e., planned statistical analyses). Researchers will need to determine if groups versus individuals will be the level of analysis, whether or not to expose all participants to treatment conditions or to use different participants and expose them to only some of the treatment, and the number of variables that will be manipulated or measured. Research method – answers the question of how the data are being collected. Are the data collected through observational methods, through case study methods, etc.

Research Strategies General approach and goals of a study Nonexperimental Descriptive or Single-variable Predictive Quasi-Experimental Experimental Can be some gray area Sometimes a strategy is linked to a method Sometimes a study is somewhere in between two strategies A single study may have multiple hypotheses, use more than one research strategy, and also use different methods. Thinking about research strategies, this include nonexperimental research, which involves descriptive (i.e., single-variable) research or predictive (i.e., correlational) research. Then we have quasi-experimental, which includes some elements of the nonexperimental method and some elements of the experimental method. Lastly, we have experimental research. It is important o not that there can be some gray area. That is, sometimes a strategy is linked to a method. For example, the case study method typically falls into the nonexperimental category. Sometimes a study is somewhere in between two strategies. Additionally, a single study may have multiple hypotheses, use more than one research strategy, and also use different methods.

Descriptive or Single Variable Research Merely describes a variable or variables Not able to say anything about whether variables are related nor make cause-effect statements In the descriptive or single-variable research area, Researchers merely describe a variable or variables. For example, they might determine that 50% of the class prefers discussions or that 65% of the class is made up of female students. Not able to say anything about whether variables are related nor make cause-effect statements. So, we can’t say exactly why 65% of the class is made up of female students.

Predictive Correlational Investigates relations between 2 or more variables Not explaining the relationship (no causal assignment) Identifying patterns in data Direction Strength Form Could have collected the data through a survey, observation, from existing records (i.e., archival), etc. Predictive (i.e., correlational research) – we investigate the relations (i.e., links, associations) between two or more variables. The idea here is that changes in one variable are consistently related to changes in a second variable. For example, is delinquency related to aggression? Is self-esteem related to academic achievement? Are grades related to salary? Here the researcher isn’t saying anything about cause – we’re not explaining the relationship. We’re just describing the relationship. Based on patterns in the data, we can determine The direction of the relationship (i.e., positive or negative) The strength of the relationships (strong, moderate, weak) The form of the relationship (linear or nonlinear) Just because we conducting predictive or correlational research, it doesn’t say anything about how exactly the data were collected (e.g., survey, observation, existing records).

Correlational 1. Direction of relationship Positive, Negative, or No relationship Positive No relation Negative Thinking about the direction of the relationship It can be positive – there is a tendency for two variables to change in the same direction. For example, as study time increases, grades also tend to increase. As exercise increases, overall health tends to increase. As blood pressure decreases, risk of a heart attack decreases. All that matters here is that change tends to be in the same direction. The relationship can also be negative – tendency for two variables to change in opposite direction. For example, as blood alcohol level increases, memory tends to decrease. As smoking increases, lung capacity tends to decrease. It is also possible to reveal the fact that two variables are unrelated to one another – there is no relation between the two. SAME DIRECTION OPPOSITE DIRECTION

Strongest Relationship Correlational 2. Strength of relationship Correlation coefficient Ranges from -1.00 to +1.00 -1.00 reflects a perfect negative relationship +1.00 reflects a perfect positive relationship Closer to zero reflects a weaker relationship Strongest Relationship In correlational research, we can also determine the strength of relationship - by calculating a correlation coefficient These can range from -1.00 to +1.00 -1.00 reflects a perfect negative relationship +1.00 reflects a perfect positive relationship Closer to zero reflects a weaker relationship Imagine you are given the following correlation coefficients (.45, -.25, -.97, .55, .04) and asked to put them in order from weakest to strongest. The correct order would be  .04, -.25, .45, .55, -.97

Correlational 3. Form of relationship Linear relation Change is fairly constant Curvilinear relation Change is not constantly the same Lastly, via correlational research, we can determine the form of the relationship between two variables. It is possible that the relationship is linear, which means the change is fairly constant. Or is possible that it is a curvilinear relation. There tends to be a curvilinear relationship between crowd size and conformity. Adding one or two people to a group of four people is associated with greater conformity than adding one or two people to a group of thirty people.

Guess the Direction & Size of Relation Positive Strong Negative Weak Negative Perfect No Relation

Predictor Variable  X-axis Criterion Variable  Y-axis Correlational Predictor Variable  X-axis Criterion Variable  Y-axis Longitudinal Current measure Thought to be the cause Longitudinal Future measure Thought to be the effect Variables – concepts that you can measure (that vary) Predictor variables – these are measured variables that researchers have not manipulated, but they are thought to have an effect on the DV. Can’t establish cause, but for theoretical or empirical reasons researchers suspect that this is the direction of the relationship. In longitudinal research (i.e., conducted over time), we would label the variable that measured first as the predictor variable. For example, if we were measuring the relationship between SAT scores and first year college grades – SAT scores would be considered the predictor variable. Criterion/outcome variables – like dependent variables (but researchers can’t be a 100% sure). In longitudinal research, this would be the variable that was measured second or later. In the example above, first year college grades would be considered the criterion variable. Sometimes this variable is also called the outcome variable in research articles. Should also note, when placing these variables on a scatterplot, the predictor variable is usually placed on the x-axis and the outcome variable (i.e., criterion) is placed on the Y-axis. Sometimes also called the DV or the outcome variable

Correlation does not imply causation Alternative Explanations Directionality problem Third-variable problem Lurking variables, third variables, confounds Time spent playing violent videogame Aggressive behavior As I noted earlier, in correlational research we cannot make causal claims. This is because there may be alternative interpretations. For example, imagine that we find a strong positive correlation between time spent playing violent videogames and aggression. If we measure these two variables at the same time, we cannot be sure which is the cause and which is the effect. We call this the directionality problem. It is possible that watching violent videogames causes increases in aggression, or is is equally plausible that people who are aggressive like to play violent videogames (i.e., aggression causes video game playing). We don’t know which it is. We also have the third-variable problem. The idea here is that there is some third variable we didn’t measure that is really responsible for the relationship. For example, perhaps low parental monitoring causes greater violent video game play and it also causes more aggressive behavior. Another example, there is a positive relationship between ice cream sales and shark attacks. Eating ice cream doesn’t cause a shark attack. There is a third variable that is responsible for this - temperate. During hotter months, people tend to eat more ice cream. During hotter, month people tend to be in the ocean putting themselves at greater risk of a shark attack. The third-variable is sometimes also referred to as a lurking variable or a confound variable. Parental Monitoring Violent videogames Aggression

Quasi-experimental Attempts to answer cause-effect questions However, includes a flaw that prevents this Usually examining pre-existing variables (i.e., characteristics about participants that can’t be manipulated) or participants self-select themselves into groups I.e., “Subject”, “Participant” “Quasi-independent” E.g., gender, smoking status There is also quasi-experimental research. Here, researchers are attempting to answer cause-effect questions; however, there is a flaw in the research that prohibits the researchers from making a true cause-effect statement. There is some other nagging variable that could possibly serve to explain the relationship. For example, if the researchers were unable to manipulate key variables or random assignment. In quasi-experimental research, researchers are typically comparing two different groups or one group at different times. Usually these are pre-existing variables – sometimes called “subject variables”’ or “quasi-independent variables”. So, for example perhaps we want to compare boys to girls, or smokers to non-smokers, or students in online course versus face-to-face course. Researchers cannot always manipulate some variables (i.e., assign some people to be boys and some to be girls). There are different types of quasi-experimental studies, which we will discuss in later lectures.

Explanatory Experimental Research Determine one variables cause changes in another variable Involves manipulation of variables, random assignment, & control Independent Variable (IV) – manipulated Randomly assigned to levels/conditions of the IV Random assignment is a procedure used to eliminate participant variables as alternative explanations Dependent Variable (DV) – measured Hold all other variables constant Experimental research – we are trying to determine the cause of some variable. We want to explain the relationship between variables. We’re saying more than that these two variables are related --- we are explaining how one variable is affecting the other variable. Experimental research involves the manipulation of variables, the use of random assignment, and high levels of control. To eliminate the directionally problem discussed earlier in correlational research, researchers will manipulate the variable they think is the cause. This variable is referred to as the independent variable. For example, researchers may examine the hypothesis that exercise leads to weight loss. Researchers could manipulate exercise by having some participants workout for 30 minutes 5 days a week and having other participants workout for 30 minutes 1 day a week. In this example, exercise is the independent variable and there are two levels or conditions (i.e., 5 days a week vs. 1 day a week). To eliminate participant variables as alternative explanations (i.e., confound variables), researchers use random assignment to determine which participants will exercise for 5 days a week and which will exercise for 1 day a week. For example, the researcher may flip a coin. If the sample size is large enough and random assignment has been used, this increases the chances that the two groups (i.e., those exercising 5 days and those exercising 1 day) are equal at the start of the study. That is, both groups should have similar weights, IQ scores, motivation, shoe size, etc. If there is a difference in weight loss at the end of the study, we can be more sure that it is due to our independent variable and not some other variable, if random assignment is used. After researchers have used random assignment to assign participants to different levels of the independent variable, they must then measure the variable that is thought to be the effect - in our example, weight. This variable is called our dependent variable. Throughout the study, researchers conducting the experiment will attempt hold all other variables constant (i.e., keep them the same). For example, perhaps those in the two exercise groups are asked to exercise at the same time of day and/or eat similar meals. Variable X (Cause) Variable Y (Effect)

Other interpretations??? Incidental (extraneous) variables Other variables that are not the focus Confound variables Vary reliably with the manipulated variable Serve as an alternative explanation Sometimes referred to as “third-variables” or “lurking variables” Incidental variables (extraneous variables) – variables that vary in the experiment (e.g., temperature of the lab), but that are not varying along with the independent variables. Sometimes the room is hot when people watch the sad movie, other times cold. Sometimes hot when people watch the happy movie, etc. This is not a confound, it is error. Extraneous/Incidental variables are what? Variables that really aren’t part of what we’re interested in examining, but may have an effect on the dependent variable. (considered noise) Confound Variables – variables that vary reliability with the independent variable (IV). E.g., A research wants to know whether sadness decreases people’s willingness to be friendly. Participants friendliness to the experimenter is measured (dependent variable). Mary is the experimenter in all of the sessions where people watch a sad movie. John is the experimenter in all of the sessions where people watch a happy movie. Results were consistent with the hypothesis: Those participants who watched a happy movie were friendlier to the experimenter than those who watched a sad movie. Experimenter is the confound – it varied with movie type. Maybe the reason why people were nicer to John than to Mary, was not because they were sad but rather because John was simply a nicer person. Cannot rule out this alternative explanation. Confounds are those variables that vary systematic with our IV; thus, making it difficult to tell whether the effect was due to the difference in the IV or something else.

More on Independent Variables Does 1 IV have an effect on the DV? The terms – levels, conditions – have similar meaning. Interaction effects (e.g., Factorial Designs) Do IVs interact to influence a DV? The terms – levels, conditions – now have different meanings. Levels refer to the different amounts of each IV. Conditions refer to the different combinations of the levels of the IVs. In an experiment, when only one independent variable is manipulated – the terms – levels, conditions – have similar meanings. You may come across experiments in which more than one independent variable is manipulated. In these types of studies, researchers think two variables might interact to have an effect on a another variable. For example, we know that alcohol influences people’s reaction time, but the how much it effects reaction time may depend on how much people recently ate. That is, people who recently ate may be less affected than people who did not. Here alcohol and recent eating may interact to influence reaction time. Researchers could examine this by randomly assigning to different levels of alcohol intake(e.g., 0 oz., 10 oz., or 20 oz.). Thus, alcohol is an independent variable with three levels. Perhaps the researchers also randomly assign some participants to consume different amounts of food (e.g., ½ sandwich or 1 sandwich) before ingesting the alcohol. Thus, amount of food would be another independent variable and it would have two levels. So, levels refers to the amounts of each independent variable. Conditions refers to the different combinations of the levels of the independent variables. In our example, there would be six conditions (3x2). Participants are assigned to (1) eat ½ sandwich and drink 0 oz. of alcohol; (2) eat ½ sandwich and drink 10 oz. of alcohol; (3) eat ½ sandwich and drink 20 oz. of alcohol; (4) eat 1 sandwich and drink 0 oz. of alcohol; (5) eat 1 sandwich and drink 10 oz. of alcohol; (6) eat 1 sandwich and drink 20 oz. of alcohol  6 conditions.

Example Task Difficulty (A) Easy (A1) Difficult (A2) Presence of others (B) Alone (B1) Easy task alone Difficult task alone Audience (B2) East task w/ audience Difficult task w/ audience Here is another example. This study has two independent variables (IV) (IV) Presence of others 2 levels: Alone vs. Audience (IV) Task difficulty 2 levels: Easy vs. Difficult 4 conditions

Think this through… Explain why we cannot make causal claims with correlational data. Be specific. Explain why we can make causal claims with experimental data (assuming well-designed study).

Mini-Review Explain why we cannot make causal claims with correlational data. Explain why we can make causal claims with experimental data (assuming well-designed study). What’s the difference between levels and conditions? If I said that the correlation between anxiety and depression was .75, how would you interpret this correlation coefficient?