Experiment Basics: Variables Psych 231: Research Methods in Psychology

Class Experiment Class Experiment Quiz 5 is due Friday Turn in your data sheets & consent forms I will analyze the data and the results will be discussed in labs Quiz 5 is due Friday Class Experiment

Results Mean: 78.0 Median: 78 Range: 52-98 If you want to go over your exam set up a time to see me (office hours or by appt.) Exam 1

Common errors: Four Cannons of scientific method (& pg 6-9 of the textbook) Exam 1

Common errors: 2 of 5 APA General Ethical Principles (& pg 51-54 of the textbook) Exam 1

Common errors: A researcher examined the relationship between music and mood. He presented two groups of participants the same video clips but the two groups received different musical soundtracks. Following the presentation of the videos, participants completed a questionnaire designed to measure their current mood. What was the IV and DV Identify the Research Design used in the study Identify a major advantage of using this research design for this study Identify a major disadvantage/limitation of using this research design for this study. Manipulated the music (IV) and measured the effects on mood (DV) Experimental Design Major advantage: ability to make causal claims, impose control, etc. Major disadvantage: lower external validity and/or generalizability Exam 1

So you want to do an experiment? You’ve got your theory. What behavior you want to examine Identified what things (variables) you think affect that behavior So you want to do an experiment?

So you want to do an experiment? You’ve got your theory. Next you need to derive predictions from the theory. These should be stated as hypotheses. In terms of conceptual variables or constructs Conceptual variables are abstract theoretical entities Consider our class experiment Theory & Hypotheses: Activation of social concepts & how connected to your social network you feel. Social vs. Non-social websites Cell phone presence So you want to do an experiment?

So you want to do an experiment? You’ve got your theory. Next you need to derive predictions from the theory. Now you need to design the experiment. You need to operationalize your variables in terms of how they will be: Manipulated Measured Controlled Be aware of the underlying assumptions connecting your constructs to your operational variables Be prepared to justify all of your choices So you want to do an experiment?

Variables Conceptual vs. Operational Conceptual variables (constructs) are abstract theoretical entities Operational variables are defined in terms within the experiment. They are concrete so that they can be measured or manipulated Conceptual Social connectedness Social concepts/words Activation of social concepts Operational Cell phone presence or absence Websites social or non-social Word scramble test Independent variables Dependent variable Variables Other Variables in our experiment: Time for unscrambling Kind of cell phone present age, gender, time of testing, … Extraneous variables

Many kinds of Variables Independent variables (Explanatory) Dependent variables (Response) Extraneous variables Control variables Random variables Confound variables Correlational designs have similar functions Many kinds of Variables Using Mythbusters to identify IV and DV Independent Variable, Dependent Variable, Constants, and Control

Many kinds of Variables Conceptual Operationalized Independent variables Dependent variables Extraneous variables Control variables Random variables Confound variables Speed Walk vs. run How wet Weight of suit Velocity of the rain Speed of the run What you are wearing Amount of rainfall Amount of wind If you look carefully at their design, it looks like they treat this as another IV (listen around 1:52 of the video) Mythbusters exp (~3 ½ mins) (alt version with CC) Many kinds of Variables Note: Mythbusters revisit the issue and come to a different conclusion Local news story & minutephysics

Many kinds of Variables Independent variables (explanatory) Dependent variables (response) Extraneous variables Control variables Random variables Confound variables Many kinds of Variables

Independent Variables The variables that are manipulated by the experimenter (sometimes called factors) Each IV must have at least two levels Remember the point of an experiment is comparison Combination of all the levels of all of the IVs results in the different conditions in an experiment Independent Variables

Independent Variables Condition 1 Condition 2 Factor A 1 factor, 2 levels Cond 1 Factor A Cond 3 Cond 2 1 factor, 3 levels Cond 1 Factor B Cond 3 Cond 2 Factor A Cond 4 Cond 6 Cond 5 2 factors, 2 x 3 levels Independent Variables

Independent Variables Mythbusters exp (~3 ½ mins) 2 factors, 2 x 2 levels Cond 1 Windy Cond 2 Speed Cond 3 Cond 4 Walk Run Wind No Wind They only talk about the effect of Speed, but they could have also talked about Wind, and the interaction of speed and wind Independent Variables

Manipulating your independent variable Methods of manipulation Straightforward Stimulus manipulation - different conditions use different stimuli Instructional manipulation – different groups are given different instructions Staged Event manipulation – manipulate characteristics of the context, setting, etc. Subject (Participant)– there are (pre-existing mostly) differences between the subjects in the different conditions leads to a quasi-experiment Social vs. non-social websites Presence or absence of cell phone Manipulating your independent variable Manipulating IVs through embodiment Student made video?

Choosing your independent variable Choosing the right levels of your independent variable Review the literature Do a pilot experiment Consider the costs, your resources, your limitations Be realistic Pick levels found in the “real world” Pay attention to the range of the levels Pick a large enough range to show the effect Aim for the middle of the range Choosing your independent variable Levels of your IV

Identifying potential problems These are things that you want to try to avoid by careful selection of the levels of your IV (may be issues for your DV as well). Demand characteristics & Reactivity Experimenter bias Floor and ceiling effects (range effects) Identifying potential problems

Demand characteristics Characteristics of the study that may give away the purpose of the experiment May influence how the participants behave in the study Examples: Experiment title: The effects of horror movies on mood Obvious manipulation: Having participants see lists of words and pictures and then later testing to see if pictures or words are remembered better Biased or leading questions: Don’t you think it’s bad to murder unborn children? Demand characteristics

Reactivity Knowing that you are being measured Just being in an experimental setting, people don’t always respond the way that they “normally” would. Cooperative Defensive Non-cooperative Cooperative “You seem like a nice person: I’ll help you get the right results” Defensive “I don’t want to look stupid/evil. I’ll do what a smart/good person is expected to do (rather than what I normally would do).” Noncooperative “This experiment is annoying. Let me screw up the results.” Reactivity

Experimenter Bias Experimenter bias (expectancy effects) The experimenter may influence the results (intentionally and unintentionally) E.g., Clever Hans One solution is to keep the experimenter (as well as the participants) “blind” as to what conditions are being tested Experimenter Bias

Range effects Floor: A value below which a response cannot be made As a result the effects of your IV (if there are indeed any) can’t be seen. Imagine a task that is so difficult, that none of your participants can do it. Ceiling: When the dependent variable reaches a level that cannot be exceeded So while there may be an effect of the IV, that effect can’t be seen because everybody has “maxed out” Imagine a task that is so easy, that everybody scores a 100% To avoid floor and ceiling effects you want to pick levels of your IV that result in middle level performance in your DV Range effects

Variables Independent variables (explanatory) Dependent variables (response) Extraneous variables Control variables Random variables Confound variables Variables

The variables that are measured by the experimenter They are “dependent” on the independent variables (if there is a relationship between the IV and DV as the hypothesis predicts). Dependent Variables

Choosing your dependent variable How to measure your your construct: Can the participant provide self-report? Introspection – specially trained observers of their own thought processes, method fell out of favor in early 1900’s Rating scales – strongly agree - agree - undecided - disagree - strongly disagree Is the dependent variable directly observable? Choice/decision Is the dependent variable indirectly observable? Physiological measures (e.g. GSR, heart rate) Behavioral measures (e.g. speed, accuracy) Choosing your dependent variable

Measuring your dependent variables Scales of measurement Errors in measurement Measuring your dependent variables

Measuring your dependent variables Scales of measurement Errors in measurement Measuring your dependent variables

Measuring your dependent variables Scales of measurement - the correspondence between the numbers representing the properties that we’re measuring The scale that you use will (partially) determine what kinds of statistical analyses you can perform Measuring your dependent variables

Scales of measurement Categorical variables (qualitative) Nominal scale Ordinal scale Quantitative variables Interval scale Ratio scale Scales of measurement

Nominal Scale: Consists of a set of categories that have different names. Label and categorize observations, Do not make any quantitative distinctions between observations. Example: Eye color: blue, green, brown, hazel Scales of measurement

Scales of measurement Categorical variables (qualitative) Nominal scale Ordinal scale Quantitative variables Interval scale Ratio scale Categories Scales of measurement

Ordinal Scale: Consists of a set of categories that are organized in an ordered sequence. Rank observations in terms of size or magnitude. Example: T-shirt size: XXL XL, Lrg, Med, Small, Scales of measurement

Scales of measurement Categorical variables Quantitative variables Nominal scale Ordinal scale Quantitative variables Interval scale Ratio scale Categories Categories with order Scales of measurement

Interval Scale: Consists of ordered categories where all of the categories are intervals of exactly the same size. Example: Fahrenheit temperature scale With an interval scale, equal differences between numbers on the scale reflect equal differences in magnitude. However, Ratios of magnitudes are not meaningful. 20º 40º 20º increase The amount of temperature increase is the same 60º 80º 20º increase 40º “Not Twice as hot” 20º Scales of measurement

Scales of measurement Categorical variables Quantitative variables Nominal scale Ordinal scale Quantitative variables Interval scale Ratio scale Categories Categories with order Ordered Categories of same size Scales of measurement

Ratio scale: An interval scale with the additional feature of an absolute zero point. Ratios of numbers DO reflect ratios of magnitude. It is easy to get ratio and interval scales confused Example: Measuring your height with playing cards Scales of measurement

Ratio scale 8 cards high Scales of measurement

Interval scale 5 cards high Scales of measurement

Scales of measurement Ratio scale Interval scale 8 cards high 0 cards high means ‘as tall as the table’ 0 cards high means ‘no height’ Scales of measurement

Scales of measurement Categorical variables Quantitative variables Nominal scale Ordinal scale Quantitative variables Interval scale Ratio scale Categories Categories with order Ordered Categories of same size Ordered Categories of same size with zero point “Best” Scale? Given a choice, usually prefer highest level of measurement possible Scales of measurement

Measuring your dependent variables Scales of measurement Errors in measurement Reliability & Validity Sampling error Measuring your dependent variables

Measuring the true score Example: Measuring intelligence? How do we measure the construct? How good is our measure? How does it compare to other measures of the construct? Is it a self-consistent measure? Internet IQ tests: Are they valid? (The Guardian Nov. 2013) Measuring the true score

Errors in measurement In search of the “true score” Reliability Do you get the same value with multiple measurements? Consistency – getting roughly the same results under similar conditions Validity Does your measure really measure the construct? Is there bias in our measurement? (systematic error) Errors in measurement

Bull’s eye = the “true score” for the construct e.g., a person’s Intelligence Dart Throw = a measurement e.g., trying to measure that person’s Intelligence Dartboard analogy

Dartboard analogy unreliable invalid - The dots are spread out Reliability = consistency Validity = measuring what is intended Bull’s eye = the “true score” for the construct Measurement error Estimate of true score Estimate of true score = average of all of the measurements unreliable invalid - The dots are spread out - The & are different Dartboard analogy

Dartboard analogy reliable valid unreliable invalid reliable invalid Bull’s eye = the “true score” Reliability = consistency Validity = measuring what is intended biased reliable valid unreliable invalid reliable invalid Dartboard analogy

Errors in measurement In search of the “true score” Reliability Do you get the same value with multiple measurements? Consistency – getting roughly the same results under similar conditions Validity Does your measure really measure the construct? Is there bias in our measurement? (systematic error) Errors in measurement

Reliability True score + measurement error A reliable measure will have a small amount of error Multiple “kinds” of reliability Test-retest Internal consistency Inter-rater reliability Reliability

Reliability Test-restest reliability Test the same participants more than once Measurement from the same person at two different times Should be consistent across different administrations Reliable Unreliable Reliability

Reliability Internal consistency reliability Multiple items testing the same construct Extent to which scores on the items of a measure correlate with each other Cronbach’s alpha (α) Split-half reliability Correlation of score on one half of the measure with the other half (randomly determined) Reliability

Reliability Inter-rater reliability At least 2 raters observe behavior Extent to which raters agree in their observations Are the raters consistent? Requires some training in judgment Not very funny Funny 5:00 4:56 Reliability

Errors in measurement In search of the “true score” Reliability Do you get the same value with multiple measurements? Consistency – getting roughly the same results under similar conditions Validity Does your measure really measure the construct? Is there bias in our measurement? (systematic error) Errors in measurement

Does your measure really measure what it is supposed to measure (the construct)? There are many “kinds” of validity Validity

Many kinds of Validity VALIDITY CONSTRUCT INTERNAL EXTERNAL FACE CRITERION- ORIENTED PREDICTIVE CONVERGENT CONCURRENT DISCRIMINANT Many kinds of Validity

Many kinds of Validity VALIDITY CONSTRUCT INTERNAL EXTERNAL FACE CRITERION- ORIENTED PREDICTIVE CONVERGENT CONCURRENT DISCRIMINANT Many kinds of Validity

At the surface level, does it look as if the measure is testing the construct? “This guy seems smart to me, and he got a high score on my IQ measure.” Face Validity

Usually requires multiple studies, a large body of evidence that supports the claim that the measure really tests the construct Construct Validity

Internal Validity The precision of the results Did the change in the DV result from the changes in the IV or does it come from something else? Internal Validity

Threats to internal validity Experimenter bias & reactivity History – an event happens the experiment Maturation – participants get older (and other changes) Selection – nonrandom selection may lead to biases Mortality (attrition) – participants drop out or can’t continue Regression toward the mean – extreme performance is often followed by performance closer to the mean The SI cover jinx | Madden Curse Threats to internal validity

Are experiments “real life” behavioral situations, or does the process of control put too much limitation on the “way things really work?” Example: Measuring driving while distracted External Validity

External Validity Variable representativeness Relevant variables for the behavior studied along which the sample may vary Subject representativeness Characteristics of sample and target population along these relevant variables Setting representativeness Ecological validity - are the properties of the research setting similar to those outside the lab External Validity

Measuring your dependent variables Scales of measurement Errors in measurement Reliability & Validity Sampling error Measuring your dependent variables

Sampling Errors in measurement Sampling error Population Everybody that the research is targeted to be about μ = 71 Sampling error The subset of the population that actually participates in the research X = 68 Sample Sampling

Sampling Population Sampling to make data collection manageable Inferential statistics used to generalize back Sampling to make data collection manageable Sample Allows us to quantify the Sampling error Sampling

Sampling Goals of “good” sampling: Key tool: Random selection Maximize Representativeness: To what extent do the characteristics of those in the sample reflect those in the population Reduce Bias: A systematic difference between those in the sample and those in the population Key tool: Random selection Sampling

Sampling Methods Probability sampling Non-probability sampling Simple random sampling Cluster sampling Stratified sampling Non-probability sampling Quota sampling Convenience sampling Have some element of random selection Random element is removed. Susceptible to biased selection There are advantages and disadvantages to each of these methods I recommend that you check out table 6.1 in the textbook pp 127-128 Here is a nice video (~5 mins.) reviewing some of the sampling techniques (Statistics Learning Centre) Sampling Methods

Simple random sampling Every individual has a equal and independent chance of being selected from the population Simple random sampling

Cluster sampling Step 1: Identify clusters Step 2: randomly select some clusters Step 3: randomly select from each selected cluster Cluster sampling

Step 1: Identify distribution of subgroups (strata) in population Step 2: randomly select from each group so that your sample distribution matches the population distribution 8/40 = 20% 20/40 = 50% 12/40 = 30% Stratified sampling

Quota sampling Step 1: identify the specific subgroups (strata) Step 2: take from each group until desired number of individuals (not using random selection) Quota sampling

Use the participants who are easy to get (e. g Use the participants who are easy to get (e.g., volunteer sign-up sheets, using a group that you already have access to, etc.) Convenience sampling

Use the participants who are easy to get (e. g Use the participants who are easy to get (e.g., volunteer sign-up sheets, using a group that you already have access to, etc.) College student bias (World of Psychology Blog) Western Culture bias “Who are the people studied in behavioral science research? A recent analysis of the top journals in six sub-disciplines of psychology from 2003 to 2007 revealed that 68% of subjects came from the United States, and a full 96% of subjects were from Western industrialized countries, specifically those in North America and Europe, as well as Australia and Israel (Arnett 2008). The make-up of these samples appears to largely reflect the country of residence of the authors, as 73% of first authors were at American universities, and 99% were at universities in Western countries. This means that 96% of psychological samples come from countries with only 12% of the world's population.” Henrich, J. Heine, S.J., & Norenzayan, A. (2010). The weirdest people in the world? (free access). Behavioral and Brain Sciences, 33(2-3), 61-83. Convenience sampling

Variables Independent variables Dependent variables Measurement Scales of measurement Errors in measurement Extraneous variables Control variables Random variables Confound variables Variables

Extraneous Variables Control variables Holding things constant - Controls for excessive random variability Random variables – may freely vary, to spread variability equally across all experimental conditions Randomization A procedure that assures that each level of an extraneous variable has an equal chance of occurring in all conditions of observation. Confound variables Variables that haven’t been accounted for (manipulated, measured, randomized, controlled) that can impact changes in the dependent variable(s) Co-varys with both the dependent AND an independent variable Extraneous Variables

Colors and words Divide into two groups: men women Instructions: Read aloud the COLOR that the words are presented in. When done raise your hand. Women first. Men please close your eyes. Okay ready? Colors and words

Blue Green Red Purple Yellow List 1

Okay, now it is the men’s turn. Remember the instructions: Read aloud the COLOR that the words are presented in. When done raise your hand. Okay ready?

Blue Green Red Purple Yellow List 2

So why the difference between the results for men versus women? Is this support for a theory that proposes: “Women are good color identifiers, men are not” Why or why not? Let’s look at the two lists. Our results

List 2 Men List 1 Women Blue Green Red Purple Yellow Blue Green Red Matched Mis-Matched

What resulted in the performance difference? Our manipulated independent variable (men vs. women) The other variable match/mis-match? Because the two variables are perfectly correlated we can’t tell This is the problem with confounds Our question of interest Blue Green Red Purple Yellow Blue Green Red Purple Yellow IV DV Confound Co-vary together ? Confound that we can’t rule out

What DIDN’T result in the performance difference? Extraneous variables Control # of words on the list The actual words that were printed Random Age of the men and women in the groups Majors, class level, seating in classroom,… These are not confounds, because they don’t co-vary with the IV Blue Green Red Purple Yellow Blue Green Red Purple Yellow

Experimental Control Our goal: To test the possibility of a systematic relationship between the variability in our IV and how that affects the variability of our DV. Control is used to: Minimize excessive variability To reduce the potential of confounds (systematic variability not part of the research design) Experimental Control

Experimental Control Our goal: To test the possibility of a systematic relationship between the variability in our IV and how that affects the variability of our DV. T = NRexp + NRother + R Nonrandom (NR) Variability NRexp: Manipulated independent variables (IV) Our hypothesis: the IV will result in changes in the DV NRother: extraneous variables (EV) which covary with IV Condfounds Random (R) Variability Imprecision in measurement (DV) Randomly varying extraneous variables (EV) Experimental Control

Experimental Control: Weight analogy Variability in a simple experiment: T = NRexp + NRother + R Absence of the treatment (NRexp = 0) Treatment group Control group “perfect experiment” - no confounds (NRother = 0) R NR exp other R NR other Experimental Control: Weight analogy

Experimental Control: Weight analogy Variability in a simple experiment: T = NRexp + NRother + R Control group Treatment group NR exp R R Difference Detector Our experiment is a “difference detector” Experimental Control: Weight analogy

Experimental Control: Weight analogy If there is an effect of the treatment then NRexp will ≠ 0 Control group Treatment group R NR exp R Difference Detector Our experiment can detect the effect of the treatment Experimental Control: Weight analogy

Things making detection difficult Potential Problems Confounding Excessive random variability Difference Detector Things making detection difficult

Potential Problems Confound If an EV co-varies with IV, then NRother component of data will be present, and may lead to misattribution of effect to IV IV DV Co-vary together EV Potential Problems

Confounding Confound R NR R Hard to detect the effect of NRexp because the effect looks like it could be from NRexp but could be due to the NRother R NR R other NR exp Difference Detector Experiment can detect an effect, but can’t tell where it is from Confounding

Confound Hard to detect the effect of NRexp because the effect looks like it could be from NRexp but could be due to the NRother These two situations look the same R NR exp other Difference Detector R R NR other Difference Detector There is an effect of the IV There is not an effect of the IV Confounding

Potential Problems Excessive random variability If experimental control procedures are not applied Then R component of data will be excessively large, and may make NRexp undetectable Potential Problems

Excessive random variability If R is large relative to NRexp then detecting a difference may be difficult R R NR exp Difference Detector Experiment can’t detect the effect of the treatment Excessive random variability

Reduced random variability But if we reduce the size of NRother and R relative to NRexp then detecting gets easier So try to minimize this by using good measures of DV, good manipulations of IV, etc. R NR exp R Difference Detector Our experiment can detect the effect of the treatment Reduced random variability

Controlling Variability How do we introduce control? Methods of Experimental Control Constancy/Randomization Comparison Production Controlling Variability

Methods of Controlling Variability Constancy/Randomization If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate Control variable: hold it constant Random variable: let it vary randomly across all of the experimental conditions Methods of Controlling Variability

Methods of Controlling Variability Comparison An experiment always makes a comparison, so it must have at least two groups Sometimes there are control groups This is often the absence of the treatment Training group No training (Control) group Without control groups if is harder to see what is really happening in the experiment It is easier to be swayed by plausibility or inappropriate comparisons Useful for eliminating potential confounds Methods of Controlling Variability

Methods of Controlling Variability Comparison An experiment always makes a comparison, so it must have at least two groups Sometimes there are control groups This is often the absence of the treatment Sometimes there are a range of values of the IV 1 week of Training group 2 weeks of Training group 3 weeks of Training group Methods of Controlling Variability

Methods of Controlling Variability Production The experimenter selects the specific values of the Independent Variables 1 week of Training group 2 weeks of Training group 3 weeks of Training group Need to do this carefully Suppose that you don’t find a difference in the DV across your different groups Is this because the IV and DV aren’t related? Or is it because your levels of IV weren’t different enough Methods of Controlling Variability

So far we’ve covered a lot of the about details experiments generally Now let’s consider some specific experimental designs. Some bad (but common) designs Some good designs 1 Factor, two levels 1 Factor, multi-levels Between & within factors Factorial (more than 1 factor) Experimental designs

Poorly designed experiments Bad design example 1: Does standing close to somebody cause them to move? “hmm… that’s an empirical question. Let’s see what happens if …” So you stand closely to people and see how long before they move Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”) Poorly designed experiments

Poorly designed experiments Bad design example 2: Testing the effectiveness of a stop smoking relaxation program The participants choose which group (relaxation or no program) to be in Poorly designed experiments

Poorly designed experiments Bad design example 2: Non-equivalent control groups Self Assignment Independent Variable Dependent Variable Training group Measure participants No training (Control) group Random Assignment Measure Problem: selection bias for the two groups, need to do random assignment to groups Poorly designed experiments

Poorly designed experiments Bad design example 3: Does a relaxation program decrease the urge to smoke? Pretest desire level – give relaxation program – posttest desire to smoke Poorly designed experiments

Poorly designed experiments Bad design example 3: One group pretest-posttest design Dependent Variable Independent Variable Dependent Variable participants Pre-test Training group Post-test Measure Pre-test No Training group Post-test Measure Add another factor Problems include: history, maturation, testing, and more Poorly designed experiments

1 factor - 2 levels Good design example How does anxiety level affect test performance? Two groups take the same test Grp1 (moderate anxiety group): 5 min lecture on the importance of good grades for success Grp2 (low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough 1 Factor (Independent variable), two levels Basically you want to compare two treatments (conditions) The statistics are pretty easy, a t-test 1 factor - 2 levels

1 factor - 2 levels Good design example How does anxiety level affect test performance? participants Low Moderate Test Random Assignment Anxiety Dependent Variable 1 factor - 2 levels

1 factor - 2 levels Good design example How does anxiety level affect test performance? One factor Use a t-test to see if these points are statistically different low moderate test performance anxiety anxiety Two levels low moderate 60 80 Observed difference between conditions T-test = Difference expected by chance 1 factor - 2 levels

1 factor - 2 levels Advantages: Simple, relatively easy to interpret the results Is the independent variable worth studying? If no effect, then usually don’t bother with a more complex design Sometimes two levels is all you need One theory predicts one pattern and another predicts a different pattern 1 factor - 2 levels

1 factor - 2 levels Interpolation Disadvantages: “True” shape of the function is hard to see Interpolation and Extrapolation are not a good idea low moderate test performance anxiety What happens within of the ranges that you test? Interpolation 1 factor - 2 levels

1 factor - 2 levels Extrapolation Disadvantages: “True” shape of the function is hard to see Interpolation and Extrapolation are not a good idea Extrapolation low moderate test performance anxiety What happens outside of the ranges that you test? high 1 factor - 2 levels

1 Factor - multilevel experiments For more complex theories you will typically need more complex designs (more than two levels of one IV) 1 factor - more than two levels Basically you want to compare more than two conditions The statistics are a little more difficult, an ANOVA (Analysis of Variance) 1 Factor - multilevel experiments

1 Factor - multilevel experiments Good design example (similar to earlier ex.) How does anxiety level affect test performance? Two groups take the same test Grp1 (moderate anxiety group): 5 min lecture on the importance of good grades for success Grp2 (low anxiety group): 5 min lecture on how good grades don’t matter, just trying is good enough Grp3 (high anxiety group): 5 min lecture on how the students must pass this test to pass the course 1 Factor - multilevel experiments

1 factor - 3 levels participants Low Moderate Test Random Assignment Anxiety Dependent Variable High 1 factor - 3 levels

1 Factor - multilevel experiments low mod test performance anxiety anxiety low mod high high 80 60 60 1 Factor - multilevel experiments

1 Factor - multilevel experiments Advantages Gives a better picture of the relationship (function) Generally, the more levels you have, the less you have to worry about your range of the independent variable 1 Factor - multilevel experiments

Relationship between Anxiety and Performance low moderate test performance anxiety 2 levels high low mod test performance anxiety 3 levels Relationship between Anxiety and Performance

1 Factor - multilevel experiments Disadvantages Needs more resources (participants and/or stimuli) Requires more complex statistical analysis (analysis of variance and pair-wise comparisons) 1 Factor - multilevel experiments

Pair-wise comparisons The ANOVA just tells you that not all of the groups are equal. If this is your conclusion (you get a “significant ANOVA”) then you should do further tests to see where the differences are High vs. Low High vs. Moderate Low vs. Moderate Pair-wise comparisons