1. Sampling & Experimental Control Psych 231: Research Methods in Psychology

2. Sampling Population Everybody that the research is targeted to be about The subset of the population that actually participates in the research Sample Errors in measurement Sampling error Why do we do we use sampling methods? Typically don’t have the resources to test everybody, so we test a subset

3. Sampling Sample Inferential statistics used to generalize back Sampling to make data collection manageable Population Allows us to quantify the Sampling error

4. Sampling Goals of “good” sampling: –Maximize Representativeness: –To what extent do the characteristics of those in the sample reflect those in the population –Reduce Bias: –A systematic difference between those in the sample and those in the population Key tool: Random selection

5. Sampling Methods Probability sampling Simple random sampling Systematic sampling Stratified sampling Non-probability sampling Convenience sampling Quota sampling Have some element of random selection Susceptible to biased selection

6. Simple random sampling Every individual has a equal and independent chance of being selected from the population

7. Systematic sampling Selecting every n th person

8. Cluster sampling Step 1: Identify groups (clusters) Step 2: randomly select from each group

9. Convenience sampling Use the participants who are easy to get

10. Quota sampling Step 1: identify the specific subgroups Step 2: take from each group until desired number of individuals

11. Extraneous Variables Control variables Holding things constant - Controls for excessive random variability Random variables – may freely vary, to spread variability equally across all experimental conditions Randomization A procedure that assures that each level of an extraneous variable has an equal chance of occurring in all conditions of observation. Confound variables Variables that haven’t been accounted for (manipulated, measured, randomized, controlled) that can impact changes in the dependent variable(s) Co-varys with both the dependent AND an independent variable

12. Colors and words Divide into two groups: men women Instructions: Read aloud the COLOR that the words are presented in. When done raise your hand. Women first. Men please close your eyes. Okay ready?

13. Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green List 1

14. Okay, now it is the men’s turn. Remember the instructions: Read aloud the COLOR that the words are presented in. When done raise your hand. Okay ready?

15. Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green List 2

16. Our results So why the difference between the results for men versus women? Is this support for a theory that proposes: “Women are good color identifiers, men are not” Why or why not? Let’s look at the two lists.

17. Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green List 2 Men Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green List 1 Women MatchedMis-Matched

18. What resulted in the perfomance difference? Our manipulated independent variable (men vs. women) The other variable match/mis-match? Because the two variables are perfectly correlated we can’t tell This is the problem with confounds Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green Blue Green Red Purple Yellow Green Purple Blue Red Yellow Blue Red Green IV DV Confound Co-vary together

19. Experimental Control Our goal: To test the possibility of a relationship between the variability in our IV and how that affects the variability of our DV. Control is used to minimize excessive variability. To reduce the potential of confounds.

20. Sources of variability (noise) Nonrandom (NR) Variability - systematic variation A. (NR exp ) manipulated independent variables (IV) i. our hypothesis is that changes in the IV will result in changes in the DV Sources of Total (T) Variability: T = NR exp + NR other + R B. (NR other ) extraneous variables (EV) which covary with IV i. Condfounds

21. Sources of variability (noise) Non-systematic variation C. Random (R) Variability Imprecision in manipulation (IV) and/or measurement (DV) Randomly varying extraneous variables (EV) Sources of Total (T) Variability: T = NR exp + NR other + R

22. Sources of variability (noise) Sources of Total (T) Variability: T = NR exp + NR other + R Goal: to reduce R and NR other so that we can detect NR exp. That is, so we can see the changes in the DV that are due to the changes in the independent variable(s).

23. Weight analogy Imagine the different sources of variability as weights R NR exp NR other R NR other Treatment group control group The effect of the treatment

24. Weight analogy If NR other and R are large relative to NR exp then detecting a difference may be difficult R NR exp NR other R NR other Difference Detector

25. Weight analogy But if we reduce the size of NR other and R relative to NR exp then detecting gets easier R NR other R NR exp NR other Difference Detector

26. Things making detection difficult Potential Problems Confounding Excessive random variability Dissimulation Difference Detector

27. Potential Problems Confounding If an EV co-varies with IV, then NR other component of data will be "significantly" large, and may lead to misattribution of effect to IV IV DV EV Co-vary together

28. Confounding R NR exp NR other Hard to detect the effect of NR exp because the effect looks like it could be from NR exp but is really (mostly) due to the NR other R Difference Detector

29. Potential Problems Excessive random variability If experimental control procedures are not applied Then R component of data will be excessively large, and may make NR exp undetectable So try to minimize this by using good measures of DV, good manipulations of IV, etc.

30. Excessive random variability R NR exp NR other NR other R Hard to detect the effect of NR exp Difference Detector

31. Potential Problems Potential problem caused by experimental control Dissimulation If EV which interacts with IV is held constant, then effect of IV is known only for that level of EV, and may lead to overgeneralization of IV effect This is a potential problem that affects the external validity

32. Controlling Variability Methods of Experimental Control Comparison Production Constancy/Randomization

33. Methods of Controlling Variability Comparison An experiment always makes a comparison, so it must have at least two groups Sometimes there are control groups This is typically the absence of the treatment Without control groups if is harder to see what is really happening in the experiment It is easier to be swayed by plausibility or inappropriate comparisons Sometimes there are just a range of values of the IV

34. Methods of Controlling Variability Production The experimenter selects the specific values of the Independent Variables Need to do this carefully Suppose that you don’t find a difference in the DV across your different groups Is this because the IV and DV aren’t related? Or is it because your levels of IV weren’t different enough

35. Methods of Controlling Variability Constancy/Randomization If there is a variable that may be related to the DV that you can’t (or don’t want to) manipulate Control variable: hold it constant Random variable: let it vary randomly across all of the experimental conditions But beware confounds, variables that are related to both the IV and DV but aren’t controlled

36. Experimental designs So far we’ve covered a lot of the about details experiments generally Now let’s consider some specific experimental designs. Some bad designs Some good designs 1 Factor, two levels 1 Factor, multi-levels Factorial (more than 1 factor) Between & within factors

37. Poorly designed experiments Example: Does standing close to somebody cause them to move? So you stand closely to people and see how long before they move Problem: no control group to establish the comparison group (this design is sometimes called “one-shot case study design”)

38. Poorly designed experiments Does a relaxation program decrease the urge to smoke? One group pretest-posttest design Pretest desire level – give relaxation program – posttest desire to smoke

39. Poorly designed experiments One group pretest-posttest design Problems include: history, maturation, testing, instrument decay, statistical regression, and more participantsPre-test Training group Post-test Measure Independent Variable Dependent Variable

40. Poorly designed experiments Example: Smoking example again, but with two groups. The subjects get to choose which group (relaxation or no program) to be in Non-equivalent control groups Problem: selection bias for the two groups, need to do random assignment to groups

41. Poorly designed experiments Non-equivalent control groups participants Training group No training (Control) group Measure Self Assignment Independent Variable Dependent Variable

42. “Well designed” experiments Post-test only designs participants Experimental group Control group Measure Random Assignment Independent Variable Dependent Variable

43. “Well designed” experiments Pretest-posttest design participants Experimental group Control group Measure Random Assignment Independent Variable Dependent Variable Measure Dependent Variable