1 Chapter 12: Data Collection Research Participants –Populations & Samples –Sampling & sample size Research Procedures –Effective v. ineffective strategies –“debugging” –Data collection session & Post exp interview Using the Internet to Collect Data –Forms of internet data collection –Advantages & Disadvantages –Validity of web-based research –Ethical issues Summary

2 Chapter 12: Data Collection Research Participants: Populations & Samples (External v. internal validity): –Target pop > Operational def > study pop > sampling >Research sample –Construct and operational definition fit ( e.g. “depression) Define: deviant worker; abused child, unemployed person Sampling –Probability Random, Stratified, Systematic, Cluster –Non-probability (convenience sampling) College Ss, “mall intercept” interviews –Purposive sampling (school funding, org department) –Snowball sampling (some examples?)

3 Chapter 12: Data Collection: Sample Size Statistical Power = 1- β (type II error) depends upon : α (Alpha level = Type I error rate) effect size (in population: small, med, large) sample size (larger = more power) Determining sample size 1.Critical effect size (Kraemer & Thiemann, ’87) Important in applied research (why? Give an example) 2.Average effect size (meta-analysis) 3.Average effect size (guess, Lipsey, ’90) What α ?.05 or.5? (see table 12-2, p. 298) Note: tear page out, save it.

4 Chapter 12: Data Collection Research Procedures Effective settings –Coherence –Simplicity –Psychological involvement Experimental realism (Ps are involved, take it seriously) Mundane realism (similar to real world, natural setting) Was Milgram’s work high/low on either, both? Consistency –Effective instructions Clarity, attention, understanding –“Debugging” the proc Have others read proc, rehearse it, pilot study

5 Data Collection: Session Similar to party planning –Be presentable –Treat guests with respect; be polite –Be considerate Post experimental interview –Ethical function: (inspect your self for red flags) –Educational function: –Methodological function: (manipulation check) –Discovery function: new questions emerge –Conducting the interview: (ind v. group debriefing) Probe for demand char Explain true purpose Ask for improvements RAs (train them, get them involved)

6 Data Collection: Internet Forms of internet collection (3) – , website, chat room Advantages –Economy, availability –Access (beyond the college student), purposive samples –Potential for increasing internal validity Reduce demand, e-effects, social desirability, real volunteers –Public research methods The ultimate in public research (Reips, 2000)? –What do you think?

7 Internet Data Collection Disadvantages –Sampling biases –Are the Ps representative? Limits on experimental stimuli and research modes Lack of control over data collection and environment –Who’s looking over their shoulder, are they distracted? –Duplicates? How to control for? P attrition (mortality) Sabotage

8 Internet Data: Recruiting Ps solicitation (listserves) Announcements Listing on research directories Search engines Control over recruitment Expiration dates Website design

9 Internet data collection Validity of web-based data –Not a big problem Psychometric properties hold up Survey data about the same But be careful about knowledge testing Ethical issues –More difficult to ensure informed consent, monitor Ps –Be careful with feedback (pos and neg) –Informed consent Legal age, identify of P –Monitoring for harm –Privacy (confidentiality v. anonymity) In org survey –Debriefing

10 Chapter 12: Data Collection Summary Research Participants –Populations & Samples –Sampling & sample size Research Procedures –Effective v. ineffective strategies –“debugging” –Data collection session & Post exp interview Using the Internet to Collect Data –Forms of internet data collection –Advantages & Disadvantages –Validity of web-based research –Ethical issues