Research Design RCPT 436 Research & Technology Applications
Units of Analysis Individuals: students, participants, patients Groups: classes, families, gangs Organizations: universities, churches, recreation departments Social Artifacts: books, paintings, songs, editorials, buildings
A word of warning… On average, students from Springfield score higher on the SATs then students from Shelbyville. What can we conclude about Lisa Simpson? NOTHING! An Ecological Fallacy results when you make conclusions about individuals based upon aggregate data.
Variables Definition: discrete phenomenon that can be observed in at least two mutually-exclusive categories –Dependent variable –Independent variables –Intervening variables
Watching a Snickers Commercial Buying a Snickers from Vending Machine Hunger Dependen t Variable Intervenin g Variable Independen t Variable
What other (independent) variables could explain our purchase of a Snickers?
Defining the scope Purpose statement Significance statement Theory or conceptual roadmap
Which Hypothesis is Which? Hypothesis –Null hypothesis –Non-directional hypothesis –Alternative hypothesis Men will perform better than women on standardized tests. The average scores for men and women will be different. There will be no statistical difference between test scores
Internal Validity Determine whether a given program is working, if it meets standards, or how it compares to other programs Properly demonstrates a causal relationship between two variables
External Validity Ability to generalize findings to the “real world” or beyond the sample Results should be reproducible across different experimental settings
Threats to External Validity 1.Selection bias 2.Experimental settings 3.Testing 4.Multiple treatment interference 5.Inadequate operational definition 6.Hawthorne effect 7.Selection interacting with any extraneous variables
Experimental Design 1.Random selection of sample 2.Pretest 3.Random assignment to experimental and control groups 4.Post-test
Variations Quasi-experimental design – study participants not randomly assigned to control or experimental group Pre-experimental – no control group Non-experimental – individuals are observed or surveyed
Qualitative Designs Ethnography: holistic understanding (participant observation) Case study: in depth study of a single phenomenon (visionary) Content analysis: written, visual, or recorded documents (profanity on TV) Historical study: studying the past (primary or secondary data; be skeptical) Rigor determined by truthfulness, applicability, and consistency