Research System Theory Hypotheses Data Verification Theory building Hypothesis generation Measurement issues Research design Sampling issues Statistical analysis Interpretation Presentation of results Generalization Culmination Modification

Research Strategies “All research strategies are seriously flawed…” McGrath, J. E. Martin, J., & Kulka, R. A. (1982). Judgment Calls in Research. Beverly Hills, CA: SAGE Publications Inc.

A Three-Horned Dilemma Different methods have different strengths: Rigor Relevance Generalizability Every research strategy either avoids two of the horns by an uneasy compromise but gets impaled, to the hilt, on the third horn; or it grabs the dilemma boldly by one horn, maximizing on it, but at the same time “sitting down” (with some pain) on the other two horns. (McGrath, 1982: 74)

Dilemmatics: McGrath Taxonomy The study of research choices and tradeoffs There is no perfect study. Research involves tradeoffs Research looks at Actors emitting Behaviors in a Context We want Generalizability from Actors  population Precise measure and control of Behavior Realistic Contexts for observation of actor behavior

J II I IV III Judgment Task Sample Survey Formal Theory Computer Simulation Field Study Field Experiment Lab Experiment Experimental Simulation RELEVANCE GENERALIZABILITY RIGOR

Q I: Field Strategies Field study – No deliberate manipulation; everything is measured – Naturally occurring setting – Example: Survey of QWL in 100 organizations Field experiment – Deliberate manipulation of one or more variables – Naturally occurring setting – Example: Hawthorne studies; Greenberg’s work RELEVANCE

Q II: Experimental Strategies Laboratory experiment – Deliberate manipulation of variables – Contrived setting – Example: Effects of communication channels on team performance, effects of feedback and goal-setting on individual performance Experimental simulation – Deliberate manipulation of variables – Contrived realistic setting – Example: Center for Creative Leadership Looking Glass simulation; Zimbardo prison experiment – RIGOR

Q III: Respondent Strategies Judgment tasks – Emphasis on task/judgment selection, often with a limited number of participants – All variables are measured – Example: “policy capturing” studies, creation of the competing values framework Sample Survey – Emphasis on sample selection – All variables measured – Example: National Survey of Organizations GENERALIZABILITY

Q IV: Theoretical Strategies Formal theory/literature reviews – No actual research participants – Summarize the literature to create new models for testing (inductive process) Computer Simulations – No actual research participants – All outcomes are computer- generated – Example: garbage can decision-making processes, monte carlo studies GENERALIZABILITY

Integrative & Hybrid Strategies Because every methodological choice is flawed or incomplete, you can decrease the effects of the trade-offs by: Using different methods across studies Using multiple methods within a single study Packaging different studies with different methods together

Alternatively, Three Study Types Experimental Quasi-Experimental Correlational or passive observational study (field) Single subject (case study)

Important Concepts: The Building Blocks of Research Methods Independent Variable Dependent Variable Extraneous Variable Hypothesis Experiment

Causal Inference Conditions needed for causality Covariation of cause and effect Temporal precedence (cause must come before effect in time) Control to rule out alternative interpretations True experiments are best suited to infer causality because the include the greatest degree of control Apply MAXIMINICON principle Maximize relevant systematic variance Minimize irrelevant systematic variance Control extraneous sources of variance

What is a TRUE experiment? There must be manipulation Manipulation of a cause results in an effect There must be random assignment to experimental conditions There must be control of extraneous variable

Experiment Advantages high degree of control strong inference of causality measurement of behavior is precise often laboratory experiments can be replicated easily Disadvantages low realism low external validity in general some phenomena cannot be analyzed in a laboratory some variables may have a weaker (or stronger) impact in the lab than they would in a natural environment

Quasi-Experiment Advantages high realism greater external validity moderate degree of control moderate to high inference of causality Disadvantages internal validity may be compromised external validity may be compromised measurement may be imprecise it may be difficult to get people to agree to participate it is often difficult to get access to many field settings

Correlational Field Study Advantages realistic data on a large number of variables can be collected the researcher's impact on the study is often lower allows exploration of contextual effects Disadvantages causality is difficult to assess internal validity may be compromised external validity may be compromised organizations may not agree to participate measurement of variables less precise than lab low response rate common

Single Subject/Case Study Advantages good for low base rate occurrences provides source of rich and descriptive data good for generating new ideas Disadvantages low internal and external validity

Threats to Validity Validity = the confidence we can have that our findings from any study are “true” All research has threats to validity – that is, things that minimize the degree to which we can embrace a particular finding as “true” Many sources of validity threats but two common ones: Research participants Researchers themselves

Threats due to Participants Roles Good subject Faithful subject Negativistic subject Apprehensive subject Role Multiplicity and Conflict Attributes of Participants Comprehension artifact They misunderstand

Threats due to Researcher Attributes of researchers Expectancies Designer, observer, and interpreter effects Data analyst Tester Poor measurement decisions

Week 2 Assignment Describe a research topic that you are interested in Write three hypothesis statements about relationships you might expect Identify what the IVs and DVs are in these relationships If there are mediators or moderators in your hypothesis statements, identify what these are Indentify the type of research strategy you would use to study this research question Use the article you identified last week as your reference