Chapter 12 Experimental Designs

Chapter Objectives understand the role and scope of experimental research in business distinguish between causal and correlational analysis explain the difference between laboratory and field experiments explain the following terms: extraneous variables, manipulation, experimental and control groups, treatment effect, matching and randomisation discuss the seven possible threats to internal validity in experimental designs describe the different types of experimental designs explain the role of simulation in experimental research describe the ethical issues involved in experimental research

Laboratory Experiment Field Experiment Experimental Designs Laboratory Experiment Field Experiment Cause: - Effect relationships established by: 1. Manipulating treatments 2. Controlling for external or exogenous variables Manipulation of Treatment: Example: Three different teaching methods given to three different groups of students Straight lectures to 10 students simulation only, to another 10 students Both lectures and simulations to 10 other students Assess which results in greatest amount of learning

Group Treatment Final Marks Simulation alone is ineffective. Experimental Group 1 Lectures 70% Experimental Group 2 Simulation 40% Experimental Group 3 Lecture & Simulation 100% Control Group No treatment 50% Simulation alone is ineffective. Lectures are more effective than no treatment at all. Both lectures and simulation are extremely effective. Cause: - Effect relationship can be established because of: Controls for age, etc. through either randomisation or matching of groups Because of an additional control group

Control of Exogenous Variables through; Random assignment of members to various groups Matched groups Control groups Example: Different treatments may have different effects on people with differing interests, ages, expertise,etc. So, a) randomly assign members to different treatment groups. The differences will be randomly distributed. Systematic bias will be reduced. b) match the different groups as closely as possible in terms of age, interest, expertise, etc. c) have an additional control group of students who ar not exposed to any of the three treatments, and see how they learn and compare.

Controlled Variables Variables that might affect the Cause - Effect relationship among the IVs and DV, and hence need to be controlled. Example: 1. Age 2. Education levels 3. Length of Service in Organisation Might affect the relationship between job characteristics and job satisfaction

Uncontrolled Variables Variables or phenomena that occur unexpectedly and can confound the results. Example: Advertising Purchasing (IV) (DV) Age Life style Sudden Unemployment (Uncontrolled Variable) (Controlled Variables)

Lab Experiements can have tight controls and hence the validity of cause – Effect findings is high – ie., they have high internal validity. But their generalisability to real life is low, because of their tight controls – ie., their external validity is low. Field Experiments (eg, different incentive plans (treatment0 in work organisations for assessing effect on productivity, have high external validity or generalisability (because they represent the actual situations), but have low internal validity (ie., cause – effect relationships are contaminated because of no controls.)

Cause and effect relationship after randomisation Groups Treatment Treatment effect (% increase in production over pre-piece rate system) Experimental group 1 $1.00 per piece 10 Experimental group 2 $1.50 per piece 15 Experimental group 3 $2.00 per piece 20 Control group (no treatment) Old hourly rate

FACTORS AFFECTING INTERNAL VALIDITY HISTORY EFFECTS MATURATION EFFECTS TESTING EFFECTS INSTRUMENTATION EFFECTS SELECTION BIAS STATISTICAL REGRESSION MORTALITY

History effects in experimental design

Maturation effects on the cause and effect relationship

Pre-test and post-test experimental group design Treatment Post-test Experimental group O1 X O2 Treatment effect = (O2 - O1)

Post-test only with experimental and control groups Treatment Outcome Experimental group X O1 Control group O2 Treatment effect = (O1 - O2)

Pre-test and post-test experimental and control groups Treatment Post-test Experimental group O1 X O2 Control Group O3 O4 Treatment effect = [(O2 - O1) – (O4 - O3)]

Solomon four-group design Pre-test Treatment Post-test 1.Treatment O1 X O2 2.Control O3 O4 3. Experimental O5 4. Control O6 Treatment effect (E) could be judged by: E 1 = (O2 - O1) E 2 = (O2 - O4 ) E 3 = (O5 - O6) E 4 = (O5 - O3 ) E 5 = (O2 - O1) – (O4 - O3 ) If all Es are similar, the cause and effect relationship is highly valid.

Major threats to internal validity in different experimental designs Types of experimental designs Major threats to internal validity 1. Pre-test and post-test with one experimental group only Testing, history, maturation 2. Post-tests only with one experimental and one control group Maturation 3. Pre-test and post-test with one experimental and one control group Mortality 4. Solomon four-group design

Simulation as experimentation

Example of a management flight simulator

Ethical Issues in Experimental Research The following practices are considered unethical: pressuring individuals to participate in experiments through coercion or applying social pressure giving out menial tasks and asking demeaning questions that diminish the subject’s self-respect deceiving subjects by deliberately misleading them as to the true purpose of the research exposing participants to physical or mental stress not allowing subjects to withdraw from the research when they want to

Ethical Issues in Experimental Research (cont’d) using the research results to disadvantage the participants, or for purposes that they would not like not explaining the procedures to be followed in the experiment exposing respondents to hazardous and unsafe environments not debriefing participants fully and accurately after the experiment is over not preserving the confidentiality of the information given by the participants withholding benefits from control groups

Decision points for embarking on an experimental design

A completely randomised design Routes Number of passengers before Treatment Number of passengers after Group 1 of nine routes O1 X1 O2 Group 2 of nine routes X2 O4 Group 3 of nine routes O3 X3 O6

A randomised block design Blocking factor: residential areas Fare reduction Suburbs Crowded urban area Retirement areas 5c X1 7c X2 10c X3 Note that the Xs above indicate only various levels of the blocking factor, and the Os (the number of passengers before and after each treatment at each level) are not shown, although these measures will be taken.

The Latin square design Day of the week Residential area Midweek Weekend Monday/Friday Suburbs X1 X2 X3 Urban Retirement

A 3 * 3 factorial design Luxury Express X1Y1 X2Y1 X3Y1 Type of bus 5c 7c 10c Luxury Express X1Y1 X2Y1 X3Y1 Standard Express X2Y2 X1Y2 X3Y2 Regular X3Y3 X2Y3 X1Y3