Research Tools and Techniques The Research Process: Step 6 (Research Design for Experiments Part B) Lecture 23

Lecture Topics Covered Previously in the Last Lecture Introduction to Experimental Designs Control and Manipulation of Independent Variable Techniques for Controlling Exogenous Variables Internal and External Validity

What we are going to Cover in this Lecture Factors Effecting the Internal Validity of Experiments

THE RESEARCH PROCESS Observation (4). Theoretical Framework (7). (6).   THE RESEARCH PROCESS (1). Observation  The Broad Problem Area   (2). Preliminary Data Gathering Interviews and Library Search (3). Problem Definition  (4). Theoretical Framework Variables Identification (5) Generation of Hypothesis (6). Scientific Research Design (7). Data Collection and Analysis (8) Deduction (9). Report Writing (10). Report Presentation (11). Managerial Decision Making  

THE ELEMENTS OF RESEARCH DESIGN 2. Type of Investigation Establishing: Causal Relationship or Co-relational 1. Purpose of Study Exploratory Descriptive Hypothesis Testing Case Study 3. Extent of Researcher Interference Minimal Moderate Excessive 4. Study Setting Contrived Non-Contrived 10. Test Application Feel for Data Goodness of Data Hypotheses 6.Unit of Analysis (Population to be studied) Individuals Dyads Groups Organizations Machines etc. 7. Sampling Design Probability Non-probability Sample Size (n) 8. Time Horizon One-Shot (Cross-Sectional) Longitudinal 9. Data Collection Methods Observation Interviews Questionnaire Physical Measurement 5. Measurement & Measures Operational Definition Scaling Categorizing Coding Problem Statement

FACTORS AFFECTING INTERNAL VALIDITY Even the best designed lab studies could be influenced by factors that might affect the internal validity of the lab experiment. These possible confounding threats pose a threat to internal validity. The seven major threats to internal validity are: HISTORY EFFECTS MATURATION EFFECTS TESTING EFFECTS INSTRUMENTATION EFFECTS SELECTION EFFECTS STATISTICAL REGRESSION EFFECTS MORTALITY EFFECTS

History Effects Certain events or factors that would have an impact on the independent variable - dependent variable relationship might unexpectedly occur while the experiment is in progress, and this history of events would confound the cause and effect relationship between the two variables, thus affecting the internal validity. Example Independent variable Dependent variable Sales Sales Promotion Dairy Farmer’s Advertisement Uncontrolled variable

Maturation Effects Cause and effect inferences can also be contaminated by the effects of the passage of time - another uncontrollable variable. Such contamination is called maturation effects. Examples of maturation effects processes could include growing older, getting tired, feeling hungry, and getting bored. Maturation effects

and doing the job faster Example Independent variable Dependent variable Enhanced Technology Efficiency Increase Gaining experience and doing the job faster Maturation effects

Testing Effects Frequently, to test the effects of a treatment, subjects are given pretest (e.g., a short questionnaire eliciting their feelings and attitudes). That is, first a measure of the dependent variable is taken (the pretest), then the treatment is given, and after that a second test, called the posttest, is administered. The difference between the posttest and pretest is then attributed to the treatment. However, the very fact that respondents were exposed to the pretest might influence their response on the posttest, which would adversely impact on internal validity.

Example of Testing Effect: We have to minus this score from posttest values of Group A and B to adjust for testing effects.

Instrumentation Effects: Instrumentation effects are another source of threat to internal validity. These effects might arise because of a change in the measurement instrument between pretest and posttest, and not because of the treatment’s differential impact at the end.

Selection Bias Effects: The threat to internal validity could also come from improper or unmatched selection of subjects for the experimental and control groups.

Statistical Regression Effects: Statistical regression occurs when members chosen for the experimental group have extreme scores on the dependent variable to begin with.

Mortality Another confounding factor on the cause and effect relationship is the mortality or attrition of the members in the experimental or control group or both, as the experiment progresses. When the group composition changes overtime across the groups, comparison between the groups become difficult, because those who dropped out of the experiment may confound the result.

How to Avoid the Confounding Factors The shorter the time span of the experiments, the less the chances of encountering the history, maturation, and mortality effects. Experiments lasting an hour or two do not usually encounter many of these problems. It is only when experiments take place over an extended period (e.g., several months) that the possibility of encountering more of the confounding factors increases.

Summary Factors Effecting the Internal Validity of Experiments