1. Chapter7 & 8 Experimental Design & Measurement of variables Adnan Khurshid

2. Experiments An experiment is defined as manipulating (changing values/situations) one or more independent variables to see how the dependent variable's) is/are affected, while also controlling the affects of additional extraneous variables.

3. Experimentation Experimental design fall into to categories 1. Lab Experiments 2. Field Experiments

4. Types of Experiments Two broad classes: Laboratory experiments: those in which the independent variable is manipulated and measures of the dependent variable are taken in a contrived, artificial setting for the purpose of controlling the many possible extraneous variables that may affect the dependent variable Field experiments: those in which the independent variables are manipulated and measurements of the dependent variable are made on test units in their natural setting

5. Experimentation Controlling the Nuisance Variables 1. Matching Group 2. Randomization

6. Experimentation Validity: The correctness or truthfulness of an inference Internal Validity: Refers to the accuracy of the inference that the IV caused the effect observed in the DV The observed change in the dependent variable is, in fact, due to the independent variable (internal validity)

7. Experimentation Field Experiments An experiment done in the natural environment in which the work goes on as usual, but the treatment is given to one or more group. In the field experiments it may not possible to control all the nuisance variables because members cannot be either randomly assigned to the group. Any cause and effect relationship found in these conditions would have wider generalizability to the other similar production settings.

8. Experimentation Threats to Internal Validity The Eight Classic Threats History Maturation Testing Instrumentation Statistical Regression Selection Subject Mortality Selection Interactions

9. Experimentation History Factors occurring external to the research situation that may appreciably influence the dependent variable, For example, if researchers measure hyperactivity in the morning for the control group and in the afternoon for the experimental group, the time of day could produce group differences that are unrelated to the independent variable (medication). Sales Promotion ---------------------- Sales Dairy Farmers Advertisement

10. Experimentation Maturation Factors occurring within research subjects over time that account for changes in the dependent variable, e.g., ill health, fatigue, depression. Independent VariableDependent variable Enhanced Technology --Efficiency Increases Getting Experience and doing Job faster

11. Experimentation Testing Testing threat is a threat to internal validity produced by a previous administration of the same test or other measure. For example, if rats practiced navigating a milk maze in a pretest condition and then performed the same task in the posttest condition after receiving a drug, learning could reduce the time required to complete the maze, independent of drug effects.

12. Experimentation Instrumentation Instrumentation threat is a threat to internal validity produced by changes in the measurement instrument itself. For example, if a scale did not return to zero after each subject was weighed, group differences in weight could be due to changes in the scale that are unrelated to the independent variable (exercise program).

13. Experimentation Statistical Regression Statistical regression threat is a threat to internal validity that can occur when subjects are assigned to conditions on the basis of extreme scores on a test. During retest, the scores of extreme scorers tend to regress toward the mean even without treatment. For example, students with extremely high scores in one basketball game may obtain less extreme scores during a second game without any real change in ability Conceptually, the initial extremely high basket ball score was raised by measurement error (representing the variability across games). When this changed randomly during the next game, high scores were no longer boosted as much as before. This resulted in a regression to the mean.

14. Experimentation Selection Selection threat is a threat to internal validity that can occur when nonrandom procedures are used to assign subjects to conditions or when random assignment fails to balance out differences among subjects across the different conditions of the experiment. For example, if a social psychologist assigned the first 20 student volunteers to an experimental condition and the next 20 to a control condition, these groups could differ on subject variables that could affect the dependent variable

15. Experiments Subject Mortality Subject mortality threat is a threat to internal validity produced by differences in dropout rates across the conditions of the experiment. For example, if dropout rates are different across three different psychotherapy conditions, this could create group differences on subject variables that could affect clinical outcome, independent of the independent variable. Thus the morality can also lower the internal validity of the experiment.

16. Experimental Design An experimental design is a procedure for devising an experimental setting such that a change in the dependent variable may be solely attributed to a change in an independent variable. Symbols of an experimental design: O = measurement of a dependent variable X = manipulation, or change, of an independent variable R = random assignment of subjects to experimental and control groups E = experimental effect

17. Experimental Design After-Only Design: X O 1 One-Group, Before-After Design: O 1 X O 2 Before-After with Control Group: Experimental group: O 1 X O 2 Control group: O 3 O 4 Where E = (O 2 – O 1 ) – (O 4 – O 3 )

18. How Valid Are Experiments? An experiment is valid if: the observed change in the dependent variable is, in fact, due to the independent variable (internal validity) if the results of the experiment apply to the “real world” outside the experimental setting (external validity)

19. From Concepts to Observations A concept is a mental image that summarizes a set of similar observations, feelings, or ideas. Operationalization is the process of connecting concepts to observations. The goal is to devise operations that measure the concepts we intend to measure—in other words to achieve measurement validity. Researchers develop an operational definition including: – what is measured – how the indicators are measured – the rules used to assign value to what is observed

20. Types of Indicators Self-report (surveys, interviews, etc.) –Report on the past (retrospective) –Report on the present –Predict the future Direct observation Scales/Indices

21. From Operationalization to Levels of Measurement When we know a variable’s level of measurement, we can better understand how cases vary on that variable and so understand more fully what we have measured.

22. Levels of Measurement The nominal (or categorical) level of measurement, which is qualitative, has no mathematical interpretation; The nominal level of measurement identifies variables whose values have no mathematical interpretation; they vary in kind or quality but not in amount. In terms of the variable “Occupation”, you can say that a lawyer is not equal to a nurse, but you cannot say that the “lawyer” is “more occupational” or “less occupational” than the nurse.

23. Ordinal Measures At this level, you specify only the order of the cases, in “greater than” and “less than” distinctions. A common ordinal measure used in social service agencies is client satisfaction. You may ask clients to indicate whether they are “very satisfied,” “satisfied,” dissatisfied,” or “very dissatisfied” with a particular service. A client who responds “very satisfied” is clearly more satisfied than a client who responds “dissatisfied” – but not twice as satisfied or 2 units more satisfied.

24. Interval Measures At the interval level of measurement, numbers represent fixed measurement units but have no absolute zero point. 4 °F-12 °F Your text uses the example of temperatures measured with the Fahrenheit scale. The temperature can definitely go below zero, as indicated in this weather forecast

25. Ratio Measures A ratio level of measurement represents fixed measuring units with an absolute zero point. Zero, in this situation, means absolutely no amount of whatever the variable indicates. On a ratio scale, 10 is two points higher than 8 and is also two times greater than 5. Ratio numbers can be added and subtracted, and because the numbers begin at an absolute zero point, they can also be multiplied and divided (so ratios can be formed between the numbers).

26. Primary Scales of Measurement Scale Numbers Nominal Assigned to Runners OrdinalRank Order of Winners IntervalPerformance Rating on a 0 to 10 Scale RatioTime to Finish, in Seconds 783 Third place Second place First place 13.314.115.2 9.69.18.2

27. Primary Scales of Measurement