2. Internal Validity EAF 410 July 25, 2001

3. Is the observed relationship due to something other than what I studied?

4. Many alternative hypotheses. They are “threats to internal validity.”

5. Validity -- 3 types b Instrument (measurement) b External (generalization) b Internal (ambiguity)

6. Internal Validity b Observed relationship among two or more variables unambiguous. b Relationship not due to “something else.”

7. Threats to Internal Validity

8. Subject Characteristics b Individuals/groups differ in unintended ways agegenderagegender ethnicityintelligenceethnicityintelligence attitudevocabularyattitudevocabulary

9. Mortality b Loss of subjects drop outdrop out don’t complete testsdon’t complete tests don’t complete questionnairesdon’t complete questionnaires

10. b Limits generalizability b Introduces bias different responses from those who dropped outdifferent responses from those who dropped out b What can be done about this threat?

11. Location b Influence of where research is conducted resourcesresources environmentenvironment powerpower b Hold location constant

12. Instrumentation b Instrument Decay nature of instrument changednature of instrument changed –scoring changes –item changes

13. Instrumentation b Data Collector Characteristics characteristics of researcher affects datacharacteristics of researcher affects data –gender –age –ethnicity –language

14. Instrumentation b Data Collector Bias unconsciously distort dataunconsciously distort data –treat groups differently –leading questions –judgement/interpretation Standardize proceduresStandardize procedures Planned ignorancePlanned ignorance

15. Testing b Improvement due to pretest subjects “alerted” to topic by pretestsubjects “alerted” to topic by pretest practice on pretest helpspractice on pretest helps

16. History b Unanticipated, unplanned for events occur during study b Individual/group experiential differences

17. Maturation b Change due to passing of time b Serious threat to studies using pre-post data for intervention group b Control with well-selected comparison group

18. Attitude of Subjects b How subjects view the study b Hawthorne Effect effects because of increased attention to subjectseffects because of increased attention to subjects

19. Regression b Group selected because of low or high scores will, on average, score closer to the average on subsequent tests b Control with equivalent comparison group

20. Implementation b Experimental group treated in unintended ways different individuals implement different treatmentsdifferent individuals implement different treatments personal bias for one treatmentpersonal bias for one treatment

21. Finding a Relationship b Threats to internal validity can reduce likelihood of finding a relationship

22. Minimize the Threats b Standardize study conditions b Obtain subject information b Obtain study details where, when, extraneous eventswhere, when, extraneous events b Choose appropriate design

23. b Threats reduced by planning. b Planning requires collection of information before the study begins. b Often too late to control them once the data have been collected.