Reliability and Validity Threats to Internal Validity Da Lee Caryl, Fall 2006

Internal Validity Defending against sources of bias arising in research design. When there is lack of internal validity, variables other than the independent(s) being studied may be responsible for part or all of the observed effect on the dependent variable(s).

 An experimenter observes an increase in the probability of salivating after presenting the smell of delicious food  Leads to the conclusion that the smell of food produces salivation.  For this conclusion to be internally valid, the experiment must be designed so that conditions other than the food smell are ruled out as potential causes for salivation.  Does the sight of food along with the smell of food cause salivation? Suppose:

Threats to Internal Validity Selection Experimental Mortality Location Instrumentation Testing History Maturation Attitude of Subjects Regression Implementation

Selection Participants in a study may have different characteristics and those differences may affect the results. Such as: AgeIntelligenceVocabulary StrengthAttitudeFluency MaturityFluencyEthnicity CoordinationSpeed Socioeconomic status

Experimental Mortality Loss of subjects from the study due to: Illness Family relocation Requirements of other activities Absent during collection of data or fail to complete tests

Location Place in which data is collected, or an intervention is carried out, may influence the results.

Instrumentation Inconsistent use of the measurement instrument. Instrument Decay – instrument changes in some way Data Collector Characteristics – affects the nature of the data they obtain. Such as gender, age, ethnicity, age, language patterns, etc. Data Collector Bias – unconsciously influence the outcome of the data

Testing Pretests may influence the result of the posttest. A group may perform better on a posttest because the pretest primed them to perform better.

History Occurrence of events that could alter the outcome or the results of the study Previous – events that occurred prior to the study Concurrent – happening during the study

Maturation Any changes that occur in the subjects during the course of the study that are not part of the study and that might affect the results of the study; such as changes due to aging and experience.

Attitude of Subjects Subjects opinion and participation can influence the outcome. Observing or studying subjects can affect their responses a.k.a. Hawthorne effect. Subjects receiving experimental treatment may perform better due to “receiving” treatment. Subjects in the control group may perform more poorly than the treatment group.

Regression Groups that are chosen due to performance characteristics, either high or low, will on the average score closer to the mean on subsequent testing regardless of what transpires during the experiment. Example: A group of students who score low on a mathematics test are given additional help. Six weeks later they are given an exam similar problems on another test, their average score has improved. Is it due to the additional help or other influences?

Implementation Personal bias in favor of one method or another. Preference for the method may account for better performance by the subjects. Individuals assigned to implement different methods, differ in ways related to the outcome.

Methods to Minimize Threats  Standardization of the conditions under which the research study is carried out will help minimize threats from history and instrumentation.  Obtain as much information as possible about the participants in the research study minimizes threats from mortality and selection.  Obtain as much information as possible about the procedural details of the research study, for example, where and when the study occurs, minimizing threats from history and instrumentation.  Choose an appropriate research design which can help control most other threats.

Journal of Agriculture Education Miller, Greg & Pilcher, Carol(2002). Can Selected Learning Strategies Influence the Success of Adult Distance Learners in Agriculture? Journal of Agricultural Education, 43, Retrieved October 15, 2006 from the World Wide Web: 34.pdf

Bibliography Fraenkel, J.R. & Wallen, N.E. (2006). Internal Validity. How to design and evaluate research in education (6 th ed.,pp ). New York: McGraw Hill. Martin, Wendy (1997). Single Group Threats to Internal Validity. Retrieved October 15, 2006 from the World Wide Web: /intval1.html. Trochim, William (2006). Internal Validity. Retrieved October 15, 2006 from the World Wide Web: