Reliability Presentation Test-Retest James Blackwood – AED 615 Fall Semester 2006

Test-Retest Reliability Test-Retest method of determining reliability is accomplished by administering a test to a group Test-Retest method of determining reliability is accomplished by administering a test to a group After a period of time has passed, the same test is re-administered to the same group. After a period of time has passed, the same test is re-administered to the same group. It is also known as stability reliability It is also known as stability reliability It is used in both qualitative and quantitative research (qualitative requires a different technique of analysis) It is used in both qualitative and quantitative research (qualitative requires a different technique of analysis)

Reliability Coefficient 1. After two tests have been administered, a reliability coefficient is calculated to determine the relationship between the two scores obtained. 2. If the same results are obtained from the two tests then the coefficient = The coefficient is influenced by the amount of time that has passed between the administration of the two tests.

Reliability Coefficient Measurement 1. The reliability coefficient is expected to be lower the longer the time interval between the tests due to the possibility of changes in the population taking the test. 2. The shorter the time gap, the higher the correlation; the longer the time gap, the lower the correlation.

Test-Retest Issues There cannot be any measurable change in the construct being measured between the two tests. There cannot be any measurable change in the construct being measured between the two tests. This method will not work when measuring a variable that is not stable in an individual. This method will not work when measuring a variable that is not stable in an individual. Unless the instrument is reliable, relationships with other variables in the study will not be identified. Unless the instrument is reliable, relationships with other variables in the study will not be identified.

Testing.. You can obtain considerably different estimates of reliability depending on the interval between tests. You can obtain considerably different estimates of reliability depending on the interval between tests. For educational research, examination of scores over a two to three month period is sufficient for test-retest reliability verification. For educational research, examination of scores over a two to three month period is sufficient for test-retest reliability verification. The time interval between the two tests should always be reported when using test-retest as a measure of reliability. The time interval between the two tests should always be reported when using test-retest as a measure of reliability.

Test-Retest Equation

Test-Retest Issues 1. Requires twice the data collection 2. Population for the test would need to be willing to repeat the test (impractical) 3. Higher costs due to multiple tests being administered 4. Only works well when practical (better for smaller population rather than a large one) 5. Researcher may not be able to retest the population

Examples of Test-Retest Educational assessment Educational assessment Drug testing Drug testing Testing Measurement Equipment Testing Measurement Equipment Medical Evaluations Medical Evaluations

Research Literature Example American Journal of Agricultural Economics Volume 71 - Number 1 (Feb 1989), pp Test-Retest Reliability of the Contingent Valuation Method: A Comparison of General Population and Visitor Responses John B. Loomis Abstract: The reliability of the contingent valuation method is evaluated by resurveying the same general households and visitors nine months after their original survey. Test-retest correlations on willingness to pay are statistically significant and ranged from.422 for the general population sample to.782 for the visitor sample. Using a paired T-test, there was no statistical difference between an individual's first and second reported willingness to pay. Chow tests comparing the original and resurvey willingness-to-pay functions showed no statistical difference at the.01 level. Reported willingness to pay is reasonably stable over the time period surveyed.

References Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate research in education (6th ed.). New York: Mc-Graw-Hill. Guttman, L. (1946). The test-retest reliability of qualitative data. Psychometrika, 11(2), Abstract retrieved October 15, 2006 from Springer Link database. Loomis, J. B. (1989) Test-retest reliability of the contingent valuation method: A comparison of general population and visitor responses. American Journal of Agricultural Economics, 71(1), Trochim, W. M., (2006). Types of reliability. Research Methods Knowledge Base. Retrieved from