Assessing Student Learning Section 4.1 Introduction to Reliability

Introduction to Reliability What is reliability?

Introduction to Reliability Concept: Dependability

Introduction to Reliability

Introduction to Reliability Concept: Dependability Calculation: Consistency

Introduction to Reliability “the trustworthiness or consistency of a measure, that is, the degree to which a test or other measurement instrument… yield[s] the same results across multiple applications to the same sample…” (Zedeck, 2014, p. 307)

Introduction to Reliability Reliable? Not reliable? Measured by degree?

Introduction to Reliability Assumptions

Introduction to Reliability Assumption #1 Multiple re-administrations of a test produce different results, each of which generates different estimates of [inter-item and test-retest] reliability. 0-1 TEST

Introduction to Reliability Reliability Induction

Introduction to Reliability Assumption #2 Reliability calculations are affected by numerous variables within the measurement process.

Introduction to Reliability “[M]easurement involves much more than the test itself. The entire testing situation and the process that produces test scores must be considered. A measurement procedure… encompasses all aspects of the testing situation, such as the occasion or time of test administration,… the particular selection of test items, the modes of test administration, and the standardized conditions of testing… It includes multiple aspects of the testing process, and it is not simply limited to the test itself. All aspects of the measurement procedure may affect the consistency of scores.” (Myer, 2010, p. 10)

Introduction to Reliability Assumption #3 Observed scores are indirect measures of latent ability.

What’s next?