Reliability of Selection Measures

Reliability Defined The degree of dependability, consistency, or stability of scores on measures used in selection

Error of Measurement Obtained Score  True Score  Error Score

Errors of Measurement -Physical characteristics = Higher reliability (HR) -Psychological attributes = Lower reliability (LR) ================================== Large amounts of measurement error = LR Small amounts of measurement Error = HR

Methods of Estimating Reliability  Examines two sets of measure  If scores from the two measurements are similar, reliability will be raised  If scores from the two measurements are dissimilar, reliability will be lowered

Methods of Estimating Reliability Statistical procedures calculated reliability coefficients range from 0.00 to 1.00 Higher the coefficient, the less the measurement error, and the higher the reliability estimate Lower the coefficient, the higher the measurement error, and the lower the reliability estimate

Methods of Estimating Reliability Selecting a method:  Dependability of data collected today be reflective of the same person in the future  To what degree do evaluations vary from one another  Accuracy of scores on measuring the true ability  Dependability of an assessment of a measure at a given moment

Methods of Estimating Reliability Test-Retest Same measure is used to collect data from the same person at two different times The higher the coefficient, the greater the approximation to the true score: closer to 1.00 The lower the coefficient, the greater likelihood of error: closer to.00

Methods of Estimating Reliability Test-Retest – Sources of error Memory Learning “True change”

Methods of Estimating Reliability Parallel or Equivalent Forms Does not use the same measure twice but will instead use equivalent versions of the measure  Each has same number of questions  Each has same level of difficulty  Averages of scores are the same

Methods of Estimating Reliability Internal Consistency Estimates How similar are different parts of the measure (e.g. different questions on a test) in what they measure?  Split-half reliability  Kuder-Richardson Formula 20  Coefficient Alpha (α) – most commonly reported

Factors Influencing The Reliability of a Measure  Method of Estimating Reliability  Individual Differences Among Respondents  Length of a Measure Test Question Difficulty Administration of a measure

Standard Error of Measurement Define: Used to obtain an estimate of error for an individual The higher the standard error, the more error present in the measure

Standard Error of Measurement Useful applications:  Forces us to think of scores on a measure not as exact points  Aid in decision making in which only one individual is involved  Determine the magnitude of how scores differ significantly from one another