Emily H. Wughalter, Ed.D. Measurement & Evaluation Spring 2010

 Reliability means the consistency in the measurement from one testing to the next trial.

 Validity means does the measure or the score measure what it purports (is supposed) to measure.

 For a measure to be valid it must be reliable; however, a measure can be reliable without it being valid.

X observed X True X Error

X observed = X true +X Error

 Testing environment should be favorable  Testing environment should be well- organized  Administrator should be competent to run a favorable testing environment  Range of talent  Motivation of the performers

 Good day vs bad day  Learning, forgetting, and fatigue  Length of the test  Test difficulty  Ability of the test to discriminate  Number of performers

 Nature of measurement device  Selection of scoring unit  Precision  Errors in measurement  Number of trials  Recording errors

 Classroom management  Warm-up opportunity

 Test-Retest (Stability Reliability)  Internal Consistency

 Difference or change scores should be avoided because of ceiling and floor effects.  Difference scores are highly unreliable scores

 Objectivity means interrater reliability or consistency from one person (rater) to another.

 A criterion score represents the score that will be used to represent an individual’s performance  The criterion score may be the mean of a series of trials.  The criterion score may be the best score for a series of trials.

 When selecting a criterion measure, whether it is the best score, the mean of all the trials score, or the mean of the best 2 or 3 trials score, a researcher must determine which of the criterion measures represents the most reliable and most valid score.

 Inconsistent scorers  Inconsistent performance  Inconsistent measures  Failure to develop a good set of instructions  Failure to follow testing procedures