Measurement Validity

Validity Are you measuring what you are supposed to measure? Fit between “reality” and “measurement” Fit between “concept” and “operational definition” Accuracy of measurement in a theoretical sense

Validity is … Relative to the purpose of testing How confident can we be that measure indicates a person’s true score on a trait? Matter of degree, rather than whether or not Have you ever heard students complain that questions on a test didn’t seem to address the material covered in class? That’s an issue of lack of validity.

Valid and Reliable A good measurement Measures what it should measure in a consistent way

Reliable but Invalid Your measurement is consistent, but not measuring what it is supposed to measure

Empirical (Criterion-related) Types of Validity Empirical (Criterion-related) Judgmental Face validity Content validity Construct validity Predictive Concurrent Convergent Discriminant

Face Validity Appears to measure what it was designed to measure. Most subjective measure of validity Weakest measure of validity Based on judgement

Content validity Does the measurement include all relevant factors and exclude irrelevant factors? Judge to what extent the content of measure is representative of facets of topic being measured? Do the questions cover the range of behaviors/ attitudes considered to be part of what you are assessing? E.g. Knowledge of Local Culture - language, customs, history - include items from all three to have good content validity

Construct validity Extent to which instrument measures the theoretical construct it is designed to measure. (e.g., intelligence, depression, self-esteem) Questionnaires results agree w predictions based on theory Extent to which instrument measure a characteristic that cannot be directly observed, but must be inferred from patterns in people’s behavior (e.g., motivation, creativity)

Empirical (Criterion-related) Validity Compare scores with one or more external variables, criteria… Correlate with related measure.

Criterion-related validity continued Examine validity of the measure by comparing to external criterion Predictive - of future ability Concurrent - ability to measure present standing

Predictive Validity - Ability of test to predict some future behavior of future ability e.g., SAT and future college performance measured by (operationalized) GPA Correlate survey’s results with some behavior that would be expected to occur, given the results E.g, Questionnaire on political attitudes would have predictive validity if results correctly predicted election outcomes.

Concurrent - ability to measure present standing e.g. correlate salespeople’s scores on a sales aptitude test w/ the $ amount of products they sell Correlate w another measure of same topic at same time.

Convergent Validity Results similar to / correlate well with those from different measure on same topic. (e.g., high or low shyness and behavior in social situations) (e.g., Satisfaction with life scale and 5-item Life satisfaction measure)

Discriminant Validity Differentiates the construct from others that may be similar. The new measure and measures of different, unrelated constructs correlate poorly. Reading ability and IQ.

When a measure is not related to other variables that it theoretically should not be related to, this measure is said to have _______ validity. A. Face B. Reliable C. Convergent D. Discriminant

A psychologist studied dependency by measuring how many times children in school asked an adult for help. A critic argued that asking for help is not a good measure of dependency and said, asking for help shows resourcefulness and initiative. The critic in this situation was disputing the ________ validity of the dependency measure. A. Face B. Construct C. Measurement error D. Reliability

You are constructing a test of knowledge of research methods. You use your textbook for this class to devise 100 true-false questions on various aspects of research methods. Without conducting any research, you can be most certain of the ________ validity of this test. A. Construct C. Criterion B. Face D. Divergent

When a measure relates consistently to other measures of the same concept it shows _______ validity. A. Convergent B. Divergent C. Face D. Internal