Measurement in Psychology: Validity Lawrence R. Gordon Psychology Research Methods I

You have a reliable measure… §It gives you a consistent measure of something every time that you measure it §SO WHAT??? §Is that all a measure needs to do?

Validity §NOTE: a scale CAN be reliable but not valid; it CANNOT be unreliable scale, but be valid! §In short, a measure has to measure what it claims to measure (or what you claim it measures) §How can you show others that you are measuring what you say you are measuring? §Example

SAT’s and College Potential §How can we show that SAT’s are actually measuring college potential?

Types of Validity §Content/Face validity l does it look like it is measuring what it should? Examples (Polygraph measures lying through GSR, EEG, heart rate? Reaction time measured with a stopwatch?) l Does the content make sense in measuring the construct? §Content/Sampling Validity l Where did the content come from? Examples (Why do we need to know insane vocabulary words and geometry on the SAT’s? Who wrote those questions?)

Types of Validity §Criterion validity l validity related to an outcome l concurrent validity (Does it relate to other variables the way the theory says it should?) Example (Does the SAT relate to Achievement Tests? IQ?) l predictive validity (Does it predict something related to the construct?) Examples (Does the SAT predict first-year college GPA? Does the SAT predict GRE scores? Does it “postdict” HS class rank?)

Types of Validity §Discriminant validity l Does the measure fail to relate to measures of other constructs? Examples (Does the SAT relate to measures of SES or self-esteem?)

Types of Validity §Construct validity l Does the measure really measure the construct it says it does? l Is the construct (and its operational definition) appropriate for the research? l Examples (Does the SAT measure college potential? Is this an appropriate measure for college potential?)

Old-fashioned Racism §Old-fashioned racism is the overt, blatant expression of racist attitudes §It’s easy to measure - ask questions like l “are you racist?” l “do you like African-Americans?” l “would you want African-Americans to attend the same schools as your children?” §scale has what type of validity?

Old-fashioned Racism (cont.) §Problems with “old-fashioned racism”, l didn’t relate to subsequent voting behavior (what type of validity lacking?) l people were not reporting that they were racist anymore 80% of White Americans were reporting that they were not racist at all (Dovidio & Gaertner, 1991) §Where do we go from here?

Validation of a Measure §Subtle measure of racism §Use of political attitudes rather than direct inquiry (not quite the same thing!) §Modern Racism Scale (McConahay, Hardee, & Batts, 1981) l How much do you agree that: “over the past few years, Blacks have gotten more economically than they deserve”? “discrimination against Blacks is no longer a problem in the United States”?

Validation of a Measure (cont.) §MRS needed to show that it was reliable and valid §How could it show that it was reliable? §How could it show that it was valid?

Modern Racism Scale INTERNAL CONSISTENCY RELIABILITY (Homogeneity) RELIABILITY ANALYSIS - SCALE (ALPHA) N of Cases = Reliability Coefficients N of Items = 7 Alpha =.7834 SPLIT-HALF RELIABILITY (Consistency) RELIABILITY ANALYSIS - SCALE (SPLIT) N of Cases = Reliability Coefficients N of Items = 7 Correlation between forms =.5993 Unequal-length Spearman-Brown = Items in part 1 / 3 Items in part 2

Validation of a Measure (cont.) §Validity of MRS l Face validity? Survey says...

MRS FACE VALIDITY

Validation of a Measure (cont.) §Validity of MRS l Face validity? Survey says... l Criterion validity? Concurrent validity? Predictive validity? –McConahay (1983) - tell me about it...

Validation of a Measure (cont.) §Validity of MRS l Face validity? Survey says... l Criterion validity? Concurrent validity? Predictive validity? –McConahay (1983) - tell me about it... l Discriminant validity?

Validation of a Measure (cont.) §Validity of MRS l Construct validity? §Since its creation, the MRS has been probably the most widely used measure of racism in psychology §What do you think about its continued validity?

McConahay -- Major Result

Construct Validity -- Your Data and Civil Unions §How does the Attitudes to Civil Union scale do? §No reliability info, but similar scales work well §Face validity? §Concurrent validity? Correlation with MRS? §Discriminant validity? Correlation with Soc Desirability §CONSTRUCT which sex more “tolerant”? which political party? which type of hometown? Related to weight? Height? GPA?