1. PSY 614 Instructor: Emily Bullock Yowell, Ph.D.
Validity
PSY 614
Instructor: Emily Bullock Yowell, Ph.D.

2. Validity
Validity: how well does a test measure what is purports to measure
Validation: process in which the test developer and test user may play a role to gather and evaluate validity evidence

3. Types of Validity Face Validity Content Validity Construct Validity
Criterion-Related Validity

4. Face Validity
Face validity: the extent to which a test measures what it appears to measure
Has received little “respect” in the study of validity
Likely more of an issue to the test taker than the test user or developer

5. Content Validity
Content Validity: describes a judgment concerning how adequately a test samples behavior representative of the universe of behavior the test was designed to sample
There is no exact statistical measure of content validity. It is typically determined by expert judgment of test content.

6. Determining Content Validity
Typical procedure (pg158 of text)
Describe content domain
Determine areas of the content domain that are measured by each test item
Compare the structure of the test with the structure of the content domain
Lawshe technique of determining content validity
Content validity in employment settings
What type of information does a measure of content validity not seem to regularly take into account?

7. Construct Validity
Construct validity: judgment about the appropriateness of inferences drawn from test scores regarding individual standings on a variable called a “construct.”
Construct: represent ideas constructed by scientists to help summarize a group of related phenomena or objects (see text pg 163)
Two essential properties
Abstract summaries of some regularity in nature
Related to or connected with concrete, observable entities or events

8. Determining Construct Validity
Evidence of homogeneity: The test is homogeneous, measuring a single construct
Test score increase or decrease as a function of age or the passage of time as theoretically predicted
Test scores obtained subsequent to some event or to the mere passage of time differ from pretest scores as theoretically predicted
Method of contrasted groups: Test scores obtained by people from distinct groups vary as predicted by theory
Test scores correlate with scores on other tests in accordance with what would be predicted from a theory
◙ Convergent validity ◙ Factor Analysis
◙ Discriminate validity

9. Criterion-Related Validity
Criterion-Related Validity: a judgment regarding how adequately a test score can be used to infer an individual's most probably standing on some measure of interest
Criterion: the measure of interest; the standard against which a test or a test score is evaluated
Criterion Contamination: a situation where the criterion measure itself has been based, at least in part, on predictor measures

10. Two Types of Criterion-Related Validity
Concurrent Validity: test scores are obtained at about the same time that the criterion measures are obtained
Predictive Validity: test scores obtained at one time and the criterion measure obtained a future time after some intervening event has taken place

11. More on Criterion-Related Validity
Validity coefficient
Incremental validity: degree to which an additional predictor explains something about the criterion measure not explained by predictors already in use

12. More on Criterion-Related Validity—Decision Theory
Base Rate: extent to which a particular attribute exists in the population
Hit Rate: proportion of people a test accurately identifies as possessing or exhibiting a particular attribute
Miss Rate: proportion of people the test fails to identify as having or not having a particular attribute
False Positive: miss wherein the test predicted that the test taker did posses the particular attribute
False Negative: miss wherein the test predicted that the test taker did not possess the particular attribute