Assessment Procedures for Counselors and Helping Professionals, 7e © 2010 Pearson Education, Inc. All rights reserved. Chapter 6 Validity

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Definitions Validity: the claims and decisions made on the basis of assessment results are sound, meaningful, and useful for the intended purpose of the results. Validity: adequacy and appropriateness of the uses of assessment results. Construct: scientifically developed concepts, ideas, or hypotheses that are used to describe or explain behavior

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Important Points Not all test results are valid for all individuals at all times. If a test is given to someone of the wrong age or background to the normative sample, the resulting decisions made from that information is invalid.

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Threats to Validity Construct Underrepresentation: a test is too narrow and fails to include important dimensions of the construct (e.g., only measuring articulation when evaluating oral expression). Construct-Irrelevant Variance: a test is too broad and includes more dimensions than those in the construct (e.g., using reading cues in a listening comprehension test)

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Threats to Validity, Cont. Within factor threats: items are poorly made, too few items to measure construct, answers can be provided without having to know the construct (e.g., can use “test taking strategies” to do well). Administration and scoring procedure errors (e.g., not following standardization, not using objective scoring)

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Threats to Validity, Cont. Test-taker characteristics: some variables (other than the construct in question) within the test taker that effect results (e.g., test anxiety, 2 nd language issues, emotional problems, rapport, etc.) Inappropriate test group: using test on individuals that it was not designed or normed for. (e.g., giving a test designed for a 2 nd grader to a 4 th grader)

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Validity and Reliability Reliability is a necessary but insufficient condition for validity. Meaning… if your scores are different each time you give a test, you aren’t measuring the same thing each time. Validity is not required for reliability. Meaning… you can consistently measure the wrong thing

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Sources of Validity Evidence DescriptionProcedures for Assessing Validity ContentEvidence based on the representativeness of test content (test items) and response processes to the content domain Table of specifications Expert judges Analysis of response processes Criterion-RelatedEvidence based on the relationship between test scores and external variables Correlation b/w test scores & other scores on similar test. Correlation b/w test scores & some other data about same concept. ConstructEvidence based on the appropriateness of inferences drawn from test scores as they relate to a particular construct Evidence of homogeneity Convergent and discriminant validity evidence Group differentiation studies Factor analysis

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Validity Coefficient >.50Very High High Moderate/Acceptable <.20Low/ Unacceptable

Assessment Procedures for Counselors and Helping Professionals, 7e Drummond/Jones © 2010 Pearson Education, Inc. All rights reserved Consequences of Testing in Validation Process If the decisions made from test data are based off of invalid results, then the decisions themselves are invalid. Test score bias and unfair use of test scores can greatly influence validity of the use of test scores.