Validity Validity: A generic term used to define the degree to which the test measures what it claims to measure.
What is measurement validity? Validity refers to the accuracy of a measure. A measurement is valid when it measures what it is suppose to measure and performs the functions that it purports to perform. Does an indicator accurately measure the variable that it is intended to measure?
VALIDITY The extent to which measures indicate what they are intended to measure. The match between the conceptual definition and the operational definition.
Validity validity types are attempting to assess the degree to which you accurately Translated your construct into the operationalization.
types of translation validity Face Validity Content Validity Criterion-Related Validity Predictive Validity Concurrent Validity Convergent Validity Discriminant Validity
Translational Validity Face/Logical Validity – When the measure is that being tested, i.e. the measure is valid by definition Content Validity – Does the measure adequately sample what is covered?
Face Validity Determined by judgements made by the researcher and based on surface appearance.
Content Validity Determined by expert judgements of the appropriateness of the contents of a measure.
Domain of Meaning Content validity refers to degree that one has representatively sampled from that domain of meaning. This type of validity receives the most attention in construction of achievement & proficiency measures with psychology & education.
Criterion Validity Predictive Validity – Can the measurement be used to predict later reality? Discriminant Validity – Is the measurement dissimilar to other measures that it theoretically should be dissimilar to?
Criterion Validity, 2 Concurrent Validity – Is the measurement correlated with a criteria that is measured at the same time? Convergent Validity – Is the measurement similar to other measures that it theoretically should be similar to?
Construct validity is an assessment of how well you translated your ideas or theories into actual programs or measures. Why is this important? Because when you think about the world or talk about it with others (land of theory) you are using words that represent concepts
What is a criterion? This is the standard by which your measure is being judged or evaluated.
How to determine Criterion Validity Find alternative measurements Make alternative and your measurements at similar times Compare results These methods work for Face/Logical & Content Validity too!
Example: Reading Achievement Test A multiple choice test on reading achievement (which is designed to measure achievement at the time of testing and not designed to predict any future behavior.) might be validated by: – comparing the scores on the test with teachers’ ratings of students’ reading abilities. Teachers’ ratings are the criterion
Predictive Validity This is determined by correlating test scores with criterion scores obtained after examinees have had a chance to perform what is predicted by the test. A measure is validated with reference to future standing on criterion variable.
Example: Employment Test The purpose of an employment test is to predict success on the job. The most appropriate test of its validity is predictive validity - to what extent does the test predict what it is supposed to predict?
Reliability
WHAT IS RELIABILITY? A thermometer is Accurate if the actual body temperature (98.6F) and what is registered on the thermometer (98.6F) are the same A thermometer is Consistent if, for multiple measures of body temperature, what is registered on the thermometer (99F) is the same each time (NOTE: a measure can be consistent and inaccurate, but not the other way around!)
TEST RELIABILITY Remember...reliability deals with measurement error ; thus a “perfectly reliable” measure is 100% error-free. – We never know a person’s “true” score Therefore, all our selection tests will only be estimates of reliability
Statistical Concept of Reliability Real Variance (Observed-Error) Observed Variance (Real+Error) واريانس واقعی ( واريانس مشاهده شده – واريانس خطا ) واريانس مشاهده شده ( واريانس واقعی + واريانس خطا )
Methods of reliability estimation 1-test---retest 2-Parallel forms/alternate forms 3-split half 4-Alfa Kronbach
WHAT FACTORS INFLUENCE TEST RELIABILITY? Test question difficulty Length of a measure The method used to estimate reliability Response format How and by whom test is administered How homogenous or similar the test items are Physical conditions of testing location and physical condition of test taker
Neither Reliable Nor Valid
Reliable And Valid
Reliable But Not Valid
Congratulation
Reliability
WHAT IS RELIABILITY? A thermometer is Accurate if the actual body temperature (98.6F) and what is registered on the thermometer (98.6F) are the same A thermometer is Consistent if, for multiple measures of body temperature, what is registered on the thermometer (99F) is the same each time (NOTE: a measure can be consistent and inaccurate, but not the other way around!)
TEST RELIABILITY Remember...reliability deals with measurement error ; thus a “perfectly reliable” measure is 100% error-free. – We never know a person’s “true” score Therefore, all our selection tests will only be estimates of reliability
Statistical Concept of Reliability Real Variance (Observed-Error) Observed Variance (Real+Error) واريانس واقعی ( واريانس مشاهده شده – واريانس خطا ) واريانس مشاهده شده ( واريانس واقعی + واريانس خطا )
Methods of reliability estimation 1-test---retest 2-Parallel forms/alternate forms 3-split half 4-Alfa Kronbach
WHAT FACTORS INFLUENCE TEST RELIABILITY? Test question difficulty Length of a measure The method used to estimate reliability Response format How and by whom test is administered How homogenous or similar the test items are Physical conditions of testing location and physical condition of test taker
Neither Reliable Nor Valid
Reliable And Valid
Reliable But Not Valid
Congratulation