Cronbach & Meehl Construct Validity in Psychological Tests.

Slides:



Advertisements
Similar presentations
Test Development.
Advertisements

Agenda Levels of measurement Measurement reliability Measurement validity Some examples Need for Cognition Horn-honking.
Chapter 8 Flashcards.
Measurement Concepts Operational Definition: is the definition of a variable in terms of the actual procedures used by the researcher to measure and/or.
Psychology Practical (Year 2) PS2001 Correlation and other topics.
 Degree to which inferences made using data are justified or supported by evidence  Some types of validity ◦ Criterion-related ◦ Content ◦ Construct.
Cal State Northridge Psy 427 Andrew Ainsworth PhD
Taking Stock Of Measurement. Basics Of Measurement Measurement: Assignment of number to objects or events according to specific rules. Conceptual variables:
VALIDITY AND RELIABILITY
MGT-491 QUANTITATIVE ANALYSIS AND RESEARCH FOR MANAGEMENT
Measurement Reliability and Validity
Correlation & Regression Chapter 15. Correlation statistical technique that is used to measure and describe a relationship between two variables (X and.
RESEARCH METHODS Lecture 18
VALIDITY.
Concept of Measurement
Concept of Reliability and Validity. Learning Objectives  Discuss the fundamentals of measurement  Understand the relationship between Reliability and.
Correlational Designs
Chapter 7 Evaluating What a Test Really Measures
Validity Lecture Overview Overview of the concept Different types of validity Threats to validity and strategies for handling them Examples of validity.
CORRELATIO NAL RESEARCH METHOD. The researcher wanted to determine if there is a significant relationship between the nursing personnel characteristics.
1 Evaluating Psychological Tests. 2 Psychological testing Suffers a credibility problem within the eyes of general public Two main problems –Tests used.
Multivariate Analysis Techniques
Relationships Among Variables
Measurement and Data Quality
Reliability, Validity, & Scaling
© 2013 Cengage Learning. Outline  Types of Cross-Cultural Research  Method validation studies  Indigenous cultural studies  Cross-cultural comparisons.
Measurement in Exercise and Sport Psychology Research EPHE 348.
Ch 6 Validity of Instrument
Instrumentation.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 14 Measurement and Data Quality.
Unanswered Questions in Typical Literature Review 1. Thoroughness – How thorough was the literature search? – Did it include a computer search and a hand.
What are Psychological Theories?. Definition  Psychological Theories are general principles that describe and explain behaviour and mental processes.
L 1 Chapter 12 Correlational Designs EDUC 640 Dr. William M. Bauer.
Chapter 4: Test administration. z scores Standard score expressed in terms of standard deviation units which indicates distance raw score is from mean.
Validity. Face Validity  The extent to which items on a test appear to be meaningful and relevant to the construct being measured.
Reliability & Validity
Assessing the Quality of Research
6. Evaluation of measuring tools: validity Psychometrics. 2012/13. Group A (English)
Measurement Models: Exploratory and Confirmatory Factor Analysis James G. Anderson, Ph.D. Purdue University.
Appraisal and Its Application to Counseling COUN 550 Saint Joseph College For Class # 3 Copyright © 2005 by R. Halstead. All rights reserved.
Copyright © 2008 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Assessing Measurement Quality in Quantitative Studies.
MEASUREMENT. MeasurementThe assignment of numbers to observed phenomena according to certain rules. Rules of CorrespondenceDefines measurement in a given.
MOI UNIVERSITY SCHOOL OF BUSINESS AND ECONOMICS CONCEPT MEASUREMENT, SCALING, VALIDITY AND RELIABILITY BY MUGAMBI G.K. M’NCHEBERE EMBA NAIROBI RESEARCH.
Validity and Item Analysis Chapter 4.  Concerns what instrument measures and how well it does so  Not something instrument “has” or “does not have”
JS Mrunalini Lecturer RAKMHSU Data Collection Considerations: Validity, Reliability, Generalizability, and Ethics.
Technical Adequacy of Tests Dr. Julie Esparza Brown SPED 512: Diagnostic Assessment.
Psychologist use statistics for 2 things
Chapter 6 - Standardized Measurement and Assessment
©2005, Pearson Education/Prentice Hall CHAPTER 6 Nonexperimental Strategies.
TEST SCORES INTERPRETATION - is a process of assigning meaning and usefulness to the scores obtained from classroom test. - This is necessary because.
How Psychologists Do Research Chapter 2. How Psychologists Do Research What makes psychological research scientific? Research Methods Descriptive studies.
Measurement Chapter 6. Measuring Variables Measurement Classifying units of analysis by categories to represent variable concepts.
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 11 Measurement and Data Quality.
5. Evaluation of measuring tools: reliability Psychometrics. 2011/12. Group A (English)
Ch. 5 Measurement Concepts.
Lecture 5 Validity and Reliability
Questions What are the sources of error in measurement?
Concept of Test Validity
Evaluation of measuring tools: validity
CHAPTER 5 MEASUREMENT CONCEPTS © 2007 The McGraw-Hill Companies, Inc.
Validity and Reliability
Reliability & Validity
پرسشنامه کارگاه.
Making Sense of Advanced Statistical Procedures in Research Articles
Reliability and Validity of Measurement
VALIDITY Ceren Çınar.
Evaluation of measuring tools: reliability
RESEARCH METHODS Lecture 18
Chapter 8 VALIDITY AND RELIABILITY
Chapter 3: How Standardized Test….
Presentation transcript:

Cronbach & Meehl Construct Validity in Psychological Tests

Validation of psychological tests has not yet been adequately conceptualized, as the APA Committee on psychological Tests learned when it undertook ( ) to specify what qualities should be investigated before a test is published. Page 281

Criterion-Oriented Validity Predictive ValidityConcurrent Validity

Criterion-Oriented Validity

Content Validity Hey, is this astronomy? Why are they talking about a universe?

In content validation, acceptance of the universe of content as defining the variable to be measured is essential.

Construct validity must be investigated whenever no criterion or universe of content is accepted as entirely adequate to define the quality to be measured. Determining what psychological constructs account for test performance is desirable for almost any test.

Thus, although the MMPI was originally established on the basis of empirical discrimination between patient groups and so-called normals (concurrent validity), continuing research has tried to provide a basis for describing the personality associated with each score pattern. Such interpretations permit the clinician to predict performance with respect to criteria which have not yet been employed in empirical validation studies

MMPI-2 now uses uniform T-scores

Use the Method of Contrasted Groups to create a Scale of Biological Sex (distinguishing the Boys from the Girls) Suppose our test has 30 items, Interpret a score of 10, 20, 30

Response Sets Social Desirability, positive impression management Yeah Saying, Acquiescence, directional bias (The Challenge of Response Sets) Response Scale Extremity, Extreme checking style (Ipsatization)

Jackson, D.N. & Messick, S.J. (1958) Content and style in personality measurement. Psychological Bulletin, 55, Rorer, L.G. (1965) The great response style myth. Psychological Bulletin, 63, Campbell, D.T. Siegman, C.R., & Rees, M.R. (1967) Direction of wording effects in the relationships between scales. Psychological Bulletin, 68, Block, J. (1965) The Challenge of Response Sets. NY: Appleton- Century-Crofts. Hamilton, D. (1968) Extreme Response Set, Psychological Bulletin, Psychological Bulletin, 69, PDF PDF

The Bender Visual Motor Gestalt Test or simply the Bender-Gestalt test is a psychological test first developed by child neuropsychiatrist Lauretta Bender. The test is used to evaluate "visual-motor maturity", to screen for developmental disorders, or to assess neurological function or brain damage.psychological testneuropsychiatristLauretta Bender

"If a new test is demonstrated to predict the scores on an older, well-established test, then an evaluation of the predictive power of the older test may be used for the new one." But accurate inferences are possible only if the two tests correlate so highly that there is negligible reliable variance in either test, independent of the other. Where the correspondence is less close, one must either retain all the separate variables operationally defined or embark on construct validation. Review worksheet on Bounds on Correlations

X Y Z

A VERY Brief Introduction to Factor Analysis

Common Factor V1 V2 V3 Observed Variables Unique Factors A VERY Brief Introduction to Factor Analysis

Specific Criteria Used Temporarily: The "Bootstraps" Effect Even when a test is constructed on the basis of a specific criterion, it may ultimately be judged to have greater construct validity than the criterion. We start with a vague concept which we associate with certain observations. We then discover empirically that these observations covary with some other observation which possesses greater reliability or is more intimately correlated with relevant experimental changes than is the original measure, or both.

Baron von Munchhausen Pull yourself up by your bootstraps "The Surprising Adventures of Baron Munchausen" (by Rudolph Erich Raspe, published in 1895) The real Baron Munchausen

Examples of the Bootstrap Effect Measurement of Temperature The Binet Simon Scale

EXPERIMENTATION TO INVESTIGATE CONSTRUCT VALIDITY Validation Procedures

Group differences. (Thurstone and Chave: Attitudes towards the church) Correlation matrices and factor analysis. Studies of internal structure.

Item-test correlations and certain reliability formulas describe internal consistency. It is unwise to list uninterpreted data of this sort under the heading "validity" in test manuals, as some authors have done. High internal consistency may lower validity. Only if the underlying theory of the trait being measured calls for high item intercorrelations do the correlations support construct validity. Negative item-test correlations may support construct validity, provided that the items with negative correlations are believed irrelevant to the postulated construct and serve as suppressor variables (31, p ; 44).

[,1] [,2] [,3] [1,] [2,] [3,] Correlations among predictor variables Suppressor Variables rxy = , ,

How do we calculate Standardized Regression weights?

Warning: Prepare to be Shocked! This is an extreme example. b = , ,

Y X2 X1 S

Studies of change over occasions. The stability of test scores ("retest reliability," Cattell's "N-technique") may be relevant to construct validation. Whether a high degree of stability is encouraging or discouraging for the proposed interpretation depends upon the theory defining the construct. N technique: The study of change over occasions

The Numerical Estimate of Construct Validity There is an understandable tendency to seek a "construct validity coefficient," A numerical statement of the degree of construct validity would be a statement of the proportion of the test score variance that is attributable to the construct variable.

If "creativity" is defined as something independent of knowledge, then a correlation of.40 between a presumed test of creativity and a test of arithmetic knowledge would indicate that at least 16 per cent of the reliable test variance is irrelevant to creativity as defined.