1. Unidimensionality (U): What’s it good for in applied research?
Richard N. Jones, Sc.D.
Hebrew Rehabilitation Center for Aged
Research and Training Institute
Boston, MA
Paul K. Crane M.D., M.P.H.
University of Washington
Seattle, WA
Psychometrics Workshop
September 20-25, 2004

2. Overview of this talk Introduction to concept of Unidimensionality (U)
Appeal to authority
What pioneers and prominent experts and authors and others have to say about U
Addressing the problem of U empirically
Discussion

3. Issues Around Unidimensionality
What does U mean?
Why is U important?
What is the test for?
What is the analysis for?
DIF studies
Item banking
Can U be determined empirically?
What to if you don’t think you have a U test?
How to respond to a reviewer with concerns about U?

4. Other Sources
Gould, S. (1981). The Mismeasure of man. New York: WW Norton & Company.
Lord, F. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
Lord, F., & Novick, M. (1968). Latent traits and item characteristic functions (Chapter 16). In Statistical Theories of Mental Test Scores (pp ). Reading, MA: Addison-Wesley.
Stout, W. (2002). Psychometrics: From practice to theory and back: 15 years of nonparametric multidimensional IRT, DIF/test equity, and skills diagnostic assessment. Psychometrika, 67(4),
McDonald, R. (1999). Test Theory: A Unified Treatment. Mahwah, NJ: Erlbaum.
Thissen, D. (2004) Comments on Developing Tailored Instruments: Item Banking and Computer Adaptive Assessment. Conference Proceedings (Advances in Health Outcomes Measurement). National Institutes of Health. Bethesda, Maryland.

5. Lord and Novick (’68) on U
Actual existence of traits not necessary for IRT
...nowhere is there any implication that traits exist in any physical or physiological sense. It is sufficient that a person behave as if he [or she] were in possession of a certain amount of each of a number of relevant traits and that he [or she] behave as if these amounts determined his [or her] behavior. (p 358)

6. Embertson and Reise (‘00) on U
U IRT models = a single latent trait is sufficient to characterize individual differences, for example
Single common factor
Multiple factors proportionally loading in items
Not appropriate when
Multiple factors load differentially in different items
Persons (groups) differ in factors that load in items (e.g., test-taking strategies, interpretations of items, substantive factors that load in items)
Mathematics ability and reading ability in written math items
Working memory and concentration in certain cognitive tests

7. Stout’s Essential U U is, essentially, local independence
Strong local independence
Probability of responding u is independent of other test item responses, conditional on q
Weak local independence
The conditional covariance (controlling for some trait q) for all test items is 0
Essential Unidimensionality
The sum of the conditional covariance (controlling for some latent trait q) of all pairs of test items from a test approaches 0 as the test length approaches infinity
Stout, W. (2002). Psychometrics: From practice to theory and back: 15 years of nonparametric multidimensional IRT, DIF/test equity, and skills diagnostic assessment. Psychometrika, 67(4),

8. Jones on U Factor Analysis solutions are sample dependent
Think about your data
Use parcels/testlets/composites where appropriate and possible
Understand logical dependencies among test items
Look at model residuals
Use an empirical device to support an argument for U
Not one is universally accepted
Argument for U is sample dependent
Sample of persons
Sample of behaviors
Model misspecification (Data-driven versus a priori latent trait models can lead to different conclusions)

9. McDonald’s Device for U
‘Hierarchical’ Factor analysis
Aka
General-specific factor model
Bi-factor model
Others?
The hierarchical model provides useful evidence of U

10. McDonald argues that evidence for unidimensionality is supported by observing that the loadings of the items in the general factor are larger than the loadings of the in the specific factors.