1. 6
Scales, Tests, & Indexes

2. 6.1 Foundations of Scales, Tests, and Indexes
Three approaches for measuring a construct:
Scales
Measure abstract concepts, like attitudes
Tests
Measure some ability or knowledge
Indexes
Combines several measures to estimate a more complex construct

3. 6.2 Scales and Scaling Scaling Unidimensional scaling types
Involves the construction of an instrument that associates qualitative constructs with quantitative metric units
Unidimensional scaling types
Thurstone or Equal-Appearing Interval Scaling
Likert or Summative Scaling
Guttman or Cumulative Scaling

4. 6.2a General Issues in Scaling
Figure 6.2 Scaling as the assignment of numbers according to a rule

5. 6.2a Types of Scales Response scale Dichotomous response
A sequential-numerical response format, such as a 1-to-5 rating format
Dichotomous response
Has two possible options (e.g., true/false or yes/no)
Interval response scale
Measured on an interval level, where the size of the interval is meaningful

6. 6.2a Differences Between Scaling and Response Scales
Table 6.1 Differences between scaling and response scales.
Ask the class to identify an example of a construct being measured for each scale and response scale.

7. 6.2b Purposes of Scaling To test a hypothesis
To discover if a construct is unidimensional or multidimensional
As part of exploratory research
To represent a construct as a single score

8. 6.2c Dimensionality Unidimensional Scales Two Dimensional Scales
Figure 6.3 Unidimensional scales: For each construct in the figure (height, thirst, and self-esteem) the measurement is on a single dimension.
Figure 6.4 A two dimensional scale: Here, a single construct is being measured in two dimensions (quantitative and verbal).
Unidimensional Scales
Two Dimensional Scales

9. 6.2c Dimensionality Figure 6.5 A three dimensional scale.
Semantic differential: A scaling method in which an object is assessed by the respondent on a set of bipolar adjective pairs.

10. 6.2d Unidimensional or Multidimensional?
Unidimensional scales are easier to use and understand
Used when:
What you are measuring is unidimensional in reality
If the concept is not unidimensional, this scale will not measure the concept well, and you need to choose a multidimensional approach

11. 6.2d Unidimensional or Multidimensional?
Unidimensional types
Thurstone scaling
Likert scaling
Guttman scaling
Thurstone Scaling: A class of scaling methods (the method of equal appearing intervals, the method of successive intervals, and the method of paired comparisons) that were designed to yield unidimensional, intervallevel, multi-item scales.
Likert or summative scaling: A method of scaling in which the items are assigned interval level scale values and the responses are gathered using an interval-level response format.
Guttman or cumulative scaling: A method of scaling in which the items are assigned scale values that allow them to be placed in a cumulative ordering with respect to the construct being scaled.

12. Thurstone Scaling: The Method of Equal Appearing Intervals
Develop the focus of the scaling project
Generate potential scale items (statements); separate items
Rate the scale items
Compute scale score values for each item
Median, interquartile range
Selecting the final scale items
Administering the scale
Median: The score found at the exact middle or 50th percentile of the set of values. One way to compute the median is to list all scores in numerical order and then locate the score in the center of the sample.
interquartile range: The difference between the 75th (upper quartile) and 25th (lower quartile) percentile scores on the distribution of a variable.
The interquartile range is an estimate of the spread or variability of the measure.

13. Displaying the Median and Interquartile Ranges of the Attitude Data
Figure 6.7 Histogram displaying the median and interquartile ranges of the attitude data.

14. Likert Scaling Define the focus
Generate the items on a 1-5 or 1-7 agree/disagree response scale
Have a group of judges rate the items
Select the items by computing the intercorrelations between all pairs of items
Administer the scale
Forced Choice Response Scale; Reversal Items
Forced-choice response scale: A response scale that does not allow for a neutral or undecided value. By definition, a forced-choice response scale has an even number of response options.
Reversal items: Items on a multi-item scale whose wording is in the opposite direction of the construct of interest. Reversal items must have their scores reversed prior to computing total scale scores.

15. The Employment Self-Esteem Likert Scale
Table 6.2: The Rosenberg scale.

16. Guttman Scaling (Scalogram Analysis)
Define the focus
Develop the items (80-100)
Have a group of judges rate the items
Develop the cumulative scale
Administer the scale
Scalogram analysis: A method of analysis of a set of scale items used when constructing a Guttman or cumulative scale. In scalogram analysis, one attempts to determine the degree to which responses to the set of items allows the items to be ordered cumulatively in one dimension with respect to the construct of interest.

17. Developing a Cumulative Scale with Guttman Scaling
Figure 6.9 Developing a cumulative scale with Guttman scaling.

18. 6.3 Tests
A test is a measure designed to measure a respondent’s knowledge, skill, or performance
The history of testing goes back to BCE
The key for a good test is reliability and validity

19. Example: A Driving Simulator
Figure 6.9 This driving simulator was used as part of a driver-training study for adolescents with attention deficit problems (Fabiano et al., 2011).
Courtesy of Kevin F. Hulme
Tests: A measure designed to measure a respondent’s knowledge, skill, or performance.

20. 6.3a Validity, Reliability, and Test Construction
Factor analysis
Consequential validity
Test-retest reliability
Item analysis
Figure 6.10 What validity issue do you see here?
Factor analysis: A multivariate statistical analysis that uses observed correlations (variability) as input and identifies a fewer number of unobservedvariables, known as factors, that describe the original data more efficiently.
Consequential validity: The approximate truth or falsity of assertions regarding the intended or unintended consequences
of test interpretation and use.
Test-retest reliability: A method of estimating the reliability or consistency of a test or measure by assessing the degree of correlation between two successive administrations.
Item Analysis: The correlation of a particular item with the total subtest or test score.

21. 6.3b Standardized Tests
A method of test construction that uses statistics and a large sample of previously taken tests to “standardize” the measurement
Includes statistics such as:
Mean, median, and mode
Percentiles
Variance and standard deviation
Correlations with other, related tests

22. 6.3c Test Fairness Many factors affect standardized test performance
Consequential validity
Many decision makers (e.g., universities) have begun to either do away with standardized test scores or to combine single test scores with other measures of ability
Does your SAT score fairly reflect your intelligence?

23. 6.3d How to Find A Good Test Test publishers
Primary research literature
Test reviews in academic journals
Buros Center for Testing at the University of Iowa
APA

24. 6.4 Indexes
Index
A quantitative score that measures a construct of interest by applying a formula or a set of rules that combines relevant data

25. 6.4a Some Common Indexes Consumer Price Index (CPI)
Socioeconomic Status (SES)
Duncan Socioeconomic Index (SEI)

26. 6.4b Constructing an Index
Conceptualize the index
Operationalize and measure the components
Develop the rules for calculating the index score
Weighted Index
Weighted Index: A quantitative score that measures a construct of interest by applying a formula or a set of rules that combines relevant data where the data components are weighted differently.

27. Discuss and Debate I Consider standardized tests Are they fair?
What are some factors that may affect a person’s performance?

28. Discuss and Debate II
Have you ever answered questions on a scale in a way that would make you look good?
Some measurements may not be accurate when respondents do not answer truthfully – what can be done?
In research, this is called the social-desirability bias. Allow the class to discuss the ways in which that bias can skew the results of a study.