2. Indexes, Scales & Scaling l Indexes Indexes l General Issues in Scaling General Issues in Scaling General Issues in Scaling l Thurstone Scaling Thurstone Scaling Thurstone Scaling l Likert Scaling Likert Scaling Likert Scaling l Guttman Scaling Guttman Scaling Guttman Scaling lgeneral concept – cumulative scoring, measure of extreme lFor purposes of the course we are not emphasizing this scaling method but you should still review the materials provided.

4. Indexes

5. Index Definition l Quantitative score –Constructed - combining two or more variables. l Index is a composite –Created through a set of rules »summation, average, weighting, complex formula l Measures something (construct) that none of the individual variables do a good job of measuring. l Items are not necessarily correlated

6. Index Examples (1) l DJIA (a.k.a., Dow 30, Dow Jones, or The Dow) –Stock Market index –Gauge of performance of the industrial sector –30 of the largest & widely held “industrial” companies –DJIA = (sum p)/d »p = share price »d = Dow Divisor n Originally # of companies (simple average). n Now <1. Current Dow is much more than the sum of the p

7. Index Examples (2) l CPI (Consumer Price Index) –Average price of consumer goods and services purchased by households –CPI calculation: price and weighting data –Weighted by: »Expenditure as fractions of the total expenditure –General economic indicator (construct) »Considered an index of consumer costs. –Sample » ~ 87 urban areas from ~ 23,000 retail & service establishments. »Rent data from ~50.000 landlords or tenants.

8. Index Examples (3) l SES (Socioeconomic status index) –Measure of an individual's or family’s economic and social position relative to others –Combines income, education, & occupation »Weighted combination –Identify occupation, then can look up the SES score to “determine” the status. –Considered controversial –Status is typically assigned to: »high SES, middle SES, and low SES n Hmmmm? Is this a scale?

9. Index Examples (4 - SES) l SES (Socioeconomic status index) –Status is typically assigned to: »high SES, middle SES, and low SES »Hmmmm? Is this a scale? –Standardized Scale »High SES = upper quartile »Middle SES = Middle two quartiles »Lower SES = bottom quartile

10. Index Construction l Conceptualize index –Construct of interest? What do you want to measure? l Components - Operationalizing the construct –What are the individual components? –How are they measured? l Rules –How will the components be combined? »Adding, average, weighted, complex formula l Validate (like any measure) –If used over time must validate over time

12. General Issues in Scaling

13. What is Scaling? l Young’s (1984) definition –Scaling is the branch of measurement that involves the construction of an instrument that associates qualitative constructs with quantitative metric units.

14. What is a Scale? l Scales are indexes, indexes are not scales l Composed of multiple items like an index –Improves reliability, precision, and scope (many constructs are broad in scope and can’t be measured with a single or few ?s) l To be considered a scale, a scaling process must be employed. l Social science: Scales measure some latent construct not easily directly measured –Beliefs, perceptions, prejudices, opinions, emotional states, personal needs, personality, etc. l Scale items correlate –required (not so for an index)

15. What Is a Scale? The assignment...

16. What Is a Scale? The assignment......of objects... Are you willing to permit immigrants to live in your country? Are you willing to permit immigrants to live in your neighborhood? Would you let your child marry an immigrant?

17. What Is a Scale? The assignment......of objects......to numbers... Are you willing to permit immigrants to live in your country? Are you willing to permit immigrants to live in your neighborhood? Would you let your child marry an immigrant?

18. What Is a Scale? The assignment......of objects......to numbers......according to a rule... Are you willing to permit immigrants to live in your country? Are you willing to permit immigrants to live in your neighborhood? Would you let your child marry an immigrant?

19. What Is a Scale? The assignment......of objects......to numbers......according to a rule... Are you willing to permit immigrants to live in your country? Are you willing to permit immigrants to live in your neighborhood? Would you let your child marry an immigrant? Cumulative

20. Scaling and the Rule l Scaling is the method for determining the rule l Involves procedures done independent of the respondent in order to come up with a numerical value for the object. l A response scale (e.g., true-false, 1-5 agree scale, etc…) is not scaling. –A response scale is just a way to collect data from people using an instrument »Attaches a response to an object or statement »No consistent rule is used –Compare Scale & Response Scale (next slide)

21. Scale Versus Response Scale Scale Response Scale

22. Scale versus Response Scale Results from a process Scale Response Scale

23. Scale Versus Response Scale Results from a process Is used to collect the response for an item Scale Response Scale

24. Scale Versus Response Scale Results from a process. Each item on scale has a scale value. Is used to collect the response for an item. Scale Response Scale

25. Scale Versus Response Scale Results from a process. Each item on scale has a scale value. Is used to collect the response for an item. Item not associated with a scale value. Scale Response Scale

26. Scale Versus Response Scale Results from a process. Each item on scale has a scale value. Is used to collect the response for an item. Item not associated with a scale value. Refers to a set of items. Scale Response Scale

27. Scale Versus Response Scale Results from a process. Each item on scale has a scale value. Is used to collect the response for an item. Item not associated with a scale value. Refers to a set of items. Used for a single item. Scale Response Scale

28. Dimensionality We are focusing on unidimensional scaling One-dimensional constructs

29. Dimensionality Height

30. Dimensionality Height TallerShorter

31. Dimensionality Height TallerShorter Thirst

32. Dimensionality Height TallerShorter Thirst More thirsty Less thirsty

33. Dimensionality One-dimensional constructs Height TallerShorter Thirst More thirsty Less thirsty Self-esteem

34. Dimensionality One-dimensional constructs Height TallerShorter Thirst More thirsty Less thirsty Self-esteem MoreLess

35. Dimensionality Two-dimensional construct

36. Dimensionality Academic ability Two-dimensional construct

37. Dimensionality Academic ability Two-dimensional construct Verbal

38. Dimensionality Academic ability Two-dimensional construct BetterWorse Verbal

39. Dimensionality Academic ability Two-dimensional construct BetterWorse Mathematical Verbal

40. Dimensionality Academic ability - Represented by (x, y) Two-dimensional construct BetterWorse Mathematical Verbal Better Worse

41. Dimensionality Three-dimensional construct

42. Dimensionality

43. Dimensionality Semantic Differential (Psychologist theorize that meaning / objects can be described / distinguished from each other by 3-D…)

44. Dimensionality Three-dimensional construct Evaluation Semantic Differential (“general theory of meaning”)

45. Dimensionality Three-dimensional construct Semanticdifferential(meaning) Evaluation UnfavorableFavorable

46. Dimensionality Semanticdifferential(meaning) Evaluation Potency UnfavorableFavorable

47. Dimensionality Semanticdifferential(meaning) Evaluation Potency UnfavorableFavorable Powerful Powerless

48. Dimensionality Semanticdifferential(meaning) Evaluation Potency Activity UnfavorableFavorable Powerful Powerless

49. Dimensionality Three-dimensional construct - Represented by (x, y, z) Semantic Differential Examples: Basketball, Book, Dentist, … Others? Evaluation Potency Activity UnfavorableFavorable Powerful Powerless Active Passive

50. Types of Scales l Easier to understand l Useful for understanding multidimensional l Many concepts may be one-dimensional l Reflect multiple properties simultaneously Unidimensional Multidimensional

51. Prominent Unidimensional Scale Types l Thurstone l Likert l Guttman lfor our purposes less emphasis will be placed on this form of scaling

53. Thurstone Scaling

54. Thurstone Scaling - Methods l Different Methods Exist lEqual-appearing intervals (easiest - 1925) »This is the only one we will look at in more detail lSuccessive intervals lPaired comparison

55. Thurstone: Equal-Appearing Intervals l A way of measuring attitudes along a single dimension by asking respondents to agree or disagree with each of a set of statements that are about that attitude. l The statements are designed to be parallel in construction, but some toward one end of the scale and some toward the other end. –each statement tries to indicate the attitude in a slightly different way. –Should have positive, negative and neutral statements

56. Equal-Appearing Intervals Method: Start with the development of a large set ( at least 80 - 100) of items (these are the statements referenced in the previous slide) that you think all reflect the same construct. Example: If measuring attitude towards a software application: Example: If measuring attitude towards a software application: * I had trouble finding what I wanted. * I liked how easy the software was. * The software has many convenient features. * The software was confusing. * etc. All scaling methods - first start with the development of the focus of the scaling project. Unidimensional scaling method means - construct of interest can be thought of as a one-dimensional.

57. Equal-Appearing Intervals: Example Example: Generate a large set (at least 80-100) of possible statements. Agree Disagree I like going to Chinese restaurants [ ] [ ] Chinese restaurants provide good value for money [ ] [ ] There are Chinese restaurants near where I live [ ] [ ] I only go to restaurants with others (never alone) [ ] [ ] Chinese restaurants are expensive [ ] [ ] etc.

58. Item examples (attitudes towards AIDS) l People get AIDS by engaging in immoral behaviour l Someone with AIDS could be just like me l AIDS is the wrath of God l AIDS is an epidemic that affects us all l People with AIDS should be treated just like everybody else. l People with AIDS are bad l People with AIDS are real people l I treat everyone the same, regardless of whether or not they have AIDS l AIDS costs the public too much l AIDS is a death sentence l AIDS is preventable l People who contract AIDS deserve it l AIDS doesn't have a preference, anyone can get it. l AIDS is the worst thing that could happen to you. l AIDS is helps control the population. l People with AIDS do not deserve our help l AIDS will never happen to me l People with AIDS deserve what they got l AIDS distracts from other diseases that deserve our attention more l People with AIDS deserve our understanding, but not necessarily special treatment l People with AIDS should be quarantined to protect the rest of society

59. Equal-Appearing Intervals Work quickly and effectively under pressure 49 Many judges (50 – 100) individually sort these statements into 11 piles where: 1 = least favorable to the concept 11 = most favorable to the concept 123456 7891011 Manage resources effectively. 4 Decide how to manage multiple tasks. 20 Organize the work when directions are not specific. 39 11 = most favorable to the concept

60. Equal-Appearing Intervals: Example l Question selection –Judges (50 – 100) rate in terms of favorability towards construct - 1 (least favorable) to 11 (most favorable). –For each statement, plot a histogram of the numbers against which the different judges scored it. –For each statement, identify the median score, and the interquartile range. »The distance between the top of the lower quartile and the bottom of the upper. –Sort the list by median value (This is the 'common' score in terms of agreement). –Select a set of statements (no less then 11) that are equal positions across the range of medians (see next slide). »Choose the one with the lowest interquartile range for each position. l Graphically presented on next slides

61. Equal-Appearing Intervals For each item (statement), plot the distribution of pile numbers. Example: Statement 1 - “I had trouble finding what I wanted”

62. Equal-Appearing Intervals For each item, plot the distribution of pile numbers. Note. This chart not related to previous chart example.

63. Equal-Appearing Intervals For each item, plot the distribution of pile numbers. Get the median.

64. Equal-Appearing Intervals For each item, plot the distribution of pile numbers. Get the median. and interquartile range (75%-25%).

65. Equal-Appearing Intervals Choose those items that are at equal intervals (from 1 to 11) and that have the smallest interquartile range. No less than 11 chosen – one for each category. Items that judges have very different views on are discarded. Work quickly and effectively under pressure 49 Manage resources effectively. 4 Decide how to manage multiple tasks. 20 Organize the work when directions are not specific. 39 1234567891011 Work quickly and effectively under pressure 49 Manage resources effectively. 4 Decide how to manage multiple tasks. 20 Organize the work when directions are not specific. 39 Organize the work when directions are not specific. 39

66. Final step l Select final scale items –select statements that are at equal intervals across the range of medians (e.g., one for each rating level, 1-11) –try to select the statement that has the least amount of variability across judges (smallest Interquartile Range )

67. Item stats

68. Final scale l For instance: –People with AIDS are like my parents (6) –Because AIDS is preventable, we should focus our resources on prevention instead of curing (5) –People with AIDS deserve what they got. (1) –Aids affects us all (10) –People with AIDS should be treated just like everybody else. (11) –AIDS will never happen to me. (3) –It's easy to get AIDS (5) –AIDS doesn't have a preference, anyone can get it (9) –AIDS is a disease that anyone can get if they are not careful (9) –If you have AIDS, you can still lead a normal life (8) –AIDS is good because it helps control the population. (2) –I can't get AIDS if I'm in a monogamous relationship. (4)

69. Details… l The value in parentheses after each statement is its scale value. l Random order of the statements with respect to scale values. l Notice that there is no item with scale value of 7 and that we have two with values of 5 and of 9 (one of these pairs will average out to a 7). l You now have a scale -- a yardstick you can use for measuring attitudes towards people with AIDS. l A person's total scale score is the average the scale scores of all the items that person agreed with

70. How Does Respondent Use? l Respondent checks agree or disagree for each item. l The respondent’s total score is the average scale value (some sources indicate use the median) of all the items Agreed With. l Items to which the respondent disagreed are discarded in score calculation.

71. Equal-Appearing Intervals l Used in psychology and education (social science research) l Very time consuming to create l Applied researchers typically don’t have the time to develop. –Instead look for existing instruments.

73. Likert Scaling

74. l Unidimensional scaling method l Development –Define what you are measuring (construct) –Generate scale items (80-100) »Researcher, expert created. »Statements to which respondents will respond on a Likert response scale. –Judges rate the times (next slide)

75. Likert Scales l Same initial process (define construct, generate a large list of items/stimuli) l group of judges rate the items - 1-to-5 rating scale where: 1.= strongly unfavourable to the concept 2.= somewhat unfavourable to the concept 3.= undecided 4.= somewhat favourable to the concept 5.= strongly favourable to the concept n Note. This is not the Likert response scale to be used by respondents

76. Likert Scales l Select the items –compute the intercorrelations between all pairs of items, based on the ratings of the judges. »Discard items that have a low item-total correlation

77. Deciding Which Items to Select l Throw out items that have a low correlation with the total (summed) score across all items. l Identify strong correlation between individual item & total Item-total correlations Internal consistency

78. Administer the scale l Each respondent is asked to rate each item on some response scale. 1.= strongly disagree 2.= disagree 3.= undecided 4.= agree 5.= strongly agree l Can also use a forced-choice response scale with an even number of responses (no middle neutral) l Total score for the respondent on the scale is the sum of their ratings for all of the items (this is why this is sometimes called a "summated" scale).

79. How Does Respondent Use? l Gives a rating for each item on the scale. l Sum these across all items. l This is why this is sometimes called a “summated” scale. l Don’t forget to reverse any reversal items.

81. Guttman Scaling

82. l Also known as –Scalogram analysis –Cumulative scaling l Purpose: –Establish a one-dimensional continuum –Used to measure how extreme the views held are –Perfectly predict item responses from total score »Seldom perfect in practice

83. Guttman Scaling Start with a large set of items (80-100) that you think all reflect the same construct and that can be answered with Agree / Disagree Organize the work when directions are not specific. 39 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4 Work quickly and effectively under pressure 49 Organize the work when directions are not specific. 39 Decide how to manage multiple tasks. 20 Manage resources effectively. 4

84. Guttman Scaling Have many judges rate each statement with respect to whether it is favorable with respect to the construct -- a yes/no rating. 3Scan a multitude of information and decide what is important. 1Manage time effectively 2Manage resources effectively. 3Scan a multitude of information and decide what is important. 4Decide how to manage multiple tasks. 5Organize the work when directions are not specific. 1Manage time effectively Rating Sheet YesNo

85. Guttman Scaling Lay out a table:

86. Guttman Scaling Lay out a table: ItemItemItemItemItemItem Respondent27538... 7YYYYYY 15YYY--Y-- 3YYYY---- 29YYYY---- 19YYY------ 32YY--Y---- 41YY-------- 6YY-------- 14Y----Y---- 33------------

87. Guttman Scaling Lay out a table: ItemItemItemItemItemItem Respondent27538... 7YYYYYY 15YYY--Y-- 3YYYY---- 29YYYY---- 19YYY------ 32YY--Y---- 41YY-------- 6YY-------- 14Y----Y---- 33------------ When sorted by row and column it will show whether there is a cumulative scale.

88. Guttman Scaling Lay out a table: ItemItemItemItemItemItem Respondent27538... 7YYYYYY 15YYY--Y-- 3YYYY---- 29YYYY---- 19YYY------ 32YY--Y---- 41YY-------- 6YY-------- 14Y----Y---- 33------------ When sorted by row and column it will show whether there is a cumulative scale. Note the exceptions.

89. Guttman Scaling l Several statistical techniques for examining the table to find a cumulative scale. l There is seldom a perfectly cumulative scale, we usually have to test how good it is. l These statistics also estimate a scale score value for each item.

90. How Does Respondent Use? l For selected items (the final scale), they simply answer whether they agree or not (yes/no). l Since each item has a scale value, we can add these values. l Total score is the sum of the scale items respondent agreed to.