1. Measurement and Observation

2. Choices During Operationalization Researchers make a number of key decisions when deciding how to measure a concept Researchers make a number of key decisions when deciding how to measure a concept Dimensions and sub-dimensions Dimensions and sub-dimensions Range of variation within dimensions Range of variation within dimensions Categories to represent range Categories to represent range Levels of measurement Levels of measurement Nominal, ordinal, interval, ratio Nominal, ordinal, interval, ratio

3. Operationalization: A deliberative process Not a simple, linear process Not a simple, linear process Complicated and fraught with trade-offs Complicated and fraught with trade-offs Iterative process with cycles of consideration Iterative process with cycles of consideration Debate over proper measurement is key Debate over proper measurement is key

4. Dimensions of the Concept Creating operational measures forces realization about lack of conceptual clarity Creating operational measures forces realization about lack of conceptual clarity List of possible dimensions may be long List of possible dimensions may be long Need to decide which ones are most relevant Need to decide which ones are most relevant Ask which ones are central to the inquiry Ask which ones are central to the inquiry Reflect on research hypotheses or theories Reflect on research hypotheses or theories

5. Range of Variation Sense of the upper and lower limits Sense of the upper and lower limits How much are you willing to combine different people into the same category? How much are you willing to combine different people into the same category? Extremely high and Extremely low may be collapsed Extremely high and Extremely low may be collapsed Eg. Income, age, height, etc. Eg. Income, age, height, etc. Opposition and support for attitudes Opposition and support for attitudes Agreement and disagreement Agreement and disagreement

6. Variation Between Extremes Degree of precision Degree of precision How detailed you need to be in measurement How detailed you need to be in measurement Eg. Age breaks or Exact age? Eg. Age breaks or Exact age? Related to purpose of study Related to purpose of study Eg. Political Party ID: Eg. Political Party ID: Dichotomy: Democrat or Republican Dichotomy: Democrat or Republican Continuum: 7-point scale w/ “independent-leaner” Continuum: 7-point scale w/ “independent-leaner”

7. Levels of Measurement Nominal Measures Nominal Measures Ordinal Measures Ordinal Measures Interval Measures Interval Measures Ratio Measures Ratio Measures

8. Nominal Measures Names for characteristics Names for characteristics Do not Exist along an Explicit continuum Do not Exist along an Explicit continuum Exhaustive Exhaustive Mutually Exclusive Mutually Exclusive Eg. Religious Affiliation Eg. Religious Affiliation Eg. Place of Birth Eg. Place of Birth

9. Ordinal Measures Can be logically rank-ordered Can be logically rank-ordered Represent relatively more of less of variable Represent relatively more of less of variable No consistent distance between points of measurement No consistent distance between points of measurement Not just different from one another Not just different from one another More of less of some attribute More of less of some attribute Eg. “Not very important,” “fairly important,” “very important” “Extremely important” Eg. “Not very important,” “fairly important,” “very important” “Extremely important”

10. Interval Measures Consistent distance separating attribute Consistent distance separating attribute We can say how much more of an attribute We can say how much more of an attribute Logical distance between attributes can be Expressed in meaningful standard intervals Logical distance between attributes can be Expressed in meaningful standard intervals Eg. Temperature Eg. Temperature 90 degrees vs. 80 degrees = 10 degree difference 90 degrees vs. 80 degrees = 10 degree difference 50 degrees vs. 40 degrees = 10 degree difference 50 degrees vs. 40 degrees = 10 degree difference Zero-point is arbitrary Zero-point is arbitrary

11. Ratio Measures In addition to all the properties of nominal, ordinal, and interval measures, ratio measures have a true zero point In addition to all the properties of nominal, ordinal, and interval measures, ratio measures have a true zero point Eg. Length of time Eg. Length of time Eg. Number of times Eg. Number of times Eg. Number of affiliations Eg. Number of affiliations Can actually state ratio of one to another Can actually state ratio of one to another X has twice as many affiliations as Y X has twice as many affiliations as Y

12. What’s that scale? Style of music in a music video Style of music in a music video Number of violent acts in a music video Number of violent acts in a music video Whether a music video has violence or not? Whether a music video has violence or not? High, Medium or Low violence in a music video High, Medium or Low violence in a music video Hair color Hair color Number of hairs on your head Number of hairs on your head Sat scores Sat scores Social Security Number Social Security Number

13. What’s that scale? A baseball player's batting average A baseball player's batting average A baseball player's field position A baseball player's field position A baseball player's position in the batting order A baseball player's position in the batting order A baseball player's uniform number A baseball player's uniform number College football rankings College football rankings IQ IQ

14. Types of questions Multiple choice questions Multiple choice questions Agree/disagree questions Agree/disagree questions Likert questions Likert questions Frequency scales Frequency scales Semantic differential scales Semantic differential scales Forced-choice statement pairs Forced-choice statement pairs Thermometer feeling scales Thermometer feeling scales Nominal checklists Nominal checklists Ordinal categories Ordinal categories Rank-order questions Rank-order questions Filter questions Filter questions Open-ended Open-ended

15. Multiple Choice Question

16. Multiple Choice with Range Options

17. Agree/Disagree Questions

18. Likert Scale

19. Frequency Scale

20. Semantic Differential Scales

21. Forced-choice Statement Pairs

22. Thermometer Feeling Scales

23. Nominal Checklist

24. Ordinal Categories

25. Rank-order Preference Questions

26. Rank-order Evaluation Questions

27. Filter Questions

28. Open-ended Questions

29. Tips on Question Construction 1. Make questions clear using simple language 1. Make questions clear using simple language 2. Keep questions concise 2. Keep questions concise 3. Provide instructions for answering questions 3. Provide instructions for answering questions Don’t assume respondent knows question style Don’t assume respondent knows question style 4. Keep research purpose in mind 4. Keep research purpose in mind Make sure items can answer research question Make sure items can answer research question 5. Don’t ask double-barreled questions 5. Don’t ask double-barreled questions E.g., “How well do you think the current Presidential Administration is handling foreign policy and the war on terrorism?” E.g., “How well do you think the current Presidential Administration is handling foreign policy and the war on terrorism?”

30. More Tips 6. Avoid leading questions 6. Avoid leading questions E.g., “Like most Americans, do you read a newspaper every day?” E.g., “Like most Americans, do you read a newspaper every day?” 7. Avoid negative questions 7. Avoid negative questions E.g., “The U.S. should not invade Iraq” Agree or disagree? E.g., “The U.S. should not invade Iraq” Agree or disagree? 8. Do not ask questions that require complicated mental calculus 8. Do not ask questions that require complicated mental calculus E.g., “In the past 30 days, how many hours have you spent watching television with your family?” E.g., “In the past 30 days, how many hours have you spent watching television with your family?” 9. Keep ordering of questions in mind 9. Keep ordering of questions in mind

31. Using Pre-Existing Measures It is okay to borrow measures It is okay to borrow measures Cite source of questions to give credit Cite source of questions to give credit Benefits of using Existing measures: Benefits of using Existing measures: Saves work Saves work Pre-tested for reliability/validity Pre-tested for reliability/validity Research becomes cumulative Research becomes cumulative

32. Pretesting Clarity in question wording Clarity in question wording Are categories: Are categories: Exhaustive? Exhaustive? Mutually Exclusive? Mutually Exclusive? Realistic time estimate Realistic time estimate Preliminary empirical analysis Preliminary empirical analysis