Measurement and Observation

Choices During Operationalization Researchers make a number of key decisions when deciding how to measure a concept Dimensions and sub-dimensions Range of variation within dimensions Categories to represent range Levels of measurement Nominal, ordinal, interval, ratio

Operationalization: A deliberative process Not a simple, linear process Complicated and fraught with trade-offs Iterative process with cycles of consideration Debate over proper measurement is key

Dimensions of the Concept Creating operational measures forces realization about lack of conceptual clarity List of possible dimensions may be long Need to decide which ones are most relevant Ask which ones are central to the inquiry Reflect on research hypotheses or theories

Range of Variation Sense of the upper and lower limits How much are you willing to combine different people into the same category? Extremely high and Extremely low may be collapsed Eg. Income, age, height, etc. Opposition and support for attitudes Agreement and disagreement

Variation Between Extremes Degree of precision How detailed you need to be in measurement Eg. Age breaks or Exact age? Related to purpose of study Eg. Political Party ID: Dichotomy: Democrat or Republican Continuum: 7-point scale w/ “independent-leaner”

Levels of Measurement Nominal Measures Ordinal Measures Interval Measures Ratio Measures

Nominal Measures Names for characteristics Do not Exist along an Explicit continuum Exhaustive Mutually Exclusive Eg. Religious Affiliation Eg. Place of Birth

Ordinal Measures Can be logically rank-ordered Represent relatively more of less of variable No consistent distance between points of measurement Not just different from one another More of less of some attribute Eg. “Not very important,” “fairly important,” “very important” “Extremely important”

Interval Measures Consistent distance separating attribute We can say how much more of an attribute Logical distance between attributes can be Expressed in meaningful standard intervals Eg. Temperature 90 degrees vs. 80 degrees = 10 degree difference 50 degrees vs. 40 degrees = 10 degree difference Zero-point is arbitrary

Ratio Measures In addition to all the properties of nominal, ordinal, and interval measures, ratio measures have a true zero point Eg. Length of time Eg. Number of times Eg. Number of affiliations Can actually state ratio of one to another X has twice as many affiliations as Y

What’s that scale? Style of music in a music video Number of violent acts in a music video Whether a music video has violence or not? High, Medium or Low violence in a music video Hair color Number of hairs on your head Sat scores Social Security Number

What’s that scale? A baseball player's batting average A baseball player's field position A baseball player's position in the batting order A baseball player's uniform number College football rankings IQ

Types of questions Multiple choice questions Agree/disagree questions Likert questions Frequency scales Semantic differential scales Forced-choice statement pairs Thermometer feeling scales Nominal checklists Ordinal categories Rank-order questions Filter questions Open-ended

Multiple Choice Question

Multiple Choice with Range Options

Agree/Disagree Questions

Likert Scale

Frequency Scale

Semantic Differential Scales

Forced-choice Statement Pairs

Thermometer Feeling Scales

Nominal Checklist

Ordinal Categories

Rank-order Preference Questions

Rank-order Evaluation Questions

Filter Questions

Open-ended Questions

Tips on Question Construction 1. Make questions clear using simple language 2. Keep questions concise 3. Provide instructions for answering questions Don’t assume respondent knows question style 4. Keep research purpose in mind Make sure items can answer research question 5. Don’t ask double-barreled questions E.g., “How well do you think the current Presidential Administration is handling foreign policy and the war on terrorism?”

More Tips 6. Avoid leading questions 7. Avoid negative questions E.g., “Like most Americans, do you read a newspaper every day?” 7. Avoid negative questions E.g., “The U.S. should not invade Iraq” Agree or disagree? 8. Do not ask questions that require complicated mental calculus E.g., “In the past 30 days, how many hours have you spent watching television with your family?” 9. Keep ordering of questions in mind

Using Pre-Existing Measures It is okay to borrow measures Cite source of questions to give credit Benefits of using Existing measures: Saves work Pre-tested for reliability/validity Research becomes cumulative

Pretesting Clarity in question wording Are categories: Exhaustive? Mutually Exclusive? Realistic time estimate Preliminary empirical analysis