Measurement and Questionnaire Design

Operationalizing From concepts to constructs to variables to measurable variables A measurable variable has been operationalized, has an operational definition Some of the greatest research creativity can be found here

Scale Types Nominal – assigns names, try to determine equality among categories Ordinal - ranks phenomena Interval – ranks with assumed equal distances between responses, categories Ratio – the properties of an interval scale with an “absolute” (meaningful) zero

Reliability Necessary but not sufficient condition for validity Can indicate –Stability (across people, times, instruments) –Internal consistency T = X – e where T is a true score, X the observed score and e is error

Stability & Consistency Check stability, consistency in a number of ways –Parallel forms to the same individual –Test – retest variation (same form) –Split-half with halves scored separately (but there are 92,378 ways of dividing a 20 item test in half) –Item – total correlations –Cronbach’s  takes the mean of all possible split-half correlations

To Increase Reliability Estimates Increase the number of items Use a sound theoretical base to construct items Items from a more heterogeneous group will be more reliable Use non-speeded tests if ability to finish rapidly isn’t assessed

Validity Are we measuring what we intend to measure? Reliability a precondition Many measures of validity are correlational

Types of Validity Face validity – commonsensical (but not necessarily obvious) Criterion (convergent) validity – do alternate measures provide similar results –Predictive –Concurrent Construct validity – the extent to which an operationalization measures the concept it is supposed to measure

Attitudes Three components –Affective (eg., intention to buy) –Cognitive (eg., intention to buy) –Behavioral (eg., purchase behavior)

Attitude Scales Having two options –Yes/no –Agree/disagree More than two options –Likert (or Likert-type) –Semantic differential, bipolar adjectives –Graphic rating scale

Attitude Scales Balanced versus unbalanced response set Forced versus non-forced response set

Questionnaire Design An art How will it be administered? Will you use open versus closed items? –Closed items should have exhaustive, mutually exclusive responses whenever possible As a rule, continuous measures are better than categorical

Questions Consider least knowledgeable respondent for wording Avoid loaded, leading questions Avoid double-barreled questions (look for every instance of the word “and”) Exorcise ambiguity Consider aided recall when possible

Question Sequence Easy questions first Sensitive questions middle or last Branch if necessary (pivot, filter) General before specific questions Avoid anchoring effects Consider reverse scoring

Layout Create interest (with a header, cover, pre- contact) Offer a results summary Use color if budget allows Use white space liberally Keep coding requirements in mind

Pilot Consult experts before question construction, design if unfamiliar with material Show experts draft survey Accurately estimate true time required to complete Permit questions during pilot administration Do the pilot in person