Content Coding Another way to “create” new measures

What is source material to code?  Could be past: Archival, Meta-analysis  Could be present: Observational, Survey, Experiment  Could any aspect of Person or Situation Affect, Behavior, Cognition, Physiology/Biology Intrapersonal, Interpersonal, Intergroup, Societal

What to do after you find it? 1. Define Variables 2. Operationalize Variables 3. Create Coding Book 4. Train Coders 5. Pilot Test 6. Code the Material 7. Statistically Analyze Codings 8. Revise Coding or Coders (if necessary)

(1) Define Variables  See prior material in class about Research Topics, Questions, Hypotheses Also…  Must have clear definition, be documented, explicit, and open to scrutiny  Must identify your inclusion and exclusion criteria  This is partly an a priori determination of what to investigate  This is partly an on-going dynamic process where you review the literature and decide what to investigate  This is partly an ad-hoc statement or summary of what you investigated

(2) Operationalize Variables  See prior material in class about translating hypotheses into measures

(3) Create Coding Book  Coding Book is a blue-print that identifies all variables and how to transform into quantitative analysis  What to include… (some options) All your variables Research respondents – sample size, age, sex, location, and all other demographic characteristics, etc. Source information - publication form, date of publication, institution, country of publication, etc. Historical and Contextual information – time frame, historical events that occurred during the time frame, linguistic or culture influences, etc. IF coding for surveys/experiments… Type of IV - description, theory, type, levels, etc. Type of DV - description, theory, type, levels, etc. Aspects of Procedures – type, level, design Aspects of Hypothesis/Theory

(4) Train Coders  Identify number If subjective coding, need two or more, and ideally three to five. If objective, then one coder is sufficient  Meeting(s) Train on coding manual Resolve comprehension issues, ambiguity issues

(5) Pilot Test  What to test? Maybe includes small portion of actual material to be coded Maybe includes similar, but not exact, material to be coded  Practice to produce consistency Did the coder code the same way from the first entry to the last entry? Might become more proficient and knowledgeable at end so code differently than first entry Is there a break in coding of days, week, months?  Possibly refine coding manual and procedures

(6) Code the Material  Coding the entire range of source material  Possible interim meetings to compare/contrast and discuss problems

(7) Statistically Analyze  See Spring Statistics class

(8) Revise Coding or Coders  If statistical analysis shows inconsistencies…  If coding errors discovered after the fact…  If coding bias discovered after the fact…  If realize want to include more source material or more variables… …, then repeat steps 1-7 including new coding or new coders

Advanced Sources  The Handbook of Research Methods in Social and Personality Psychology, Edited by Reis and Judd, has good chapters on content coding: Chapter 12 and Chapter 8