1. COM 633: Content Analysis Measurement; Validity Kimberly A. Neuendorf, Ph.D. Cleveland State University Fall 2010

2. Measurement “Assignment of numerals to objects or events according to rules” (Stevens, 1951)

3. Operationalization (Measurement) Categories that are exhaustive and mutually exclusive High and appropriate level of measurement Ratio Interval Ordinal Nominal

4. Evaluating Measures: Comparing Reliability, Accuracy (lack of Bias), and Precision Targets…see p. 114 of book. See also: Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park, CA: Sage Publications.

6. Validity Internal Conceptual definition =matches= Operationalization The standard of having a “good” measure—”Are we hitting the target we want?” External Generalizability Sample Ecological validity Replicability

7. Validity “Face” Criterion Relate to established standard or behavior Concurrent Predictive e.g., Oliver & Kalyanaraman (2002) study of sexual and violent content in trailers and films... Criterion is the MPAA rating

8. Validity Content Construct

9. Validity Content

10. Validity Content Construct = ???

11. Validity Content High reliability...

12. Validity Construct Related to other measures (constructs) in ways consistent with theory Convergent—relationships as expected Discriminant—lack of relationships, as expected

13. Construct Validity Smith volume (1992) on “thematic content analysis”—psychographic applications—has examples Trust-mistrust coding system for TAT stories construct validated against: Other thematic coding systems (e.g., Affiliation; Intimacy) Demographics (e.g., Mothers’ feeding habits during infancy (more or less principled), moderated by gender & age) Self-reported behaviors (e.g., Reported positive relationships with others; Divorce, family fights in last year; Drug use) Personality measures (e.g., Impulsivity; Locus of Control)

14. Validity Assessment & CATA Short et al. (2010) provides a model for validity and CATA, using tests of a new dictionary for “entrepreneurial orientation” Content Validity Deductive Inductive External Validity Reliability Dimensionality (a type of Construct Validity--Discriminant) Predictive Validity (seems to really be Construct Validity— Convergent)