Construct Validity And its Threats Jill Hoxmeier H615: Advanced Research Design October 10, 2013 “Boy, a few more like that and I’ll be ready for Gamblers Anonymous”

Twin Problems of Construct Validity Construct Validity: making inferences from the sampling particulars of a study to the higher-order constructs they represent Understanding them Assessing them “Thinking without the positing of categories and concepts in general would be as impossible as breathing in a vacuum” (Einstein, 1949)

Meehl & Cronbach Construct Validity in Psychological Tests Three steps to evaluate construct validity: Articulation of a set of theoretical constructs and their interrelations Developing ways to measure the hypothetical constructs proposed by a theory Empirically testing the hypothesized relations Cronbach & Meehl, 1955 Evolution of “Attitude” From a loose variable comprised of beliefs, norms, intentions… … to Fishbein-ian perspective that attitude = person’s evaluation of behavioral consequence x strength of belief that behavior will lead to those consequences

Why do we need valid constructs? Research cannot be done without constructs Constructs are the central means we have for connecting the operations used in an experiment to theory Construct labels carry social, political, and economic implications The creation and defense of basic constructs is a fundamental task of all science

Construct Validity Fostered by: Starting with a clear explication of the person, setting, treatment, and outcome constructs of interest Carefully selecting instances that match those constructs Assessing the match between instances and constructs for “slippage” Revising construct descriptions accordingly

Understanding Psychological Domains “…there are often no natural units of measurement…” (Shonkoff & Philllips, 2000, p.83-83) Pattern-matching logic: does the given instance sufficiently match the prototypical features to warrant using the category label?

How do you measure your domain? Think about your own research interest… What are the domains or constructs that are measured? Have you discovered from the literature any challenges in measuring those domains? What are ways investigators in your field have revised constructs to overcome those challenges?

Program of Research A theory of constructs must emphasize 1) operationalizing each construct several ways within and across studies 2) probing the pattern match between the multivariate characteristics of instances and characteristics of the target construct 3) acknowledge legitimate debate about the quality of the match given the socially constructed nature of both operations and constructs

Threats to Construct Validity Inadequate explication of constructs Construct confounding Mono-operation bias Mono-method bias Confounding constructs with levels of constructs Treatment sensitive factorial structure Reactive self-report changes Reactivity to the experimental situation Experimenter expectancies Novelty and disruptive effects Compensatory rivalry Resentful demoralization Treatment diffusion What (potential) threats are most relevant to your research? And how do you plan to guard against such threats?

Questions for You! What role does qualitative research play in contributing to construct validity? How do we balance the theory construct testing of quantitative research and the “naturalistic generalizations” that may come from readers of “thick descriptions” that qualitative researchers provide?