KNR 295 Measurement Slide 1 Measurement Theory & Construct Validity Chapter 3

KNR 295 Measurement Slide 2 Measurement validity = construct validity

KNR 295 Measurement Slide 3 Construct Validity

KNR 295 Measurement Slide 4 Construct Validity  Determined by Operationalization

KNR 295 Measurement Slide 5 Construct Validity  So construct validity assesses how well your procedures/measures match your ideas/theories  General = construct  Specific = operationalization

KNR 295 Measurement Slide 6 Construct Validity  Two views –  Definitionalist  “The construct,the whole construct, and nothing but the construct”  Impossible!  Relativist  Define your construct  Explain how and why what you’re doing measures the construct in question  Produce some evidence (whether we do this depends on the kind of research in question) US

KNR 295 Measurement Slide 7 Construct Validity  Translation validity  Face validity  Content validity  Criterion-related validity  Predictive validity  Concurrent validity  Convergent validity  Discriminant validity

KNR 295 Measurement Slide 8 Construct Validity  Translation validity vs. Criterion- related validity  Translation validity assesses whether the operationalization matches what you know of the construct  Criterion validity actually measures this assessment (uses other measures to assess the construct validity) What we do is translation validity – arguing about the construct validity but not measuring it directly

KNR 295 Measurement Slide 9 Construct Validity  Translation validity  Face validity:  Does it look like you got it right?  Ask others…more objective  Content validity  Good definition of the construct  Good match between your measure and the definition  E.G. Fitness program – does it abide by ACSM guidelines? (e.g. construct…Body fat. Measure…sum of skinfolds)

KNR 295 Measurement Slide 10 Construct Validity  Criterion-related validity – all involve some direct test of CV  Predictive validity  Does it predict what it ought to?  E.G. Does sum of skinfolds predict cardiovascular disease?  Concurrent validity  Can your measure discriminate between 2 similar groups?  E.G. Measure sum of skinfolds of males and females – females should be higher than males(?)

KNR 295 Measurement Slide 11 Construct Validity  Criterion-related validity  Convergent validity  Correlation between this operationalization and other similar ones  E.G. Sum of skinfolds and…BMI, underwater weighing, cadaver dissection…  Discriminant validity  This operationalization is different from other stuff that is not supposed to measure the same thing  E.G. Sum of skinfolds vs. age, vs. weight, vs. gender, and so on.  Note – this is why these items are included in popular equations converting sum of skinfolds to BF%

KNR 295 Measurement Slide 12 Construct validity  Convergent vs. Discriminant Validity

KNR 295 Measurement Slide 13 Construct validity  Convergent vs. Discriminant Validity  Note – once you’ve done this, you still need translation validity to establish that the measures are what you purport them to be

KNR 295 Measurement Slide 14 Threats to Construct Validity  The laundry list…  Use this to “ask the right questions” about the studies you critique  & so to begin…

KNR 295 Measurement Slide 15 Threats to Construct Validity  Inadequate preoperational explication of constructs  Construct not defined carefully enough

KNR 295 Measurement Slide 16 Threats to Construct Validity  Mono-operation bias (independent variable)  Only one example of the construct  E.G. only one training program…but there are many out there  Mono-method bias (dependent variable)  Only one example of the construct  E.G. only one strength measure for a program that trained the whole body

KNR 295 Measurement Slide 17 Threats to Construct Validity  Interaction of different treatments  Use one or more control groups to isolate cause  E.G. You want to show that strength training improves self-esteem…but it could have been as a result of meeting you 3 times per week, not strength training…so use a control group to compare results

KNR 295 Measurement Slide 18 Threats to Construct Validity  Interaction of testing and treatment  Imagine I was interested in whether research methods improved reasoning skills…  If I tested you every week on some IQ tests, these become part of the treatment, and impair construct validity (you might be getting better because of the test, not because of being in research methods)

KNR 295 Measurement Slide 19 Threats to Construct Validity  Restricted generalizability across constructs  “Unintended consequences”  E.G. Finnish epidemiological study –  divide people into groups according to level of smoking & cholesterol  Take half of each group and assign to fitness & nutrition program, half untreated  All in fitness and nutrition program reduced smoking and improved fitness measures  BUT they also had increased all cause mortality!!! (may be an apocryphal story)

KNR 295 Measurement Slide 20 Threats to Construct Validity  Confounding constructs and levels of constructs  “I’m finding out whether aerobics or strength training has the greater impact on muscle tone”  Ok, but what amount of strength training? How often are they doing aerobics?  The labels are not in sufficient detail  Could be that only the particular versions of these programs that you used will produce the results you found

KNR 295 Measurement Slide 21 Threats to Construct Validity  Social threats to construct validity  Hypothesis guessing  I never collect data using people that have completed my motor learning class…  Evaluation apprehension  Experimental booth in Bangor…  Experimenter expectancies  I love my research…

KNR 295 Measurement Slide 22 Reliability and Levels of Measurement  We’ll leave those till we deal with conclusion validity  They are really the concern of a good stats course