1. V ALIDITY IN Q UALITATIVE R ESEARCH

2. V ALIDITY How accurate are the conclusions you make based on your data analysis? A matter of degree Non-reification – validity is not a static entity

3. V ALIDITY IN Q UANTITATIVE R ESEARCH Content validity : How well does the measure cover the domain of the subject it is testing? Criterion-related validity : Predictive validity : How well does the measure predict appropriate success and failure? Concurrent validity : How well does the measure predict appropriate success and failure in agreement with other (typically performance-based) similar measures?

4. V ALIDITY IN Q UANTITATIVE R ESEARCH Construct validity – Is the measure a valid measure of the construct? Face validity : Does the measure meet the expectations for a valid test?

5. V ALIDITY IN Q UANTITATIVE R ESEARCH Threats to internal validity: Rival hypotheses, confounding variables, and alternative explanations (is the treatment in this case responsible for the observed changes?)

6. V ALIDITY IN Q UANTITATIVE R ESEARCH ThreatWays to Reduce Sampling error and chance errorUse inferential statistics. History: Events that occur prior to the post-test might cause the effect Use control group that is also subject to the events under question. Instrumentation Consistency in instrument Quality of instrument (validity/ reliability) Testing: Use of two or more testings with the same (or closely related) instrument – early testing experience may influence later results. Use control group – should exhibit the effects of testing minus the treatment. Regression: Occurs when extreme groups are selected for treatment without a control group. Use control group.

7. V ALIDITY IN Q UANTITATIVE R ESEARCH ThreatWays to Reduce Mortality: Individuals leave an experimental group prior to completion of the study. Use control group with similar mortality. Maturation: Growth or change, unrelated to the treatment, that might affect the measured effect occurs prior to completion of the study Use control group. Instrument decay: Changes in the way an instrument is used to measure the effect during the study (e.g. re- interpretation of scoring categories) Instrument solidification prior to use  Also consider ceiling and floor effects. Selection: Subjects are selected according to a factor that causes the same effect as the treatment. Random assignment to groups

8. V ALIDITY IN Q UANTITATIVE R ESEARCH Threats to external validity: Restrictive conditions and explanations (To whom and what circumstances can the results be generalized?) Six areas of research design that may restrict generalizibility: 1. Subjects 2. Situation 3. Treatment 4. Observation or measure 5. Basis for sensing changes 6. Procedure

9. V ALIDITY IN Q UANTITATIVE R ESEARCH ThreatWays to Reduce Obtrusiveness and reactivity:  Hawthorne effect  Hypothesis guessing  Guinea pigs  Desire for treatment  Rivalry between experimental and control group (random assignment)  Novelty effect  Demoralization (control group does not get special help)  Diffusion (treatment is communicated outside the context of the study)  Emotional bonding Use unobtrusive procedures as much as possible

10. V ALIDITY IN Q UANTITATIVE R ESEARCH ThreatWays to Reduce Researcher expectancy: Those giving the treatment are aware of its potential effects Keep researchers blind to which subjects are experimental or control Multiple treatment interaction: Residual effects of one treatment influence a later treatment Rotate the treatment into first position Testing-treatment interaction: Treatment effect is affected by pre- testing Post-test only design Solomon four-group design Selection-treatment interaction: Treatment affects who is selected (e.g., people who are in particular need of the treatment) Use of volunteers for treatment and control groups (weakens external validity) Mortality-treatment interaction: Subjects drop out in reaction to the treatment Use control group with similar mortality.

11. R ELIABILITY IN Q UANTITATIVE R ESEARCH Reliability: Does the measure yield consistent results? Does it measure consistently? Inter-observer/ inter-rater reliability Internal consistency: Are all items measuring appropriate and similar things? Equivalence: Are different forms of the measure equivalent? Stability: Is the behavior stable over time?

12. V ALIDITY IN Q UALITATIVE R ESEARCH Researcher as instrument: Are you seeing/hearing what is really there? Are you seeing/hearing what is important to you? Are you presenting findings that accurately reflect the reality of the studied context? How will you know when You’ve collected enough data?

13. V ALIDITY IN Q UALITATIVE R ESEARCH Coherence Comprehensiveness Authenticity Plausibility Trustworthiness Reflexivity Particularity Utility Accuracy Transferability

14. V ALIDITY IN Q UALITATIVE R ESEARCH Maxwell (1992) Descriptive validity Interpretive validity Theoretical validity Generalizability Evaluative validity

15. V ALIDITY IN Q UALITATIVE R ESEARCH Strategies: Methodological fidelity Data saturation Triangulation Discrepant (negative) case analysis Multi-method approaches Researcher-as-instrument Memoing Audit trail Member-checking