1. Evaluating Qualitative Research
INFO 272. Qualitative Research Methods
17 April 2008
Evaluating Qualitative Research

2. Criteria for Quant Research
The “Holy Trinity”
Reliability
Validity
Representativeness and Generalizability
Internal
External
Content
Criterion
Construct

3. Functional Equivalence
Criteria for evaluating quantitative research is not directly applicable to qualitative research
Kvale on epistemology
Abandoning a correspondence theory of truth
Defensible (rather than absolute) knowledge claims requiring argumentation

4. Functional Equivalence
We must abstract existing quantitative standards to be respecified for qualitative research

5. Functional Equivalence
Quantitative Tradition
Qualitative Tradition
Reliability of measures (c)
Confidence (c)
Relevance (r)
Triangulation and reflexivity (c)
Internal validity (c)
Transparency and procedural clarity (c)
Sample size (c)
Corpus construction (c, r)
Representative sampling (r)
Thick description (c, r)
External validity (r)
Local surprise (r)
Validity of measures (r)
Communicative validation (r)

6. Triangulation and Reflexivity (c)
i.e. interviews about Internet use supplemented by observation

7. Transparency (c)

8. Corpus Construction (c, r)
Maximizing the diversity of unknown representations and mapping those representations
Representativeness and ‘external validity’ is a matter of argumentation

9. Thick Description (c, r)
‘high-fidelity’ reportage: verbatim quotes – demonstrating the provenance of a claim
Footnotes and sources
But also, do you get a whole picture of the social world, its elements, and how they are interlinked?

10. Local Surprise (r) Surprise in relation to a common-sense view
Surprise in relation to theoretical expectation
Solely confirming evidence (just as totally consistent evidence) should raise suspicion

11. Communicative Validation (r)
Gaining feedback from research participants (and others?)
Remember interviewing technique of ‘interpreting’ on the fly to get confirmation from interviewees

12. Additional thoughts on validity
Kvale:
Checking, questioning findings through an iterative approach
Pragmatic validity

13. Kvale on generalizability
Statistical – formal and explicit
Analytical – generalization through assertional logic

14. Defending Ethnography
Focus groups…what people say is not what they do
Not generalizable…rapidly expanding corpus of individual and research community experience
Not a science…a discovery science for handling the complex social world that does not function well as a laboratory
Show me results…creative presentation modes