1. Dealing with Validity, Reliability, and Ethics
Chapter Nine
Dealing with Validity, Reliability, and Ethics

2. Validity and Reliability
Validity and reliability in a qualitative study are about providing information and rationale for the study’s processes and adequate evidence so that readers can determine the results are trustworthy.
Lichtman’s criteria for good qualitative research:
Explains the researcher’s role in relation to those studied.
Makes a case that the topic of the study is important
Clearly presents how the study was done,
Convincingly presents the findings of the study.

3. Internal Validity or Credibility
Internal validity deals with the question of how research findings match reality.
In QR, interpretations of reality are accessed directly through participants’ observations and interviews.
Validity in QR must be assessed
in terms of whether the findings are credible given the data presented.
in relationship to the purposes and circumstances of the research

4. Techniques to Assess Internal Validity or Credibility
Triangulation—using more than one data collection method, multiple sources of data, multiple investigators or multiple theories—is probably the most well known strategy to shore up the internal validity of a study.
Member Checks/Respondent Validation involves soliciting feedback on your preliminary or emerging findings from some of the people that you interviewed.

5. Techniques to Assess Internal Validity or Credibility, cont’d.
Adequate Engagement in Data Collection makes sense when you are trying to get as close as possible to participants’ understanding of a phenomenon. The best rule of thumb is that the data and emerging findings must feel saturated.
Researcher’s Position/Reflexivity examines how a particular researcher’s values and expectations influenced the conduct and conclusions of the study.
Peer Examination or Peer Review involves asking one or more colleagues to scan some of the raw data and assess whether the findings are plausible based on the data.

6. Reliability or Consistency
Reliability is problematic in the social sciences because human behavior is never static, nor is what many experience necessarily more reliable than what one person experiences.
Repetition is not a proxy for the establishment of truth in the QR context; measurements, observation, and people can be repeatedly wrong.
The question is whether the results are consistent with the data collected.
The human instrument can become more reliable through training and practice.

7. Audit Trail
You can optimize consistency, and therefore reliability, by making a detailed account of how the study was conducted and how the data were analyzed.
To construct an audit trail, keep a research journal or record real time memos on the process of conducting the research.
In books and theses, the audit trail is in the methodology chapter (often with supporting appendices).
Journal articles tend to have a very abbreviated audit trail or methodology section.

8. External Validity or Transferability
External validity is concerned with the extent to which the findings of one study can be generalized to other situations.
Generalizability in the statistical sense cannot occur in QR.
Ways to assess validity in QR: transferability, working hypotheses , concrete universals, reader or user generalizability; rich, thick description, maximum variation sampling
Some research designs require alternate and/or additional conceptualizations of validity.

9. How Ethical Considerations Relate to the Trustworthiness of Qualitative Research
Part of ensuring the trustworthiness of a study is that the researcher is trustworthy in carrying out the study in as ethical a manner as possible.
Ethical issues can be procedural, situational, or relational. Examples:
Interviewing/Observation: Participants say or do things they will later be embarrassed about, or reveal information they had not intended to disclose. Painful memories surface in an interview.
Documents: Personal records are problematic unless willingly surrendered for research. Online data sources raise questions of informed consent, authenticity, and public domain status.
Data analysis: Investigator bias can cause exclusion of data.
Disseminating findings: An investigator can lose control over data and its subsequent use. Anonymity of case study participants is hard to preserve at the local level.

10. Patton’s Ethical Issues Checklist
Explaining the purpose of the inquiry and methods to be used
Confidentiality
Data access and ownership
Data collection boundaries
Reciprocity (what’s in it for the interviewee and issues of compensation)
Ethical and methodological choices
Ethical versus legal issues
Promises
Risk assessment
Informed consent