Dealing with Validity, Reliability, and Ethics Chapter Nine Dealing with Validity, Reliability, and Ethics

Validity and Reliability Validity and reliability in a qualitative study are about providing information and rationale for the study’s processes and adequate evidence so that readers can determine the results are trustworthy. Lichtman’s criteria for good qualitative research: Explains the researcher’s role in relation to those studied. Makes a case that the topic of the study is important Clearly presents how the study was done, Convincingly presents the findings of the study.

Internal Validity or Credibility Internal validity deals with the question of how research findings match reality. In QR, interpretations of reality are accessed directly through participants’ observations and interviews. Validity in QR must be assessed in terms of whether the findings are credible given the data presented. in relationship to the purposes and circumstances of the research

Techniques to Assess Internal Validity or Credibility Triangulation—using more than one data collection method, multiple sources of data, multiple investigators or multiple theories—is probably the most well known strategy to shore up the internal validity of a study. Member Checks/Respondent Validation involves soliciting feedback on your preliminary or emerging findings from some of the people that you interviewed.

Techniques to Assess Internal Validity or Credibility, cont’d. Adequate Engagement in Data Collection makes sense when you are trying to get as close as possible to participants’ understanding of a phenomenon. The best rule of thumb is that the data and emerging findings must feel saturated. Researcher’s Position/Reflexivity examines how a particular researcher’s values and expectations influenced the conduct and conclusions of the study. Peer Examination or Peer Review involves asking one or more colleagues to scan some of the raw data and assess whether the findings are plausible based on the data.

Reliability or Consistency Reliability is problematic in the social sciences because human behavior is never static, nor is what many experience necessarily more reliable than what one person experiences. Repetition is not a proxy for the establishment of truth in the QR context; measurements, observation, and people can be repeatedly wrong. The question is whether the results are consistent with the data collected. The human instrument can become more reliable through training and practice.

Audit Trail You can optimize consistency, and therefore reliability, by making a detailed account of how the study was conducted and how the data were analyzed. To construct an audit trail, keep a research journal or record real time memos on the process of conducting the research. In books and theses, the audit trail is in the methodology chapter (often with supporting appendices). Journal articles tend to have a very abbreviated audit trail or methodology section.

External Validity or Transferability External validity is concerned with the extent to which the findings of one study can be generalized to other situations. Generalizability in the statistical sense cannot occur in QR. Ways to assess validity in QR: transferability, working hypotheses , concrete universals, reader or user generalizability; rich, thick description, maximum variation sampling Some research designs require alternate and/or additional conceptualizations of validity.

How Ethical Considerations Relate to the Trustworthiness of Qualitative Research Part of ensuring the trustworthiness of a study is that the researcher is trustworthy in carrying out the study in as ethical a manner as possible. Ethical issues can be procedural, situational, or relational. Examples: Interviewing/Observation: Participants say or do things they will later be embarrassed about, or reveal information they had not intended to disclose. Painful memories surface in an interview. Documents: Personal records are problematic unless willingly surrendered for research. Online data sources raise questions of informed consent, authenticity, and public domain status. Data analysis: Investigator bias can cause exclusion of data. Disseminating findings: An investigator can lose control over data and its subsequent use. Anonymity of case study participants is hard to preserve at the local level.

Patton’s Ethical Issues Checklist Explaining the purpose of the inquiry and methods to be used Confidentiality Data access and ownership Data collection boundaries Reciprocity (what’s in it for the interviewee and issues of compensation) Ethical and methodological choices Ethical versus legal issues Promises Risk assessment Informed consent