Creswell Qualitative Inquiry 2e 10.1 Chapter 10 Standards of Validation and Evaluation

Creswell Qualitative Inquiry 2e 10.2 Key Questions What are some of the qualitative perspectives on validation? What are some alternative procedures useful in establishing validation? How is reliability used in qualitative research? What are some alternative stances on evaluating the quality of qualitative research? How do these stances differ by types of approaches to qualitative inquiry?

Creswell Qualitative Inquiry 2e 10.3 Perspectives on Validity Qualitative parallel equivalents to quantitative (LeCompte & Goetz, 1982) –Perspective: Parallel qualitative equivalents to counterparts in experimental and survey research –Terms: Internal Validity, External Validity, Reliability, Objectivity New terms should be used (Lincoln & Guba, 1985) –Perspective: Alternative terms that apply more to naturalistic axioms –Terms: Credibility, Transferability, Dependability, Confirmability

Creswell Qualitative Inquiry 2e 10.4 Perspectives and Terms About Verification (cont.) New terms focused on credibility (Eisner, 1991) –Perspective: Alternative terms that focus on reasonable standards for judging the credibility of qualitative research –Terms: Structural Corroboration, Consensual Validation, Referential Adequacy Feminist perspective (Lather, 1993) –Perspective: Reconceptualized: Four frames of validity –Terms: Iconic Validity, Paralogic Validity, Rhizomatic Validity, Situated/Embedded Voluptuous Validity

Creswell Qualitative Inquiry 2e 10.5 Perspectives and Terms About Verification (cont.) Metaphor for (Richardson, 1990) –Perspective: Reconceptualized: Metaphor of a crystal –Terms: Crystals: Grow, change, alter, reflect externalities, refract within themselves Distraction (Wolcott, 1994) –Perspective: Distraction: Neither guides nor informs –Terms: Understanding (better term to use than validity)

Creswell Qualitative Inquiry 2e 10.6 Types of Validation Interpretivist approach (Angen, 2000) –Ethical Validation: All research agendas must question their underlying moral assumptions, their political and ethical implications, and the equitable treatment of diverse voices Provides practical answers to questions Stimulates new dialogue –Substantive Validation: Researchers understand their own understanding of the topic that is derived from other sources and document this in the study Self-reflection process Interaction with the subject matter to co-create interpretations Written accounts must resonate with audiences -

Creswell Qualitative Inquiry 2e 10.7 Types of Validation (cont.) Synthesis of approaches (Whittemore, Chase, & Mendele, 2001). Found four primary criteria: –Credibility: the results are an accurate interpretation of the participant’s meaning –Authenticity: different voices are heard –Criticality: critical appraisal of all aspects of the research –Integrity: the investigators are self-critical

Creswell Qualitative Inquiry 2e 10.8 Validation Strategies (cont.) Prolonged engagement in the field –Builds trust with the participant –The researcher learns from the culture –The researcher can check for misinformation Triangulation –Using multiple sources –Using multiple research methods –Using multiple investigators –Using multiple theories to provide corroborating evidence

Creswell Qualitative Inquiry 2e 10.9 Validation Strategies (cont.) Peer Review –Provides an external check of the research process –Asks questions about methods, meanings, and interpretations –Provides a record of notes by both peer and researcher Negative Case Analysis –Uses disconfirming evidence to get alternative points of view –Enables the researcher to revise the initial hypothesis until all the cases fit

Creswell Qualitative Inquiry 2e Validation Strategies (cont.) Rich, Thick Description –Involves descriptive detail of the participants and the setting –Allows the reader to make decisions regarding any transferability of findings External Audits –Uses an external consultant who examines both the product, process, and account to assess accuracy –Examines whether or not the findings and interpretation are supported by evidence –Uses an auditor not connected to the study or the researcher

Creswell Qualitative Inquiry 2e Validation Strategies Clarifying researcher bias: The researcher comments on past experiences that have likely shaped the interpretation and approach to the study In member checking –Solicits participants’ views on the credibility of the findings and interpretations –A critical technique for establishing credibility –A focus group could be used –The preliminary analysis of descriptions and themes is given to participants NOT transcripts or raw data

Creswell Qualitative Inquiry 2e Reliability Perspectives in Qualitative Research Observations will enhance reliability through: –Use of multiple data analysts –Use of fieldnote conventions such as double quotation marks to indicate verbatim quotes Transcript analysis can enhance reliability through : –Use of multiple data analysts –Use of fieldnote conventions such as double quotation marks to indicate verbatim quotes

Creswell Qualitative Inquiry 2e Reliability Perspectives in Qualitative Research (cont.) Textual analysis can enhance reliability through: –Use multiple coders and calculate intercoder reliability –Use agreed upon standardized categories Interview procedures can enhance reliability through: –Pretest interview protocols –Train interviewers –Use of fixed-choice answers –Reliability checks of open-ended questions

Creswell Qualitative Inquiry 2e Evaluation Criteria: A Methods Approach Howe & Eisenhardt 1990 The data collection and analysis are driven by the research questions Data collection and analysis techniques are completely applied in a technical sense The researcher’s assumptions are made explicit including the researcher’s own subjectivity

Creswell Qualitative Inquiry 2e Evaluation Criteria: A Methods Approach Howe & Eisenhardt 1990 (cont.) The study has overall warrant including being robust and using respected theoretical explanations The study has value both in informing and improving practice

Creswell Qualitative Inquiry 2e Evaluation Criteria: A Postmodern, Interpretive Approach Lincoln 1995 Standards set in inquiry community (guidelines for publication) Positionality (“text” honest and authentic) Community (serves community purposes) Voice (participants heard) Critical subjectivity (researcher heightened self-awareness/creates social transformation)

Creswell Qualitative Inquiry 2e Evaluation Criteria: A Postmodern, Interpretive Approach From Lincoln 1995 (cont.) Reciprocity (between researcher and participants) Sacredness of relationships (respect for participants) Sharing privileges (sharing of rewards with participants)

Creswell Qualitative Inquiry 2e Evaluation Criteria: General, Interpretive Richardson & St. Pierre 2005 Substantive contribution (significant understanding of social life) Aesthetic merit (practices open up text, artistically shaped, not boring) Reflexivity (adequate self-awareness, self- exposure to reader) Impact (affects the reader emotionally, intellectually, moved to action)

Creswell Qualitative Inquiry 2e Evaluation Criteria: Methodological Approach Creswell 2007 Rigorous data collection (multiple forms, extensive data) Consistent with philosophical assumptions of qualitative research (evolving design, multiple perspectives) Employs tradition of inquiry (e.g., Case Study, Grounded Theory, Narrative) Starts with focus on central phenomenon

Creswell Qualitative Inquiry 2e Evaluation Criteria: Methodological Approach Creswell 2007 (cont.) Written persuasively Multiple levels of analysis Narrative engages the reader Includes strategies to confirm accuracy

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Narrative Research The researcher focuses on single individual (or 2 - 3) The researcher collects stories about a significant issue related to this individual’s life The researcher develops a chronology that connects different phrases or aspects of a story The researcher tells a story that restories the story of the study participants

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Narrative Research (cont.) The researcher tells a persuasive story told in a literary way The researcher reports themes that build from the story to tell a broader analysis The researcher brings reflexivity into the study

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Phenomenological Research The researcher conveys an understanding of the philosophical tenants of phenomenology The researcher has a phenomenon to study that is concisely articulated The researcher uses procedures of data analysis in phenomenology The researcher conveys the overall essence of the experience of the participants, which includes a description of the experience and the context in which it occurred

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Grounded Theory Research The researcher studies a process, action, or interaction as the key element of the study The researcher uses a coding process that contains the three central areas of coding found in grounded theory, open coding, axial coding, and selective coding The researcher presents a theoretical model in the form of a visual diagram

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Grounded Theory Research (cont.) The researcher develops a storyline or propositions that connect the categories in the theoretical model and that present further questions to be answered The researcher uses reflexivity or self- disclosure about his or her stance in the study

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Ethnographic Research The researcher identifies a culture- sharing group The researcher provides a detailed description of the culture-sharing group The researcher examines themes related to the culture-sharing group

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Ethnographic Research The researcher addresses field issues that have surfaced The researcher explains how the culture-sharing group works The researcher self-discloses and is reflexive about his/her position

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Case Study Research The researcher provides a clear identification of the case or cases The researcher uses multiple data sources The researcher identifies the type of case study procedures (e.g., intrinsic, instrumental) The researcher provides a clear description of the case

Creswell Qualitative Inquiry 2e Evaluation Guidelines for Case Study Research The researcher identifies themes in the case The researcher makes assertions or generalizations from the case analysis The researcher is reflexive and self-disclosing

Creswell Qualitative Inquiry 2e Chapter 10 Standards of Validation and Evaluation