Completed some Focus Groups, now what do I do? With special thanks to Becki Elkins from Cornell College

Distinctive Features of Qualitative Analysis Focuses on Text transcripts of interviews notes from participant observations Researchers seek to describe the text data in ways that capture the setting or people in their own terms (through their voice and actions)… “Qualitative data analysis is an iterative and reflexive process that begins as data are being collected, rather than after data collection has ceased (Stake, 1995). 9/21/2018

Assumptions… …of constructivist qualitative research Realities are multiple and socially-constructed. There is a mutually-shaping relationship between the knower and the known. Inquiry is value-laden. The purpose of research (assessment) is understanding. (adapted from Whitt, E.J.)

Before you analyze – Trustworthiness is the key

Standards of Rigor or Trustworthiness… Credibility Transferability Dependability Confirmability

Trustworthiness: Credibility Are the researcher’s interpretations credible to the study participants (a.k.a., respondents)? Established through: Triangulation Peer debriefing Member checks Search for alternative evidence / explanations

Trustworthiness: Transferability To what extent are the findings applicable to other settings? Established through: Thick, rich description of the study context, design, and participants Is a decision made by the reader, not an assertion by the researcher

Trustworthiness: Dependability To what extent were the research methods decisions made consistently and appropriately throughout the course of the study? Established through: Audit trail Peer Review Member checks

Trustworthiness: Confirmability Do the results of the study make sense? And, can they be confirmed by others? Established through: Audit trail Peer Review Member checks

Data Analysis

Data Analysis “Data analysis is the process of bringing order, structure, and meaning to the mass of collected data. It is a messy, ambiguous, time-consuming, creative, and fascinating process. It does not proceed in linear fashion; it is not neat. Qualitative data analysis is a search for general statements about relationships among categories of data...” ~ Marshall & Rossman

Two ways to start: Rubric Coding Pre-established framework High need for Consistency Specified learning outcomes Rubric Organic Not sure what you’ll find Want to see what comes out of the data/experience Coding

Generic Data Analysis Through Coding Organize the data Familiarize yourself with the data Generate categories, themes, and patterns Code the data Search for alternative explanations Write the report ~ Rossman & Rallis

Organize the Data Lots of options: Transcripts Content analysis forms Interview logs (Merriam, 1998) Your notes / journals Pictures / documents Pulling survey questions

Organize the Data: Example

Organize the Data: Example

Organize the Data: Example

Familiarize Yourself with the Data Read, read, read your data Transcribe? Listen, listen, listen to your data Hold a conversation with your data Keep notes about your thoughts, ideas, questions

Generate Categories, Themes, Patterns Identify salient themes, recurring ideas, recurring language, patterns of belief Focus: assessment question Interrogate the data Concept mapping – important ideas

Code the Data Iterative process Determining what is evidence of a category or theme What words, phrases, examples illustrate the concepts you identified What have we missed? Mechanics By hand By qualitative software analysis package

Coding: Example Theme: Off-campus study offers opportunities for practical application.

Coding: Example Theme: Off-campus study offers opportunities for practical application.

Search for Alternative Explanations Challenge the patterns that seem apparent What other plausible explanations / interpretations exist?

Common questions Q: Can I double-code? (e.g., 1 person mentions more than 1 code in a response) A: YES Q: Do I have to include everything? A: No but keep it consistent and record your reasoning Q: Should my codes be general or specific? A: It depends on your audience but if in doubt go specific, you can always broaden later

Write the Report Writing is essential to qualitative data analysis “Writing is thinking.” (Wolcott) Fuzzy writing stems from fuzzy thinking. Consider Purpose of the study Audience Format Voice Content How best to “tell the story”

Questions to Consider What was the purpose of the assessment? What were the results of the assessment? What are the limitations of the assessment? Have I adequately outlined them? What conclusions / interpretations did I draw? What does my audience need to hear? What format will fit my audience’s needs?

Report in narrative form One respondent explains that being a diversity campus means having people from different background, ethnicities and races. They acknowledge that while the institution is not perfect, they still consider it diverse. Another echoes these thoughts by saying “Although only somewhat. While we have some diversity of people of different ethnic or racial backgrounds, I feel we are still primarily of the same backgrounds for the majority.” Tip: when inserting quotes, use italics as a way of distinguishing between paraphrased texted and direct quotes.

Number by Theme: EXAMPLE OF REPORTING RESULTS FOR QUALITATIVE (CONSTRUCTED-RESPONSE) ITEMS (con’t) The coded data can be used to provide histograms that easily summarize the responses from students in the study.

Combine the two: When asked why they thought the campus was diverse, 77 respondents indicated that it is having students from different backgrounds. As one respondent explains “A lot of people from different backgrounds go here, and people holding a variety of beliefs/opinions can be found”.

EXAMPLE OF REPORTING RESULTS FOR QUALITATIVE (CONSTRUCTED-RESPONSE) ITEMS For items that involve qualitative data it is helpful to provided a summary of the responses as well as the raw responses. To summarize the data, we usually read through the responses and let categories emerge from the raw data and then we code the data to produce the frequency with which various comments occur.

Visualize it

Which format to choose? Narrative Descriptive Good for program feedback and evaluation Staff tend to like the story quality Words easier to understand Number Decision makers work with numbers Summarizes quickly and can be read fast Track trends more easily Leaves out context

Find a peer reviewer to start Where to start? Organize the data Rubric or coding? Find a peer reviewer to start