1. Completed some Focus Groups, now what do I do?
With special thanks to Becki Elkins from Cornell College

2. Distinctive Features of Qualitative Analysis
Focuses on Text
transcripts of interviews
notes from participant observations
Researchers seek to describe the text data in ways that capture the setting or people in their own terms (through their voice and actions)…
“Qualitative data analysis is an iterative and reflexive process that begins as data are being collected, rather than after data collection has ceased (Stake, 1995).
9/21/2018

3. Assumptions… …of constructivist qualitative research
Realities are multiple and socially-constructed.
There is a mutually-shaping relationship between the knower and the known.
Inquiry is value-laden.
The purpose of research (assessment) is understanding.
(adapted from Whitt, E.J.)

4. Before you analyze – Trustworthiness is the key

5. Standards of Rigor or Trustworthiness…
Credibility
Transferability
Dependability
Confirmability

6. Trustworthiness: Credibility
Are the researcher’s interpretations credible to the study participants (a.k.a., respondents)?
Established through:
Triangulation
Peer debriefing
Member checks
Search for alternative evidence / explanations

7. Trustworthiness: Transferability
To what extent are the findings applicable to other settings?
Established through:
Thick, rich description of the study context, design, and participants
Is a decision made by the reader, not an assertion by the researcher

8. Trustworthiness: Dependability
To what extent were the research methods decisions made consistently and appropriately throughout the course of the study?
Established through:
Audit trail
Peer Review
Member checks

9. Trustworthiness: Confirmability
Do the results of the study make sense? And, can they be confirmed by others?
Established through:
Audit trail
Peer Review
Member checks

10. Data Analysis

11. Data Analysis
“Data analysis is the process of bringing order, structure, and meaning to the mass of collected data. It is a messy, ambiguous, time-consuming, creative, and fascinating process. It does not proceed in linear fashion; it is not neat. Qualitative data analysis is a search for general statements about relationships among categories of data...”
~ Marshall & Rossman

12. Two ways to start: Rubric Coding Pre-established framework
High need for Consistency
Specified learning outcomes
Rubric
Organic
Not sure what you’ll find
Want to see what comes out of the data/experience
Coding

13. Generic Data Analysis Through Coding
Organize the data
Familiarize yourself with the data
Generate categories, themes, and patterns
Code the data
Search for alternative explanations
Write the report
~ Rossman & Rallis

14. Organize the Data Lots of options: Transcripts Content analysis forms
Interview logs (Merriam, 1998)
Your notes / journals
Pictures / documents
Pulling survey questions

15. Organize the Data: Example

16. Organize the Data: Example

17. Organize the Data: Example

18. Familiarize Yourself with the Data
Read, read, read your data
Transcribe?
Listen, listen, listen to your data
Hold a conversation with your data
Keep notes about your thoughts, ideas, questions

19. Generate Categories, Themes, Patterns
Identify salient themes, recurring ideas, recurring language, patterns of belief
Focus: assessment question
Interrogate the data
Concept mapping – important ideas

20. Code the Data Iterative process
Determining what is evidence of a category or theme
What words, phrases, examples illustrate the concepts you identified
What have we missed?
Mechanics
By hand
By qualitative software analysis package

21. Coding: Example
Theme: Off-campus study offers opportunities for practical application.

22. Coding: Example
Theme: Off-campus study offers opportunities for practical application.

23. Search for Alternative Explanations
Challenge the patterns that seem apparent
What other plausible explanations / interpretations exist?

24. Common questions
Q: Can I double-code? (e.g., 1 person mentions more than 1 code in a response) A: YES Q: Do I have to include everything? A: No but keep it consistent and record your reasoning Q: Should my codes be general or specific? A: It depends on your audience but if in doubt go specific, you can always broaden later

25. Write the Report Writing is essential to qualitative data analysis
“Writing is thinking.” (Wolcott)
Fuzzy writing stems from fuzzy thinking.
Consider
Purpose of the study
Audience
Format
Voice
Content
How best to “tell the story”

26. Questions to Consider What was the purpose of the assessment?
What were the results of the assessment?
What are the limitations of the assessment? Have I adequately outlined them?
What conclusions / interpretations did I draw?
What does my audience need to hear?
What format will fit my audience’s needs?

27. Report in narrative form
One respondent explains that being a diversity campus means having people from different background, ethnicities and races. They acknowledge that while the institution is not perfect, they still consider it diverse.
Another echoes these thoughts by saying “Although only somewhat. While we have some diversity of people of different ethnic or racial backgrounds, I feel we are still primarily of the same backgrounds for the majority.”
Tip: when inserting quotes, use italics as a way of distinguishing between paraphrased texted and direct quotes.

28. Number by Theme:
EXAMPLE OF REPORTING RESULTS FOR QUALITATIVE (CONSTRUCTED-RESPONSE) ITEMS (con’t)
The coded data can be used to provide histograms that easily summarize the responses from students in the study.

29. Combine the two:
When asked why they thought the campus was diverse, 77 respondents indicated that it is having students from different backgrounds. As one respondent explains “A lot of people from different backgrounds go here, and people holding a variety of beliefs/opinions can be found”.

30. EXAMPLE OF REPORTING RESULTS FOR QUALITATIVE (CONSTRUCTED-RESPONSE) ITEMS
For items that involve qualitative data it is helpful to provided a summary of the responses as well as the raw responses. To summarize the data, we usually read through the responses and let categories emerge from the raw data and then we code the data to produce the frequency with which various comments occur.

31. Visualize it

32. Which format to choose? Narrative Descriptive
Good for program feedback and evaluation
Staff tend to like the story quality
Words easier to understand
Number
Decision makers work with numbers
Summarizes quickly and can be read fast
Track trends more easily
Leaves out context

33. Find a peer reviewer to start
Where to start?
Organize the data
Rubric or coding?
Find a peer reviewer to start