1. Qualitative & Mixed Methods
Module 14

2. Overview Qualitative Research Mixed Methods Broad view of features
Many traditions
Contrasts with quantitative research
Qualtitative method roadmap
Data collection methods
Standards of evaluation
Mixed Methods
Approaches to mixing
Examples

3. Research Methods
In the social sciences, there are 3 generic types of research methods
Quantitative methods
Qualitative methods
‘Mixed methods’ blending quantitative and qualitative

4. Qualitative Research
Focus on making sense of or interpreting social or human phenomena: experiences, behaviors, organizational functioning, social movements, interactional relationships, descriptions of culture
Phenomena are studied in their natural settings
Investigators interpret phenomena in terms of the meanings people bring to them
Denzin and Lincoln, 2000.

5. Qualitative work: Common features
The researcher is the primary instrument for data collection and analysis
Involves fieldwork
Attends to low inference descriptive data

6. Qualitative Methods - Traditions
Purpose
Methods
Allied Discipline
Ethnography
Cultural description
Intensive fieldwork
Anthropology
Phenomenology
Explore lived experience
Analysis of cases to formulate meaning
Philosophy
Psychology
Biography
Meaning of experience as recorded through story
Interviews and documents exploring the life of an individual
Literature
Literary Criticism
Grounded Theory
Theory development, provide an explanation
Constant comparative method
Sociology

7. Qualitative Research Attends to description of how vs. how many
Is an inductive and open process: the investigator builds concepts, hypotheses, and theories from analysis of the data.
The data collection and analysis process is iterative.
The investigator returns to the data with new questions and ideas to explore until there is a very deep understanding of the phenomenon / issue.

8. Qualitative Quantitative
Assumptions
Reality is socially constructed
Primacy of subject
*Variables are complex, interwoven and difficult to measure
Relationships between variables are generally described as observed patterns or cases
Assumptions
Social facts have an objective reality
Primacy of method
*Variables are measured with existing tools
Relationships between variables can be assessed using standard statistics

9. Qualitative Quantitative
Purpose
Contextualization
Interpretation
Understanding peoples’ perspectives
Researcher’s Role
Personal involvement and partiality
Emphatic understanding
Design
Flexible and emergent
Purpose
Generalizable findings
Prediction
Causal explanations
Researcher’s Role
Detachment and impartiality
Objective portrayal
Design
a priori and inflexible

10. Qualitative Quantitative
Purpose
Contextualization
Interpretation
Understanding peoples’ perspectives
Researcher’s Role
Personal involvement and partiality
Emphatic understanding
Design
Flexible and emergent
Purpose
Generalizable findings
Prediction
Causal explanations
Researcher’s Role
Detachment and impartiality
Objective portrayal
Design
a priori and inflexible

11. Qualitative Quantitative
Approach
End with hypotheses and grounded theory
Emergence and portrayal
Naturalistic
Inductive
Searches for patterns
Seeks pluralism, complexity
Minor use of numerical indices
Thick description through writing
Approach
Begins with hypotheses and theories
Manipulation and control
Experimentation
Deductive
Component analysis
Seeks consensus, the norm
Reduces all data to numerical indices
Precise technical language, numerical presentation

12. Questions?

13. The Circle of Inquiry Identify Knowledge Gap
Search for Existing Information
Disseminate & Implement
Focus the Study Question
Analyze & Interpret Results
Collect Data
Design the Study
Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company,

14. Qualitative Data Collection Methods
In-depth interviews
Narratives, detailed responses to probing questions
Direct observation
Field notes containing descriptions of activities, behaviors, actions, interactions, and processes
Document analysis
Written documents: records, memoranda, correspondence, reports, diaries

15. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating
Crabtree & Miller, Doing Qualitative Research nd Ed.

16. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
Representing the account
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating

17. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
Actively reflecting, Specifically how what is happening is influencing & shaping the interpretive process. Determining next steps.
Representing the account
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating

18. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
3 general strategies:
- Template
- Editing
- Immersion /Crystalization
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating

19. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
Representing the account
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating

20. Qualitative Methods Road Map
Gathering Process
Sampling / Data collection Describing Organizing
Representing the account
Connecting
Corroborating / Legitimating

21. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
Representing
the account
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating
Analysis Process

22. Qualitative Methods Road Map
Interpretive Process
Sampling / Data collection Describing Organizing
Representing
the account
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating

23. Qualitative Methods Road Map
Sampling / Data collection Describing Organizing
Reflexivity
Representing the account
Do this as 2 slides – so that stepwise things show up – complex and lots of new concepts.
Connecting
Corroborating / Legitimating

24. Questions?

25. Standards of Qualitative Research
God is in the details. R. Preston, 1994, p. 298
The devil is in the details. Popular adage.
Frankel, Standards of Qualitative Research. In Doing Qualitative Research, 2nd Ed.

26. Standards for Evaluation
Quantitative
Internal Validity
External Validity
Reliability
Objectivity
Qualitative
Credibility / Trustworthiness
(Transferability)
Confirmability, dependability
Engagement, reflexivity

27. Trustworthiness Ways to achieve trustworthiness:
Member checks: recycling interpretation back to the key informants
Searching for disconfirming evidence
Triangulation: multiple data sources and multiple methods
Thick description: a thorough description of the context of the study

28. Confirmability Ways to address confirmability:
Collection of data in ways that allow for audits
Audio recordings
Full transcripts of interviews
Collection of low inference observational data
Engaging a team approach
Independent auditors

29. Reflexivity Ways to address reflexivity:
Document beliefs, framework, theories underlying approach to the problem before beginning the data collection.
Actively journal reflections, possible biases/ limitations to ‘lens’ or ‘gaze’. Document how to overcome limitations to more fully examine phenomena.
Engage other perspectives in team analysis.
Report reflexivity in final product

30. Summary: What’s Involved in Qualitative Inquiry
Spend time in the field gathering data
Engage in data analysis to reduce raw data to themes or categories; complex & time-consuming
Write extensively to show multiple perspectives in order to substantiate findings
Participate in a form of research that does not have firm guidelines or specific procedures, and is evolving and constantly changing
Source: Creswell JW Qualitative inquiry and research design: choosing among five traditions. Thousand Oaks, CA: Sage.

31. Qualitative/Quantitative
Both approaches are highly respected
When done well, both contribute equally to the knowledge base
When possible, researchers should blend the methods
enriches the data to enable deeper understanding
validates the conclusions from more than one perspective

32. Summary of Quantitative and Qualitative features
Role of researcher
Detached, objective
Engaged
Purpose
Test hypothesis
Describe;
Develop theory
Data collection methods
A priori, structured
Flexible, iterative
Data analysis
Deductive, statistical
Inductive, iterative leading to more data collection and revision of question
Type of data
Numbers
Predominately Words
Product
Status of hypothesis
Rich description; Theory development;
A book

33. Mixed Methods
General consensus that no longer a quantitative vs. qualitative methods – both are necessary.

34. Mixed Methods
Attack the research problem with an arsenal of methods that have non-overlapping weaknesses in addition to complementary strengths.
J Brewer & A Hunter Foundations of multimethod research: Synthesizing styles. Sage page 4.

35. Mixed Methods Scope of mixing methods – within study
– within line of inquiry
– within broad topic area

36. Approaches to mixing methods
Sequential studies
Quantitative Qualitative
Qualitative Quantitative

37. Approaches to mixing methods
Mixing methods in the same study
Dominant/ Less Dominant
Secondary method plays a small role
Triangulation
Equal weight, both Quantitative and Qualitative data collected & analyzed in a complementary manner.

38. Examples of mixing
Data transformation – convert data of one type to the other can be analyzed together.
Typology development – one type of data used to develop a typology that is used to drive analysis with other type of data.
Extreme case analysis – pursue data collection or analysis of data of the other type with the intent of refining the initial explanation for the extreme case.

39. Example line of inquiry

40. Direct Observation of Primary Care
To accurately measure, using direct observation, the level of preventive services delivered to eligible patients, and to document the context of the competing demands of the primary care medical encounter
To determine the optimal non-observational method…
To use a multimethod approach to identify and explore which characteristics of the medical encounter, physicians, patients and the office environment are associated with delivery of preventive services to eligible patients

41. Direct Observation of Primary Care
Cross-sectional observation of 84 family practices & 4454 patient visits to 138 physicians in Ohio
Direct Observation
Davis Observation Code
Checklists
Medical Record Reviews
Patient Exit questionnaire
Billing Data
Practice Environment Checklist
Ethnographic Fieldnotes

42. Prevention & Competing Demands
PI – Crabtree
Aim: Understand primary care practice from the ground up through intensive direct observation of the practice environment and patient care to better understand how these factors affect preventive service delivery

43. Prevention & Competing Demands
In-depth multimethod comparative case study of 18 family practices & 1,600 visits to 56 clinicians in Nebraska
Longer direct observation of practice environment recorded in checklists and field notes (4-8 weeks of observation)
Direct observation of 30 encounters/clinician recorded in checklists and field notes
Chart audits of patients who were observed
Interviews of all clinicians, most staff, some community members

44. Study To Enhance Prevention by Understanding Practice (STEP-UP)
Randomized clinical trial of 80 family practices in Ohio
Multimethod assessment (MAP) of values, structures, and processes
Patient survey and medical record review to assess preventive service delivery at 6 month intervals
Practice-individualized intervention

45. Doctor-patient communication study
Specifically this study aims to:
1. Characterize the way a ‘teachable moment’ arises and is constructed within the patient-physician interaction.
2. Identify factors that enhance or impede the use of illness visits to provide health behavior advice.
3. Determine the effectiveness of a ‘teachable moment’ for increasing: patient recall of advice, motivation to modify behavior, and change in health behavior.

46. Doctor-patient communication study Graphic representation of mixing

47. Doctor-patient communication study Outputs
Primary paper #1 Discovery of the teachable moment communication process. Primary paper #2 Quantitative testing the association of TM vs. other kinds of advice with outcomes

48. Doctor-patient communication study Embedded example
National Heart Lung & Blood guideline for assessment and treatment of obesity.
- coding template of key guideline elements
- audio recordings, apply template
- findings from quantitative prompted qualitative examination. (Unplanned)

49. Mixing methods
Expertise/training in methods you want to mix - team members - consultants - cross training - learning by doing*

50. Summary

51. The Power of Qualitative Data
Stories are powerful and memorable.
have a concrete, vivid, meaningful flavor that often proves far more convincing and memorable to a reader – another researcher, a policymaker, a practitioner
Reports include explanations of processes in context and derive explanations
Enable investigators to address complex social issues where little is understood. Methods push beyond initial conceptions to generate or revise conceptual frameworks
Source: Miles MB, Huberman AM Qualitative data analysis: an expanded sourcebook. 2nd ed. Thousand Oaks, CA: Sage.

52. Making Decisions about which Methods to Use
What is the research question?
What is the goal / purpose of the study?
What kinds of information are needed to answer the Q?
When is the information needed?
What resources are available to conduct the study?
Given answers to the preceding questions, what methods are appropriate?
Source: Patton MQ Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage. p. 12

53. Questions?

54. Sue Flocke saf6@case.edu

55. Qualitative Analysis
Affixing codes to interview data and field notes from observations
Noting reflections in the margins
Sorting and sifting through these materials to identify similar phrases, relationships, patterns, themes, commonalities, & differences
Isolating patterns, processes, commonalities, & differences and incorporating methods to further explore them into the next wave of data collection
Gradually developing a small set of generalizations about what consistently appears in the data
Confronting those generalizations with a formalized body of knowledge in the form of constructs or theories

56. Themes of Qualitative Inquiry
Definition
Naturalistic Inquiry
Studying real-world situations as they unfold naturally; non-manipulative; unobtrusive; lack of pre-determined constraints on outcomes
Holistic perspective
The whole phenomenon is understood as a complex system that is more than the sum of its parts; not reduced to a few discrete variables and cause and effect relationships
Personal contact and insight
Direct, close contact with people, situation, or phenomenon under study; researcher’s insights are important to inquiry and understanding
Source: Patton MQ Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage.

57. Themes of Qualitative Inquiry
Definition
Dynamic systems
Attention to process; assumes change is constant and ongoing
Unique case orientation
Assumes each case is special and unique; captures details of individual cases; cross-case analysis follows from individual case studies
Context sensitivity
Places findings in a social, historical, and temporal context; dubious of generalizations from other times and settings
Empathic neutrality
Overriding objective is to understand the subject of investigation in all its complexity rather than proving, advocating, or advancing personal agendas
Design flexibility
Open to adapting inquiry as understanding deepens and situations change; avoids getting locked into rigid designs; pursues new paths of discovery as they emerge
Source: Patton MQ Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage.