1. Qualitative Analysis Educational Skills Workshop 2011-2012 Arianne Teherani, PhD Bridget O’Brien, PhD

2. Creative Commons License Attribution-NonCommercial-Share Alike 3.0 Unported You are free: to Share — to copy, distribute and transmit the work to Remix — to adapt the work Under the following conditions: Attribution. You must give the original authors credit (but not in any way that suggests that they endorse you or your use of the work). Noncommercial. You may not use this work for commercial purposes. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. See http://creativecommons.org/licenses/by-nc-sa/3.0/ for full license.http://creativecommons.org/licenses/by-nc-sa/3.0/

3. Overview  Evaluating qualitative research  Overview of qualitative research  Qualitative data collection methods  Analyzing qualitative data  Combining qualitative and quantitative

4. Exercise 1: Judging Qualitative Research (Analyses) Do you have a clear sense of the context of the study and the role of the researcher(s)? How well do you feel the methodology used in this study captured the research question? Explain some of the strengths and/or limitations. Are there sufficient examples / paraphrasing to support the findings / argument? Has the author convinced you that his/her interpretation is plausible? How? Has the author discussed alternate interpretations?

5. Qualitative Research  Research about lives, behavior, organizational functioning, interactional relationships  Exploratory  Open-ended  Data = Words  Analytic procedures are interpretivist and concurrent with and post data collection

6. Research Multiple Purposes of Qualitative Methods Curriculum Development Instrument Development Evaluation

7. Qualitative Research Questions What are the qualities of an ideal mentor? How are the professional identities of students shaped during clerkship X? How do clerkship students characterize the struggles they face when transitioning to the clerkships? How do clinical teachers characterize students’ struggles? How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school?

8. Qualitative Paradigm Sets of beliefs that guide action Qualitative research takes place within a vast spectrum of paradigms: –Positivism –Post-positivism –Constructivism –Critical Theory –Post-modernism See: Bunniss & Kelly. Research paradigms in medical education. Medical Education, 2010, 44: 358-366.

9. Qualitative Approaches Explore social processes through interpretation of data Three Most Common in Medical Education: Ethnography Grounded theory Case study Phenomenology Hermeneutics Narrative research Action research

10. Sampling Purposeful –Seeking the best sources of information about the phenomenon of interest Theoretical –Seeking the best sources of information to confirm / disconfirm your developing explanatory model “Participants are not recruited on a representative basis, but rather because of their expert knowledge of the phenomenon under inquiry” (Green & Thorogood, 2005)

11. Qualitative Data Collection  Observation  Interview  Focus groups  Open-ended prompts  Artifacts (documents, photos)

12. Observation Spend time with a group of people as they carry out their daily activities, to understand their way of life and how they make sense of the world in which they live. –To understand experiences of group members, their activities, interactions, discussions –To understand how contextual factors influence the activities and decisions of group members

13. Observation  Types of observation –Field Notes –Sketches –Relation diagrams  Continuum of roles that range from complete participation to complete observation  Type of role adopted depends on research question

14. Interviews  Types of Interviews –Structured—verbal questionnaire, scripted questions –Semi-structured—outline of topics guides questions –Informal--conversational, questions arise in context –Retrospective—focuses on past events

15. Focus Group Small groups with moderator Concentrated data on topic of interest in short time span Group interview with insight into interactions on topic (compared to interviews) Important that topic of interest would be easy for participants to discuss in group

16. Data Collection Decisions Methods Types of Data Focus groups Interviews Open ended prompts Observations Collect Artifacts How Interactive? Researcher’s Role? Audio Recording Notes Transcripts Written response Typed response Notes Video Documents Photos / Visual Video

17. Example: How does participation in a teamwork & communication skills curriculum improve small group process in Year 1 of medical school? Observation of small groups pre & post Interviews with small group leaders (retrospective comparison) In-depth interviews with students Focus group with students Written evaluations of group process by students and faculty

18. Exercise 2: Analysis A researcher conducted 5 focus groups at different medical schools. Each group had 3-7 third-year medical students and the students were at least 2 months into the clerkship year. The focus groups were recorded and transcribed. This researcher approaches you for help coding the focus group data and developing a report of the findings. You will work in groups. Read the passages individually and in your group, start the initial analysis discussion. Please document the steps you take to analyze the data and use quote/paraphrases to support your argument.

19. Qualitative Data Analysis Two purposes  To identify and explain themes  To pull loosely related themes together in order to understand and explain relationships

20. Coding: Step 1 The process of breaking down, examining, comparing, conceptualizing, and categorizing data. 1.Conceptualize and label 2.Categorize 3.Name/define your category 4.Validate against examples

21. Example Coding Sheet

22. Coding: Step 2 Putting the data back together in new ways by making connections between categories and sub-categories. –Link sub-categories to categories –Define set of conditions under which categories work –Verify statements against data –Define differences/similarities between categories –Propose and track relationships against data

23. Qualitative Analysis Continuum Content Analysis (quasi-qual) Qualitative Content Analysis Thematic Analysis Grounded Theory Data analysis Count the frequency of words or phrases Generate a list of categories – a coding scheme Code, organize categories into constructs / themes Collect, code, analyze data simultaneously Constant Comparative ReportingFrequencies Discuss themes (sometimes counts) Describe a model to explain a phenomena RigorInter-rater reliability Multiple coders; Check for consistency & reconcile differences Multiple coders Review codes & themes, revise, & reconcile differences Seek out confirming / disconfirming info Multiple coders & reviewers

24. Analysis Considerations Techniques to ensure rigor (credibility or trustworthiness) Interrater agreement Triangulation of data - cross checking Triangulation of sources – member checking Transparency Reproducibility (agreement over time, events or settings)

25. Exercise 3: Writing up Findings You have finished analyzing the focus group data you collected on students’ struggles in the third year. Now you are ready to write up your findings. Please practice writing up one of your findings, making sure to consider the following: –How will you organize the results section? –How will you provide evidence to support your findings? –Are there any techniques you might use to enhance trustworthiness or credibility of your results? –What limitations would be important to mention?

26. Tools for Coding & Data Management Demo of Nvivo Other options: Atlas.ti, HyperRESEARCH, SPSS text analysis

27. Teaching Material References  Dr. Carol Kamin, UIC, Department of Medical Education  Lofland & Lofland. Analyzing social settings.  Rubin, H.J. & Rubin, I.S. (1995). Qualitative interviewing: The art of hearing data. Thousand Oaks, CA: Sage Publications  Strauss, A. & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: Sage Publications  Cote, L., & Turgeon, J. (2005). Appraising qualitative research articles in medicine and medical education. Medical Teacher, 27(1), 71-75.  Also, bibliography will be emailed