Discussion Questions 3 Analyzing Qualitative Data

How do you analyze qualitative data? Data reduction Data display Drawing and verifying conclusions Give examples from Griggs Article.

Data Reduction: Organizing data to make it more meaningful Summarizing and paraphrasing Selecting some themes and exluding others Putting specific instances into larger patterns Quantification into numbers and ranks (coding into categories) Look for differences across groups

Data Display Forms Narrative text and quotations Tables, matrices and graphs Checklists Always arrange in a systematic way Lets look at: Example 1: Tables 1 and 2 (p ) Example 2: Table 3 (p.26) Example 3: Figure 2 (p.28)

Drawing Conclusions Remember the research question and note for regularities and patterns Derive explanations Draw causal flows Come up with propositions

Verifying Conclusions Verifying is checking results by: Triangulation for validity Use different measures for the same sample Use different samples and look for common results Get feedback from respondents Compare two set of respondents known to differ Do not ignore the outliers Replicate findings ina second study

How do you evaluate validity and reliability with qualitative data? Validity: Meaningfulness of results Apparent Internal Instrumental Theoretical Construct Consultative Skyes Article Reliability: Consistency of results Two different researchers? The same researcher? Transferability Confirmability Dependability Interrater agreement methods

Content Analysis What is it? What is the unit of analysis? How is it done? Procedure? Coders? Judges? How to assess reliability in content analysis? Zimmer and Golden Article

Content Analysis Methodology Formulate the research question Select the unit of analysis (content): Single word, theme, symbol Fiction, movie, drama, TV program Entire article, book, etc. Code the content Quantify the coded data Check reliability by: Ability to form categories Extent of agreement between/among coders Analyze and interpret the findings Holsti, O.R. (1969), Content analysis for Social Sciences and Humanities, Wesley Publishing Co.

Content Analysis Methodology Sample: 894 respondents out of 1600 people selected as representative population Research question: Please describe your image of ___________? Stages: Stage 1: Content Analysis of Themes Stage 2: Aggregation of Themes Stage 3: Selection of Final Categories Stage 4: Reliability Analysis

Methodology: Stages in Content Analysis

Reliability Reliability Analysis 2: Researchers reduced 47 dimensions to 10. Inter-rater reliabilities of all 3 pairs of judges were again calculated and found to be significant Judges# of matching (Out of 220) Percent Match A & B % A & C % B & C %

Reliability Reliability Analysis 1: Three judges independently coded 220 themes into 47 dimensions and the probability that any two judges will assign any one theme to the same category by chance alone is found to be extremely small using binomial probability distribution Stage 4: Judges# of matching (Out of 220) Percent Match A & B % A & C % B & C %

Testing the significance of Judge Agreements: Interjudge reliability Binomial Test of proportions (z-Test) Appendix of Zimmer and Golden Article Cohen’s Kappa Reading will be provided Several Others Wiil be discussed as we proceed in the course

Cohen’s Kappa Measure J1 A B C D ETotal A B C D E Total

How to calculate Cohen’s Kappa Probability of Observed Agreements Probability of Expected Agreements Take the difference Divide by expected agreements Formula: Kappa: PO-PE PE