1. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Research Methods in Human-Computer Interaction Chapter 11- Analyzing Qualitative Data

2. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Overview Introduction Stages of qualitative analysis Grounded theory Content analysis Analyzing text content Analyzing multimedia content

3. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Stages of qualitative analysis Identify components of the substance Study properties and dimensions of each component Understand and make inference about the substance

4. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Grounded theory An inductive research method

5. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Procedures of grounded theory open coding development of concepts grouping concepts into categories formation of a theory

6. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Grounded theory Advantages – a systematic approach to analyzing qualitative, mostly text-based, data, – generating theory out of qualitative data that can be backed up by ample evidence of the coding – Interplay between data collection and analysis

7. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Grounded theory Disadvantages – Researcher can be overwhelmed by the details of the data – The theory generated is hard to evaluate – Findings may be subject to bias

8. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Content analysis A more specific view: a systematic, replicable technique for compressing many words of text into fewer content categories based on explicit rules of coding A broader view: any technique for making inferences by objectively and systematically identifying specified characteristics of messages

9. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Content categories

10. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Preparing for content analysis Define the data set Define the population Clean up the data Understand the context of the data

11. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Analyzing text data A priori coding – Identify coding categories – Coding – Reliability check Emergent coding – Multiple coders identify coding categories based on subset of data – Consolidate category list – Code a subset of data – Reliability check – Repeat the process until satisfactory result is met – Code the rest of the data

12. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Identify coding categories Theoretical framework Researcher-denoted concepts In-vivo codes Building a code structure (nomenclature)

13. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Coding the text Look for key items

14. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Coding the text Ask questions about the data Making comparisons – Between different coding category – Between different participant group – Between existing data and previous literature Using computer software

15. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Ensure high quality analysis Validity – constructing a multi-faceted argument in favor of your interpretation of the data – Constructing a database – Data source triangulation – Interpretation should account for as much as possible of the data – Alternative interpretations may also help

16. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Ensure high quality analysis Reliability check – Stability also called intra-coder reliability examines whether the same coder rates the data in the same way throughout the coding process – Reproducibility also called inter-coder reliability or investigator triangulation examines whether different coders code the same data in a consistent way

17. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Reliability measures Percent agreement Cohen’s Kappa: K = (Pa − Pc)/(1 − Pc) % Agreement = Number of cases coded the same way Total number of cases

18. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Reliability check Agreement matrix

19. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Reliability check Interpretation of Cohen’s Kappa

20. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Subjective vs. objective coder Subjective coders – Knowledge and experience can help interpret the data – Less training required – May cause inflated reliability Objective coders – Less likely to cause inflated reliability – Lack of knowledge affect the ability to understand the data – More training required

21. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 Analyzing multimedia content The supporting techniques and methods are less mature compared to text data analysis Approaches: – Manual analysis: highly labor intensive and time consuming More accurate – Completely automated analysis: Faster, less amount of work Highly inaccurate – Partially automated approach Combines the advantages of the manual process and the completely automated process

22. ©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11 End-of-chapter Summary Discussion questions Research design exercise