CS 594: Empirical Methods in HCC A Brief Introduction to Content Analysis Dr. Debaleena Chattopadhyay Department of Computer Science debchatt@uic.edu debaleena.com hci.cs.uic.edu

Content analysis A quantitative approach to analyze unstructured or “qualitative” data, maybe text, audio, or video data. Within the scope of human-centered computing, content analysis implies a series of specific steps aimed at ensuring systematic sampling, coding, and often counting of content to make quantitative inference. Content analysis and GTM are often confused as the same. They are NOT—although they are often considered equivalent. Content analysis is more straightforward and often “easier” than GTM. https://us.sagepub.com/sites/default/files/upm-binaries/53216_ch_12.pdf

Advantages and Disadvantages of Content Analysis Content analysis is quantitative, systematic, and objective technique for describing the observable. We count what is tangible and observable. We cannot count abstract or latent (hidden) notions. Any abstract concept, such as learning outcome, enjoyment, immersion, or group cohesion need to be operationalized in terms of variables that are observable, and thus can be measured. Strength: emphasis on the systematic coding, counting, and analysis of content. Limitation: Most effective if used for comparisons. How we interpret the final counts directly affects the validity of the analysis. Intercoder reliability gives a quantitative measure of reliability.

Steps of doing Content Analysis Develop a hypothesis or research question about the HCI content. Define the content to be analyzed. Sample the universe of content. “Universe” has the same meaning for content as “population” does for people. Also called “coding frames”. Select units for coding (or unit of analysis). Develop a coding scheme. Assign each occurrence of a unit in the sample to a code in the coding scheme. Count occurrences of the coded units and report their frequencies.

Example RQ: Do users ask for permission when sharing content on a open-control public display? Content to be analyzed: videos of classes/ meetings Sampling: all meetings in Continuum stratified sampling— meetings of different sizes and types Units for coding: content showed on a shared display Coding scheme: verbal, non-verbal, explicit, implicit Count Occurrences of the Coded Units and Report Their Frequencies

A simple coding sheet and summary data Permission types Tally Count 1 verbal III 3 2 Non-verbal IIIIIII 7 … N % of time verbal permission asked 5/50 = 10% Explicitly asked for permission 10/50 = 5% Implicitly asked for permission

Interaction analysis Most commonly used in HCI; borrows from the tradition of content analysis of human interactions. For example, three broad categories of group behavior are task-oriented, group-oriented, and self-centered. Task-oriented individuals focus on the group’s work Group-oriented individuals work to ensure that the group remains cohesive Self-centered individuals may refuse to participate or at the other extreme may dominate discussions.

Things you may count Words Sentences Paragraphs Characters Semantics Concepts Themes Images Sequences

Coding scheme Apply an established code or develop your own When developing your own coding scheme, you may use open coding or GTM. http://www.indiana.edu/~educy520/sec6342/week_07/content_analysis_1.pdf

Steps to create a coding scheme (when not using GTM) Review literature Identify constructs Identify features of content that reflect your constructs Choose indicators Finalize coding categories

Good coding categories Exhaustive: there should be a coding category that each recording unit can be placed in. Mutually exclusive: Each recording unit should fit into only one category on a given scoring dimension Derived from a single classification principle: keep conceptually different dimensions of analysis separate Independent: Each category should be independent of other categories Adequate to answer the questions asked of the data should cover the entire concept or nearly so (content validity) Should exclude spurious variables/related concepts that are not supposed to be measured by the category scheme Differences among categories must be meaningful

Qualitative vs. Quantitative content analysis Earlier definition of content analysis is as a quantitative research method. to classify written or oral materials into identified categories of similar meanings Criticized as oversimplification as a result of breaking down text etc. into quantifiable units in the analytic process Hence qualitative content analysis was developed. “a research method for subjective interpretation of the content of text data through the systematic classification process of coding and identifying themes or patterns”

Inductive and Deductive approaches to qualitative content analysis

GTM vs. Content Analysis The use of grounded theory aims to generate a substantive theory that will explain a phenomenon in a specific context and suited to its supposed use. The emphasis in GTM is theory development (Strauss & Corbin, 1994). GTM is appropriate when no theory exists or when a theory exists that is too abstract to be tested. Qualitative content analysis aims to “systematically describe the meaning” of materials in a certain respect that the researcher specified from research questions. Both use coding, but content analysis does not focus on finding relationships among categories or theory building; instead, it focuses on extracting categories from the data. https://core.ac.uk/download/pdf/51087267.pdf https://theschoolofme.wordpress.com/2015/05/11/grounded-theory-vs-content-analysis-or-the-day-i-stopped-worrying-and-learned-to-love-the-datasel/

Similarities Based on naturalistic inquiry Flexibility of using multiple sources of data Systematic steps in analysis Seeking themes through coding process Content to be coded into categories or themes Naturalistic inquiry is an approach to understanding the social world in which the researcher observes, describes, and interprets the experiences and actions of specific people and groups in societal and cultural context.

Differences https://core.ac.uk/download/pdf/51087267.pdf

Differences (cont.…)