1. Data analysis, interpretation, and presentation
Chapter 8
Data analysis, interpretation, and presentation

2. Outline Qualitative and quantitative Simple quantitative analysis
Simple qualitative analysis
Tools to support data analysis
Using theoretical frameworks
Presenting the findings

3. Qualitative and quantitative
Quantitative data is data that is in the form of numbers, or that can easily be translated into numbers
Examples: number of years of work experience, number of projects, number of minutes to carry out a task

4. Qualitative and quantitative
Qualitative data is data that is difficult to measure, count, or express in numerical terms that make sense

5. Qualitative and quantitative
All the forms of data gathering discussed in previous chapter may result in qualitative and quantitative data
Examples:
on a questionnaire
participants’ age or number of software packages used are quantitative data
comments are qualitative data

6. Data analysis First steps in analyzing data Interviews Questionnaires
data transcription, finding categories or patterns of response
Questionnaires
data cleansing, data filtering (subsets of data)
Observations
Notes write-up, data transcription, synchronization between different data recordings

7. Data analysis Simple quantitative analysis Averages Percentages
Mean (ค่าเฉลี่ย)
Median (มัธยฐาน)
Mode (ฐานนิยม)
Percentages
Data collation (Excel)
Finding out outliers
Finding out patterns (graphical representation)

8. Simple qualitative analysis
Unstructured - are not directed by a script. Rich but not replicable.
Structured - are tightly scripted, often like a questionnaire. Replicable but may lack richness.
Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability.
From:

9. Data analysis Simple qualitative analysis
Identifying recurring patterns or themes
Categorizing data
Analyzing critical incidents

10. Data analysis Simple qualitative analysis
Identifying recurring patterns or themes
Study goals provide an orienting focus for the formulation of themes
A balance between generalness and specificity
Relate to various aspects, e.g., to behavior, to user group, to places and situations

11. Data analysis Simple qualitative analysis Categorizing data
The data is divided up into elements and each element is then categorized
The categorization scheme may arise from the data itself, or it might originate elsewhere in a well-recognized categorization scheme, or a combination of these two approaches
The goal of the study largely determines which categories to use

12. Data analysis Simple qualitative analysis Categorizing data
The challenging aspects are
1) determining meaningful categories that are not overlap each other in any way
2) deciding on the appropriate granularity for the categories, e.g. at word, phrase, sentence, or paragraph level
The categorization scheme used must be reliable -> inter-rater reliability

13. Data analysis Simple qualitative analysis Categorizing data
Content analysis involves categorizing the data and then studying the frequency of category occurrences
Discourse analysis focuses on the dialog, i.e. the meaning of what is said, and how words are used to convey meaning
Conversation analysis focuses on how conversation is conducted, i.e. how conversation starts, how turn-taking is structured, and other rules of conversations

14. Data analysis Simple qualitative analysis Analyzing critical incidents
Two basic principles
1) reporting facts regarding behavior is preferable to the collection of interpretations, ratings, and opinions based on general impressions -> well-planned observation sessions
2) reporting should be limited to those behaviors which, according to competent observers, make a significant contributions to the activity in either desirable or an undesirable way

15. Tools to support data analysis
N6 (formerly NUD*IST) supports
annotation and coding of textual data
Nvivo provides more sophisticated facilities for exploring, merging, and manipulating codes and data than N6
SPSS

16. Using theoretical frameworks
Grounded theory
Distributed cognition
Activity theory

17. Grounded theory The theory derived is grounded in the data
The aim is to develop a theory that fits a set of collected data
In this context, theory means ‘a set of well-developed concepts related through statements of relationship, which together constitute an integrated framework that can be used to explain or predict phenomena’ (Strauss & Corbin, 1998)

18. Grounded theory
Development of a ‘grounded’ theory progresses through alternating data collection and data analysis
Data gathering is driven by the emerging theory
This approach continues until no new insights emerge and the theory is well-developed

19. Grounded theory
Researchers need to maintain a balance between objectivity and sensitivity
Objectivity is needed to maintain accurate and impartial interpretation of events
Sensitivity is required to notice the subtleties in the data and identify relationships between concepts

20. Grounded theory
Main idea is to identify and define the properties and dimensions of relevant categories and then to use these as the basis for constructing a theory
Category identification and definition is achieved by ‘coding’ the data

21. Grounded Theory Three levels of ‘coding’ Open: identify categories
Axial: flesh out and link to subcategories
Selective: form theoretical scheme

22. Grounded Theory Questioning Analysis of a word, phrase, or sentence
Comparisons
Researchers are encouraged to draw on own theoretical backgrounds to inform analysis

23. Distributed Cognition
The people, environment & artifacts are regarded as one cognitive system
Used for analyzing collaborative work
Focuses on information propagation & transformation
From:

24. Distributed Cognition
Results in an event-driven description which emphasizes information and its propagation through the cognitive system under study
The granularity of analysis varies depending on the activities and cognitive system being observed and the research or design questions being asked

25. Distributed Cognition
Ed Hutchins emphasizes that an important part of doing a distributed cognition analysis is to have a deep understanding of the work domain that is being studied
There is no single way of doing a distributed cognition analysis

26. Distributed Cognition
A good way to begin analyzing and interpreting the data collected is to describe the official work practices
Any breakdowns, incidents, or unusual happenings should be highlighted
While writing these observations down, it is good to start posing specific research questions related to them and contemplate further

27. Distributed Cognition
Problems can be described in terms of the communication pathways that are being hindered or the breakdowns arising due to information not propagating effectively from one representational state to another

28. Distributed Cognition
It can show when different technologies and the representations displayed via them are effective at mediating certain work activities and how well they are coordinated
Concepts:
Cognitive system
Communicative pathways
Propagation of representational states

29. Activity Theory
Explains human behavior in terms of our practical activity with the world
Provides a framework that focuses analysis around the concept of an ‘activity’ and helps to identify tensions between the different elements of the system
Two key models: one outlines what constitutes an ‘activity’; one models the mediating role of artifacts

30. Individual model
From:

31. Individual model
Operations – routinized behaviors that require little conscious attention, e.g. rapid typing
Actions – behavior that is characterized by conscious planning, e.g. producing a glossary
Activity – provides a minimum meaningful context for understanding the individual actions, e.g. writing an essay

32. Individual model
There may be many different operations capable of fulfilling an action
Many actions capable of serving the same activity

33. Individual model There is an intimate and fluid link between levels
Actions can become operations as they become more automatic
Operations can become actions when an operation encounters an obstacle, thus requiring conscious planning

34. Individual model
If ‘motive’ changes then an activity can become an action
Activities relate to others while actions may be part of different activities

35. The role of artifacts Artifacts can be physical, e.g.
A book
A stone
Artifacts can be abstract, e.g.
A system of symbols
A set of rules

36. Engeström’s (1999) activity system model
From:

37. Presenting the findings
Only make claims that your data can support
The best way to present your findings depends on the audience, the purpose, and the data gathering and analysis undertaken
Graphical representations may be appropriate for presentation
From:

38. Presenting the findings
Other techniques are:
Rigorous (clear syntax and semantics) notations, e.g. UML
Using stories, e.g. to create scenarios
Summarizing the findings
From:

39. Summary
The data analysis that can be done depends on the data gathering that was done
Qualitative and quantitative data may be gathered from any of the three main data gathering approaches
Percentages and averages are commonly used in Interaction Design

40. Summary
Mean, median and mode are different kinds of ‘average’ and can have very different answers for the same set of data
Grounded Theory, Distributed Cognition and Activity Theory are theoretical frameworks to support data analysis
Presentation of the findings should not overstate the evidence

41. Chapter Review
Chapter 1-7

42. Chapter 1 What is interaction design?
Interaction Designer vs. Software Engineers
User experience
Usability goals
Effectiveness (ประสิทธิผล)
Efficiency (ประสิทธิภาพ)
Safety
Utility
Learnability
Memorability

43. Chapter 1 Usability Criteria Norman’s Design Principles Visibility
Feedback
Constraints
Consistency
Affordances

44. Chapter 2 Understanding the problem space
Are there problems with an existing product or user experience? If so, what are they?
Why do you think there are problems?
How do you think your proposed design ideas might overcome these?
If you have not identified any problems and instead are designing for a new user experience, how do you think your proposed design ideas support, change, or extend current ways of doing things?

45. Chapter 2 Conceptual model Metaphors and analogies
Concepts: task-domain objects, objects’ attributes, operations performed on objects
Relationships between concepts
Mappings between concepts and task-domain the system is designed to support

46. Chapter 2 Interface metaphors Interaction Types Instructing Conversing
Manipulating
Exploring

47. Chapter 3 Users’ cognition Attention Perception and recognition Memory
Learning
Reading, speaking, and listening
Problem-solving, planning, reasoning, decision-making

48. Chapter 3 Cognitive frameworks Mental models Theory of action
Information processing
External cognition
Distributed cognition

49. Chapter 4 Social mechanisms in communication and collaboration
Conversational mechanisms
Coordination mechanisms
Awareness mechanisms

50. Chapter 5 Affective aspects User frustration Persuasive technologies
Sources of user frustration, e.g. error messages, appearance
Persuasive technologies
Anthropomorphism in interaction design, i.e. virtual agents

51. Chapter 6 Interface types Command interfaces WIMP/GUI interfaces
Advanced graphical interfaces
Web-based interfaces
Speech interfaces
Mobile interfaces

52. Chapter 7
Data gathering
Interviews
Questionnaires
Observation