1. cpsc 344/544
2010/11 Term 1
Qualitative analysis coding (intro exercises) affinity diagrams CSCI 4163/ winter 2014

2. 2 general research approaches
deductive approach vs inductive approach

3. deductive research approach
THEORY
top-down approach
HYPOTHESIS
Top-down approach, waterfall
OBSERVATION
CONFIRMATION

4. inductive research approach
THEORY
TENTATIVE HYPOTHESIS
Bottom up approach
End goal is describe observations, formulate a theory..
PATTERN
OBSERVATION
bottom-up approach

5. deductive or inductive Deductive – theory testing
Inductive – theory building

6. Exercise 1 Open coding Inductive analysis Exploratory research
Theory building research

7. Exercise 2 Coding with pre-defined categories Deductive analysis
cpsc 344/544 - class 04a W1
Exercise 2
Coding with pre-defined categories
Deductive analysis
Theory Testing

8. The Qualitative Analytical Process

9. Agenda 2 Main QA approaches Coding exercise 1 Coding exercise 2
Slides on Qualitative Analysis
Brainstorming Exercise
Affinity Diagramming Exercise

10. 1. Analysis Considerations
Words
Context (tone and inflection)
Internal consistency (opinion shifts during groups)
Frequency and intensity of comments (counting, content analysis)
Specificity
Trends/themes
Iteration (data collection and analysis is an iterative process moving back and forth)

11. 2. The Procedures Coding/indexing Categorisation Abstraction
Comparison
Dimensionalisation (relationships)
Integration
Iteration
Refutation (subjecting inferences to scrutiny)
Interpretation (grasp of meaning - difficult to describe procedurally)

12. Qualitative Inquiry - Purpose
The purpose of qualitative inquiry is to produce findings. The Data Collection process is not an end in itself. The culminating activities of qualitative inquiry are analysis, interpretation, and presentation of findings.

13. Qualitative Inquiry - Challenge
To make sense of massive amounts of data, reduce the volume of information, identify significant patterns and construct a framework for communicating the essence of what the data reveal

14. Qualitative Inquiry - Problem
‘…have few agreed-on canons for qualitative data analysis, in the sense of shared ground rules for drawing conclusions and verifying sturdiness’ (Miles and Huberman, 1984)

15. The Creativity of Qualitative Inquiry
‘..the human element of qualitative inquiry is both its strength and weakness - its strength is fully using human insight and experience, its weakness is being so heavily dependent on the researcher’s skill, training, intellect, discipline, and creativity. The researcher is the instrument of qualitative inquiry, so the quality of the research depends heavily on the qualities of that human being’ (Patton, 1988)

16. The Science and Art of Qualitative Inquiry (Patton, 1988)
The scientific part is systematic, analytical, rigorous, disciplined, and critical in perspective
The Art
The artistic part is exploring, playful, metaphorical, insightful, and creative

17. cpsc 344/544 - class 04a W1
Critical Thinking
‘Critical Thinking calls for a persistent effort to examine any belief or supposed form of knowledge in the light of the evidence that supports it and the further conclusions to which it tends’ (Glaser, 1941)
or more simply!
Critical Thinking means weighting up the arguments and evidence for and against.

18. Critical Thinking Key points (Glaser, 1941):
cpsc 344/544 - class 04a
Critical Thinking W1
Key points (Glaser, 1941):
Persistence: Considering an issue carefully and more than once
Evidence: Evaluating the evidence put forward in support of the belief or viewpoint
Implications: Considering where the belief or viewpoint leads; what conclusions would follow; are these suitable and rational; and if not, should the belief or viewpoint be reconsidered

19. Guidance for Creative Thinking
Be open
Generate options
Divergence before convergence
Use multiple stimuli - triangulate
Side track, zig-zag, and circumnavigate
Change patterns of thinking
Make linkages
Trust yourself
Work and play at it
1. Creativity begins with openness to multiple possibilities
2. There is always more than one way to think about or do something
3. Begin by exploring a variety of directions and possibilities before focusing on the details. Branch out, go on mental excursions, and brainstorm multiple perspectives before converging on the most promising
4. Creativity training often includes exposure to many different avenues of expression: drawing, music, role-playing, constructing futuristic scenarios. Synthesising through triangulation is a creative use of multiple stimuli.
5. Creativity is seldom a result of purely linear and logical induction or deduction. The creative person explores back and forth, round and about, in and out, over and under.
6. Habits, standard, operating procedures, and patterned thinking are barriers to creativity. Become aware of and change your patterned ways of thinking and behaving
7. Many creative exercises include practice in learning how to connect the seemingly unconnected.
8. Self-doubt short circuits creative impulses.
9. Creativity is not all fun. It takes hard work, background research, and mental preparation
10. Creativity is not all work. It can be and should be play and fun.

20. The Credibility of Qualitative Analysis
Rigorous techniques and methods for gathering high- quality data that is carefully analysed, with attention to issues of validity, reliability, and triangulation
The credibility of the researcher, which is dependent on training, experience, track record, status, and presentation of self
Philosophical belief in the phenomenological paradigm, that is, a fundamental appreciation of naturalistic inquiry, qualitative methods, inductive analysis and holistic thinking

21. A Credible Qualitative Study
A credible qualitative study needs to address the following issues:
What techniques and methods were used to ensure the integrity, validity, and accuracy of the findings
What does the researcher bring to study in terms of qualifications, experience, and perspective
What paradigm orientation and assumptions ground the study

22. Principles of Analysing Qualitative Data
cpsc 344/544 - class 04a W1
Principles of Analysing Qualitative Data
Proceed systematically and rigorously (minimise human error)
Record process, memos, journals, etc.
Focus on responding to research questions
Appropriate level of interpretation appropriate for situation
Time (process of inquiry and analysis are often simultaneous)
Seek to explain or enlighten
Evolutionary/emerging
Evolution and emergence
Interpretation and consequent understanding tends to evolve as the researcher explores, connects and re-reads data. Initial ideas are tentative and final theories emerge only after repeated and exhaustive exploration.

23. Qualitative Research: Common Features of Analytic Methods (Miles & Huberman,1994)
Affixing codes to a set of field notes drawn from data collection
Noting reflections or other remarks in margin
Sorting or shifting through the materials to identify similar phrases, relationships between themes, distinct differences between subgroups and common sequences

24. Qualitative Research: Common Features of Analytic Methods (Miles & Huberman,1994)
Isolating patterns and processes, commonalties and differences, and taking them out to the filed in the next wave of data collection
Gradually elaborating a small set of generalisations that cover the consistencies discerned in the data base
Confronting those generalisations with a formalised body of knowledge in the form of constructs or theories

25. Or to pre-existing designs back here
cpsc 344/544 - class 04a W1
Interface Design and Usability Engineering
Articulate:
who users are
their key tasks
Brainstorm designs
Refined designs
Completed designs
Goals:
Task centered system design
Participatory design
User-centered design
Psychology of everyday things
User involvement
Representation & metaphors
Graphical screen design
Interface guidelines
Style guides
Participatory interaction
Task / Cognitive scenario walk-through
Evaluate
Usability testing
Heuristic evaluation
Field testing
Methods:
low fidelity prototyping methods
high fidelity prototyping methods
Or to pre-existing designs back here
Products:
User and task descriptions
Throw-away paper prototypes
Testable prototypes
Alpha/beta systems or complete specification

26. brainstorming the point is: the point is NOT:
cpsc 344/544
2010/11 Term 1
brainstorming
the point is:
to generate MANY, WIDE-RANGING ideas
 nutty and absurd are GOOD. go for the extremes (to get out of the rut)
 riff off other’s ideas.
the point is NOT:
to generate excellent, complete, feasible ideas … pressure stifles
to develop or critique ideas … go wide. deep is for later.

27. cpsc 344/544
2010/11 Term 1
process
prepare a list of topics / questions ahead of time; or in a preliminary brainstorm
facilitator takes team through list of topics switch topic when energy ramps down
Note taker takes notes (very important)
switch roles so everyone can play
ground rules
Follow up

28. brainstorming is like popcorn
cpsc 344/544
2010/11 Term 1
brainstorming is like popcorn
It is better to draw this on board or tablet, explaining as you go; scanned an image to make it clear to other presenters
- draw a plot of idea frequency as a function of time (gaussian)
- then draw another plot, with a series of guassians, each picking up where the previous one died down. “switch” points for facilitator
- draw attention to this metaphor

29. cpsc 344/544
2010/11 Term 1
ground rules
Postpone and withhold your judgment of ideas: never criticize
Encourage wild and exaggerated ideas
Quantity counts at this stage, not quality
Switch topics when the popcorn slows down
Build on the ideas put forward by others
Every person and every idea has equal worth
Elect a facilitator (calls switches) and a note- taker (one thought per post it!)

30. Form 5 groups assign a facilitator, note taker
cpsc 344/544
2010/11 Term 1
Form 5 groups assign a facilitator, note taker
Problem:
How to design the user interface for a car proximity detection system
Brainstorm 3 aspects of the problem: (e.g., physical form factor, safety issues, input techniques, etc.)
go: 5 minutes
HIDE SLIDE IN POSTED VERSION

31. follow up collect the notes
cpsc 344/544
2010/11 Term 1
follow up
collect the notes
go through carefully, with judgment turned on
look for
interesting, surprising ideas that might work
ideas that will combine well
promising directions on which you should brainstorm more
keep your notes. at a later design stage, come back to them and see if anything else has become useful in the meantime.

32. work consolidation: abstracting specific insights
cpsc 344/544
2010/11 Term 1
work consolidation: abstracting specific insights
one tool: the affinity diagram
can use to “consolidate” insights from collected or generated data. for example:
brainstorming about design problems  categories of problems
brainstorming about design ideas  categories of ideas
comments from users  categories of desirable / successful features

33. how do you make an affinity diagram?
cpsc 344/544
2010/11 Term 1
how do you make an affinity diagram?
team writes down all data & insights on post-it notes; be sure you can link the post-it detail back to its source!
stick one post-it on the wall a whiteboard or big sheet of butcher paper is best
arrange the other post-its around it, grouping by affinity to each other. iteration will be required.
look at each group and see what it has in common; name and describe each group.
“snapshot” the result for documentation
digital photo  your design website or notebook
transfer post-its onto xerox paper, 1 sheet / notes-cluster  scan  website

34. why does an affinity diagram work?
cpsc 344/544
2010/11 Term 1
why does an affinity diagram work?
use physical arrangement/proximity to understand connections
openness to serendipity
low cost to rearrange ideas
many variants:
arrange along axes rather than by affinity
tie causes to effects
group evidence under assertions

36. affinity diagram exercise
cpsc 344/544
2010/11 Term 1
affinity diagram exercise
Now take your notes from the earlier brainstorming and create an affinity diagram
go: 8 minutes
Start at 10:40

37. cpsc 344/544
2010/11 Term 1
debrief
End at 10:50