Data Gathering and Analysis

Slides:



Advertisements
Similar presentations
CSCI 4163 / CSCI 6904 – Winter Housekeeping  Register from the waitlist  Course website under construction  Need to form MP1 groups by January.
Advertisements

Chapter 7 Data Gathering 1.
Data gathering. Overview Four key issues of data gathering Data recording Interviews Questionnaires Observation Choosing and combining techniques.
©2011 1www.id-book.com Data analysis, interpretation and presentation Chapter 8.
Chapter 8 Data Analysis, Interpretation and Presentation.
© Done by : Latifa Ebrahim Al Doy Chapter 8 Data analysis, interpretation and presentation and presentation.
Data analysis and interpretation. Agenda Part 2 comments – Average score: 87 Part 3: due in 2 weeks Data analysis.
Data gathering.
Asking users & experts. The aims Discuss the role of interviews & questionnaires in evaluation. Teach basic questionnaire design. Describe how do interviews,
Observing users.
Observing users. The aims Discuss the benefits & challenges of different types of observation. Describe how to observe as an on-looker, a participant,
Chapter 7 GATHERING DATA.
CSCI 4163 / CSCI 6904 – Winter Housekeeping  Write a question/comment about today’s reading on the whiteboard (chocolate!)  Make sure to sign.
Human Computer Interface
Research Methods in Computer Science Lecture: Quantitative and Qualitative Data Analysis | Department of Science | Interactive Graphics System.
Data analysis, interpretation and presentation
Data gathering. Overview Four key issues of data gathering Data recording Interviews Questionnaires Observation Choosing and combining techniques.
Chapter 7 Data Gathering 1.
Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess CPE/CSC 484: User-Centered Design and.
Interviews. Unstructured - are not directed by a script. Rich but not replicable. Structured - are tightly scripted, often like a questionnaire. Replicable.
 Data gathering and analysis  Class activity  Discuss eye tracking assignment.
Chapter 12 Observing Users Li, Jia Li, Wei. Outline What and when to observe Approaches to observation How to observe How to collect data Indirect observation.
Data analysis, interpretation and presentation (Chapter 8 – Interaction Design Text) The kind of analysis can be performed on the data depends on the goals.
AVI/Psych 358/IE 340: Human Factors Data Analysis October 22-24, 2008.
Data analysis and interpretation. Project part 3 Watch for comments on your evaluation plans Finish your plan – Finalize questions, tasks – Prepare scripts.
HISTORY OF HCI REQUIREMENTS DESIGN USER CENTERED DESIGN PROCESS DATA GATHERING EVALUATION Midterm: 10/2 What do you want it to be?
CSCI 4163 / CSCI 6904 – Winter Housekeeping  Clarification about due date for reading comments/questions  Skills sheet  Active listening handout.
Data Analysis, Interpretation, and Presentation Devin Spivey Asmae Mesbahi El Aouame Rahul Potghan.
Observing users. The aims Discuss the benefits & challenges of different types of observation. Describe how to observe as an on-looker, a participant,
AVI/Psych 358/IE 340: Human Factors Data Gathering October 6, 2008.
AVI/Psych 358/IE 340: Human Factors Data Gathering October 3, 2008.
Asking users & experts. The aims Discuss the role of interviews & questionnaires in evaluation. Teach basic questionnaire design. Describe how do interviews,
Data analysis, interpretation, and presentation
12-CRS-0106 REVISED 8 FEB 2013 CSG2C3/ Interaksi Manusia dan Komputer (IMK) TIM Dosen IMK USER CENTERED DESIGN KK SIDE 2/5/20161.
Observing users. What and when to observe Goals & questions determine the paradigms and techniques used. Observation is valuable any time during design.
Data gathering (Chapter 7 Interaction Design Text)
Analysis Perspectives for qualitative data Basing data analysis around theoretical frameworks provides further insight Three such frameworks are: –Grounded.
Evaluation and Assessment of Instructional Design Module #4 Designing Effective Instructional Design Research Tools Part 2: Data-Collection Techniques.
Observation Direct observation in the field –Structuring frameworks –Degree of participation (insider or outsider) –Ethnography Direct observation in controlled.
User centered design IS336 with Dr. Basit Qureshi Fall 2015.
Lecture 4 Supplement – Data Gathering Sampath Jayarathna Cal Poly Pomona Based on slides created by Ian Sommerville & Gary Kimura 1.
Data Analysis, Interpretation and Presentation
Interviews & focus groups
Chapter 7 GATHERING DATA.
Imran Hussain University of Management and Technology (UMT)
Lecture3 Data Gathering 1.
Data Analysis, Interpretation and Presentation
Data analysis, interpretation
Quantitative and qualitative
Chapter 7 Data Gathering 1.
Data analysis and interpretation
Data analysis, interpretation and presentation
Human-Computer Interaction: User Study Examples
Chapter 7 GATHERING DATA.
GATHERING DATA.
Data analysis, interpretation and presentation
Data Analysis, Interpretation and Presentation
From Controlled to Natural Settings
Chapter 7 GATHERING DATA.
Observing users.
Data analysis, interpretation and presentation
Data analysis, interpretation and presentation
Data gathering.
Interviews & focus groups
Establishing Requirements
CSCE 315 – Programming Studio Spring 2017 Project 3, Lecture 3
Human-Computer Interaction: Overview of User Studies
Data Analysis Dr. Sampath Jayarathna Old Dominion University
Chapter 8 DATA GATHERING.
Data Analysis, Interpretation, and Presentation
Presentation transcript:

Data Gathering and Analysis Sampath Jayarathna Cal Poly Pomona Credit for some of the slides in this lecture goes to www.id-book.com

Data recording Notes, audio, video, photographs can be used individually or in combination: Notes plus photographs Audio plus photographs Video Different challenges and advantages with each combination

Interviews Unstructured - are not directed by a script. Rich but not replicable. Structured - are tightly scripted, often like a questionnaire. Replicable but may lack richness. Semi-structured - guided by a script but interesting issues can be explored in more depth. Can provide a good balance between richness and replicability. Focus groups – a group interview

Interview questions Two types: ‘closed questions’ have a predetermined answer format, e.g.. ‘yes’ or ‘no’ ‘open questions’ do not have a predetermined format Closed questions are easier to analyze Avoid: Long questions Compound sentences - split them into two Jargon and language that the interviewee may not understand Leading questions that make assumptions e.g.. why do you like …? Unconscious biases e.g.. gender stereotypes

Enriching the interview process Props - devices for prompting interviewee, e.g. use a prototype, scenario

Questionnaires Questions can be closed or open Closed questions are easier to analyze, and may be distributed and analyzed by computer Can be administered to large populations Disseminated by paper, email and the web Sampling can be a problem when the size of a population is unknown as is common online evaluation

Question and response format ‘Yes’ and ‘No’ checkboxes Checkboxes that offer many options Rating scales Likert scales Strongly disagree (1), disagree(2), undecided (3), agree(4), strongly agree (5) semantic scales (fair…..biased), (weak….strong) 3, 5, 7 or more points Open-ended responses

Observation Direct observation in the field Structuring frameworks Degree of participation (insider or outsider) Ethnography Direct observation in controlled environments Indirect observation: tracking users’ activities Diaries Interaction logging Video and photographs collected remotely by drones or other equipment

Ethnography Ethnography is a philosophy with a set of techniques that include participant observation and interviews Ethnographers immerse themselves in the culture that they study A researcher’s degree of participation can vary along a scale from ‘outside’ to ‘inside’ Analyzing video and data logs can be time-consuming Collections of comments, incidents, and artifacts are made

Online Ethnography Virtual, Online, Netnography Online and offline activity Interaction online differs from face-to-face Virtual worlds have a persistence that physical worlds do not have Ethical considerations and presentation of results are different

Observation in a controlled environment Direct observation Think aloud techniques Indirect observation – tracking users’ activities Diaries Interaction logs Web analytics Video, audio, photos, notes are used to capture data in both types of observations

A section of Google analytics dashboard for id-book.com

Data Analysis Discuss the difference between qualitative and quantitative data and analysis. Enable you to analyze data gathered from: Questionnaires. Interviews. Observation studies.

Quantitative and qualitative Quantitative data expressed as numbers numerical methods to ascertain size, magnitude, amount Qualitative data difficult to measure sensibly as numbers, e.g. count number of words to measure dissatisfaction expresses the nature of elements and is represented as themes, patterns, stories Be careful how you manipulate data and numbers!

Simple quantitative analysis Averages, Percentages Mean: add up values and divide by number of data points Median: middle value of data when ranked Mode: figure that appears most often in the data Be careful not to mislead with numbers! Graphical representations give overview of data Google Forms

Simple qualitative analysis Recurring patterns or themes Emergent from data, dependent on observation framework if used Categorizing data Categorization scheme may be emergent or pre-specified Looking for critical incidents Helps to focus in on key events

Theoretical frameworks for qualitative analysis Basing data analysis around theoretical frameworks provides further insight Two such frameworks are: Grounded Theory Distributed Cognition

Grounded Theory Aims to derive theory from systematic analysis of data Based on categorization approach (called here ‘coding’) Three levels of ‘coding’ Open: identify categories Axial: flesh out and link to subcategories (process of relating categories) Selective: form theoretical scheme (choose one category to be core category) Researchers are encouraged to draw on own theoretical backgrounds to inform analysis

Code book used in grounded theory analysis

Distributed Cognition The people, environment & artefacts are regarded as one cognitive system Used for analyzing collaborative work Focuses on information propagation & transformation

Tools to support data analysis Spreadsheet – simple to use, basic graphs Statistical packages, e.g. SPSS Qualitative data analysis tools Categorization and theme-based analysis Quantitative analysis of text-based data Nvivo http://www.qsrinternational.com/ Atlas.ti http://atlasti.com/ QDA Miner https://provalisresearch.com/

Presenting the findings Only make claims that your data can support The best way to present your findings depends on the audience, the purpose, and the data gathering and analysis undertaken Graphical representations (as discussed above) may be appropriate for presentation Other techniques are: Rigorous notations, e.g. UML Using stories, e.g. to create scenarios Summarizing the findings