HCI Research Project

Research Paradigms Theoretical (in the style of mathematics) –Mathematical deduction –Simulation –Analysis of algorithms Experimental (in the style of psychology) –Hypotheses checked with controlled experimentation

Theoretical Research The researcher: Checks many examples Conjectures a result in the form of a theorem Constructs the proof of the theorem from known results

Experimental Research The researcher: Poses hypotheses –Extensive background reading –Appropriate theoretical model Conducts controlled experiment Performs statistical analysis of data Draws conclusions

Sample Research Questions Why don’t typical users invoke advanced search strategies? What is the preferred method of making a choice from a lengthy list? Note behavior of scroll bar, key index What is the best semantics for the Tab key operating on a Web form? What is the best shopping cart checkout design?

Sample Research Questions (2) What is the task analysis of purchasing an airplane ticket? How should information scent be shown on search reports? How should editing capabilities for a contact list be implemented for a telephone interface? Does providing a site map improve web site navigation?

Questions to Hypotheses Key is measurability Identify –Independent variable(s): those things the researcher can control –Dependent variables: those things the research can measure Aside: What is measurable?

Measuring Things Objectively –Time: to do, to correct, to learn –Counts: tasks performed, errors made –Distance, or other geometrical attributes –Text: density, complexity, reading level Subjectively –Preference –Rating scale

Good Experimental Design Limit number of independent variables Limit number of values for each independent variable Make dependent variables easy to measure Develop good working definitions Control for external bias –Guard against learning and practice

Good Experimental Design (2) Base hypotheses on theoretical model Phrase hypotheses in measurable terms Choose subjects representative of larger group

Example 1 From Goren-Bar, What Do Users Prefer Hypothesis 1: …. users in general will find the user-oriented search method as more suitable for searching the web, compared to search methods that do not consider the user perspective.

Example 1 (continued) Hypothesis 2: Novice and knowledgeable users will not have the same response to both search approaches.

Example 2 From St. Amant, … Navigation and Intelligent Assistance Assume that the precision and recall of an intelligent assistant can be controlled. Hypothesis 1: Is any assistance better than no assistance?

Example 2 (continued) Hypothesis 2: Does task performance change with the precision of the assistance? (Precision = proportion of assistance that is relevant) Hypothesis 3: Does task performance change with the recall of the assistance? (Recall = proportion of relevant information that is provided)

Example 3 From Basgodan, Touch in Shared Virtual Environments Hypothesis: Touch is an essential element in performing a task in a shared virtual environment.

Example 4 From Beck and Shneiderman, HCI course investigations Question: How big should a mouse pad be?

Human Subjects Informed consent Minimized risks Institutional Review Board (IRB) Expedited review Application form

Organization Teams of three –Membership assigned by instructor –Members choose leader, reporter

Items to be Submitted Initial phase –Team membership preferences –Hypotheses, independent and dependent variables –Bibliography –IRB approval form—includes experimental design and materials –Oral presentation of project Investigative phase –Progress reports as required

Items to be Submitted (2) Final phase –Draft of Chapters 1 and 2 of report –Draft of Chapters 3, 4, and 5 of report –Final report: oral and written –Poster abstract and design

Sources for Ideas CHI Conference Proceedings ACM Transactions on HCI Course web site Your frustrating experiences