1. Chapter 25 Analysis and interpretation of user observation evaluation data

2. UIDE Chapter 25

3. Introduction: How to Analyze and Interpret Data from Your Evaluation Collating the Data Summarizing the Data –Extract key comments from the collated data.

4. UIDE Chapter 25 Reviewing the Data to Identify Usability Problems Usability Defects (characteristics) –Irritates or confuses the user –Makes a system hard to install, learn, or use –Causes mental overload for the user –Causes poor user performance –Violates design standards or guidelines –Reduces trust or credibility of the system –Tends to cause repeated errors –Could make the system hard to market

5. UIDE Chapter 25 Working with Quantitative Data Tabulations, charts, and rankings for visual rep. Descriptive statistics: mean(average), median(middle value), mode(most common value) Inferential statistics: tests of statistical significance yielding probability.

6. UIDE Chapter 25 Working with Quantitative Data Standard Deviation –For each value x, subtract the overall avg (x) from x, –then multiply that result by itself (otherwise known as determining the square of that value). –Sum up all those squared values. –Then divide that result by (n-1). –Then, find the square root of that sum Ref:http://www.robertniles.com/stats/stdev.shtml

7. UIDE Chapter 25 Working with Quantitative Data Standard Deviation

8. UIDE Chapter 25 Working with Quantitative Data Standard Deviation –One standard deviation away from the mean in either direction on the horizontal axis (the red area on the above graph) accounts for somewhere around 68 percent of the data in this group. Two standard deviations away from the mean (the red and green areas) account for roughly 95 percent of the data. And three standard deviations (the red, green and blue areas) account for about 99 percent of the data Ref:http://www.robertniles.com/stats/stdev.shtml

9. UIDE Chapter 25 Working with Quantitative Data Coefficient of Variation –A measure of variability (or dispersion) of a distribution that is equal to the standard deviation expressed as a percentage of the mean standard deviation / mean x 100% –Used to measure the imprecision in survey estimates introduced by sampling. A coefficient of variation of 1 percent would indicate that an estimate could vary slightly due to sampling error, while a coefficient of variation of 50 percent means that the estimate is very imprecise

10. UIDE Chapter 25 Working with Quantitative Data Poisson Distribution –The probability of a number of events occurring in a fixed period of time if these events occur with a known average rate and independently of the time since the last event. –Can be applied to systems with a large number of possible events, each of which is rare

11. UIDE Chapter 25

12. Interpretation of User-Observation Data –Assigning Severities High – Medium – Low Exact reference Blend of factors (frequency, severity, recoverability, cost) The human factor

13. UIDE Chapter 25 Interpretation of User-Observation Data –Recommending Changes Who recommends changes – evaluators, BSA, developers, users?

14. UIDE Chapter 25 Writing the Evaluation Report –Should You Describe Your Method? For academics – yes, for business usually no –Describing Your Results Graphics, screenshots help Have data and communicate sense that you have data, but keep data in background