1. Chapter 4 Expert Reviews, Usability, Testing, Surveys, and Continuing Assessments Saba Alavi,Jacob Hicks,Victor Chen

2. Introduction Feedback is the breakfast of champions Testing is the dinner of the gods

3. Expert Reviews Heuristic evaluation Guidelines review Consistency inspection Cognition review Formal usability inspection Good thing but not perfect –“For every PhD, there is an equal and opposite PhD.”

4. Usability Studies Sped up projects and reduced cost Plans are made in advance Must obtain informed consent Thinking aloud Videotaping Field test Can-You-Break-This Competitive Usability testing

5. Surveys Large and statistically significant Decide on analysis before distributing Should be specific as possible

6. Acceptance Tests For large implementation projects, the customer or manager usually sets objective and measurable goals for hardware and software performance. Typically, a set of test cases is specified for the software, with possible response-time requirements for the hardware-software combination- if completed product fails to meet these acceptance criteria, the system must be revoked until success is demonstrated. Acceptance test should be established when the requirements document is written or when a contract is offered.

7. Where we can use them? Time for users to learn specific functions Speed of task performance Rate pf errors by users User retention of commands over time Subjective user satisfaction

8. Couple things…… If they establish precise acceptance criteria, both the customer and the interface developer can benefit. Different from usability tests The central goal of acceptance testing is not to detect flaws, but rather to verify adherence to requirements.

9. Evaluation During Active Use Careful design/thorough testing are key assets to a good system, but earnest effort to satisfy the needs of active users can be rewarded by a grateful community. Nothing is perfect but you can always work towards perfection [Good customer service is always rewarding]. Gradual dissemination is desirable so that problems can be fixed with minimal disruption [i.e. Microsoft ‘packages/service packs?]

10. Evaluation During Active Use Major changes should be limited to regularly released revisions on an annual or semi-annual basis. The more those changes can be predicted, the less likely there is to be resistance to those changes. A balance between stable access to key resources and offering new services is the way to go...

11. Methods of evaluation during active use Interviews and focus-group discussions Continuous user-performance data logging Trade-offs regarding user privacy vs. information logged Online/telephone consultants “tech support” – oxymoron these days? Online suggestion box/trouble reporting Online bulletin board/newsgroups User newsletters & conferences

12. Controlled Psychologically Oriented Experiments The power of traditional scientific method (SM) can be employed to the study of interfaces, especially classical experimental methods of psychology. Reductionist approach yields narrow, but reliable results. Overview of SM applied to HCI: Deal with practical problem, consider theoretical framework Develop hypothesis Identify independent variables to change Assign dependent variables to measure Select test subjects Control biasing factors Statistically analyize data Resolve problem, refine theory, affirm/negate hypothesis Controlled experiments powerful tool in terms of fine tuning HCIs

13. Practioner’s Summary HCI developers evaluate designs by conducting expert reviews, usability tests, surveys, and rigorous acceptance tests Post system release warrants continuous performance evaluations. In addition to providing a functioning system that meets the requirements, social feedback mechanisms should be implemented (i.e. online surveys, interviews, discussions, conferences, suggestion boxes, etc.)

14. Researcher’s Agenda Various research specialists can provide their insight to construct more effective system evaluation methods. Subjective evaluation of HCI can be augmented with the aid of experts in psychological experimentation. Input from experimental, cognitive, psychologists can highlight importances of various human aspects of conputer use: Reducing novice user anxiety Thought process of average user to design a more cohesive interface?