1. Cengage Learning Opposing Viewpoints In Context Usability Test Hao Zeng, Mikko Tuomela, Josh Keyes, Travis LaFleur SI 622 Group 3 April 14, 2011

2. Cengage Learning  Publisher of print and digital information services for academic, professional and library markets  Delivers customized learning solutions for colleges, universities, professors, students, and libraries

3. Opposing Viewpoints In Context (OVIC)  OVIC is an online resource covering today’s hottest social issues  It helps students research, analyze, and organize broad variety of data  Used for conducting research, writing assignments, debates and more

4. Test Recruiting Sent email to SI Informatics email list Gave general information about study along with how much time it would take Promised them a $25 Amazon gift card after their participation We aimed for 5-7 subjects Scheduled 7, 2 canceled so we tested 5 subjects

5. Usability Test Tasks  4 Tasks  15 minutes  Client: focus on search and multimedia content +  Try to simulate normal operations that a general user will usually do in the system

6. Usability Test Tasks 1.Find two articles about nuclear power, one in favor and one opposed. Translate the second article into French and send it to your French friend (hao.hci@gmail.com). 2.Find 2 radio broadcasts about the 2011 earthquake and tsunami in Japan, save them to your personal favorites in the system, and then start playing one of them. 3.Find a magazine article about unemployment and download the article’s APA citation. 4.Find an article related to stem cell research from a peer reviewed journal that was published in 2005.

7. Test Logging Logging sheet for each task o From 2 to 4 subtasks  Completed? Y / N o General questions  Surprised? Y / N  Stuck? Y / N  Confused? Y / N o Space for comments and notes To be filled either by pen or by computer (Word document)

8. Moderation & Debriefing Introduction script Sign consent form Pre-test questionnaire o Background, demographics, experience Begin tasks Encourage talk aloud Post-test questionnaire o Reflect on experience o Likes, dislikes, improvements Ask for questions Thank and dismiss

9. Recording

10. Location and Setup 1150B/1150C at North Quad 3-4 researchers present Audio recording by computer

11. Usability Test Findings and Recommendations

12. Usability Test Findings: #1 The “Listen” appears on top of the content of every document for people with visual impairment; But it causes significant confusion for general users.

13. Usability Test Findings: #1

15. Usability Test Recommendations: #1 Recommendation: Rename to "Read the Screen" and move to Tools section.

16. Usability Test Findings: #2 Pro and Con Indicators Are Needed In Search Results System is designed to give users arguments from "both sides" of an issue Sides of articles aren't readily identifiable in search results

17. Usability Testing Recommendation #2 Implement short summary to let user know if articles is "Pro" or "Con" for the subject that they are searching about "Pro" or "Con" summary could be anywhere from one sentence up to a short paragraph. Summary text could be placed to left of search results, just make column with results in it a bit skinnier (which would be ok)

18. Usability Testing Recommendation #2

19. Usability Test Findings: #3 Basic Search is used often, but is not fully satisfactory.

20. Usability Test Findings: #3 Does not handle Google-style queries "stem cell research 2005" "Japan tsunami audio" Recommendation: improve search algorithm to accept more parameters (not just keyword)

21. Usability Test Findings: #3 Recommendation: add filtering options from advanced search into results page for basic search

22. Usability Test Findings: #4 Finding 4: Options in Advanced Search Are Ambiguous Users were confused about what "Content Level" means Because they don't know what content level means, they can't make sense of the options that they can select It frustrated users when they weren't able to tailor the advanced search options clearly

23. Usability Test Recommendations: #4 Cengage should name "Content Level" something that is more descriptive of what the feature actually filters Cengage could add a help section that describes all of the features that are offered in the Advanced Search

24. Thank you! Questions?