1. U.C. Berkeley Calendar Network Usability Evaluation Nadine Fiebrich & Myra Liu IS214 May 4, 2004

2. Introduction Background Description of BCN tool http://dream.sims.berkeley.edu/EventCalendar/Documents/ThirdIn teractivePrototype/login.php Usability Methods 1. Interviews 2. Usability Tests 3. Survey

3. Method #1: Interviews Conducted in January as part of needs assessment process Goals To gather information about:  Current calendar/event management processes  Technical expertise and development capabilities  Likes and dislikes about current system/methods  Communicating events within the UCB community  Event data  Recommendations Participants  12 participants  3 technical sophistication categories (high, medium, low) Procedure  1.5 hours per interview  2 group members - one conducting interview, one taking notes

4. Method #1: Interviews Summary of results  Current process  Last minute changes  No security concerns  Private vs. public  Archived events  Push vs. pull  Sufficient data model  Personal calendars  Maintain ‘look and feel’

5. Method #2: Usability Tests Conducted several rounds of usability testing from early March to late April Goals  Ensure that participants can find and use the various functions in the interface  Determine whether the functionality provided is valuable  Determine whether the interaction flow is natural and intuitive and matches current user workflow  Determine whether the application will meet the needs of these calendar administrator Participants  3 participants - 2 calendar administrators, 1 calendar content manager  All participants are familiar with web-based applications and other calendaring tools Procedure  3 task scenarios

6. Method #2: Usability Tests Summary of results  Task Time Completion ParticipantScenario 1Scenario 2Scenario 3 13m48s4m16s6m05s 23m17s4m28s6m09s 33m55s4m45s5m14s Avg. Time/Task 3m40s4m30s5m49s

7. Method #2: Usability Tests Summary of results  Event Manager  Users had trouble finding some functions in the sub-nav  Users would like to be able to search all tabs  Users did not notice confirmation messages  Format Calendar  Need more explanatory text  Clarify terminology  Event Details  The flow from this page doesn’t make sense if the user doesn’t want to take any action  Create Event form  Need more explanatory text

8. Method #3: Survey Goals  Capture participants overall satisfaction  Gather opinions on specific areas of functionality  Gather standard set of responses that can be compared across participants Participants  3 participants - 2 calendar administrators, 1 calendar content manager Procedure  After our usability tests, we emailed our test participants with a follow-up survey in a Word document

9. Method #3: Survey Summary of results  Users found the system relatively easy to use, though more explanatory text and help is needed in many areas  Users thought the system would be valuable to their department as well as other campus departments  Users thought the functionality provided was valuable, but some functions more than others depending on their needs

10. Conclusions Specifically the usability methods have been useful for: Interviews  Used interview data to design initial workflow and supported tasks of our application  Determine how our tool can fit into the UCB community  Gathered ideas for functionality Usability Tests  Conducted usability testing for each iteration of our design  Used results to improve specific parts of the interface Survey  Get standardized user ratings of the system  Use results to prioritize development of functionality

11. Lessons Learned  Craft the language used in task scenarios for user testing carefully, as unclear language can cause unnecessary user confusion  You can iterate a lot longer than you might think and continue to improve your design  You continue to learn from usability testing even with just a few users  Interviewing users after you have a prototype can yield even more useful and specific information

12. Questions?