1. September 29, 2011 Case Study: Usability Testing for IT Training Website Redesign Project Jae Kim, Jenny Hertel, Greg Hanek, Tom Mason, Chris Payne

2. usable@indiana.edu The User eXperience Group (UXG) assists IT teams improve the user experience of their products and services through a variety of user-centered design and evaluation methods, including: Expert Reviews Information Architecture Reviews Design Critiques Prototypes and Mockups Focus Groups User Testing

3. WHAT UXG Consultants observe people using a system as they attempt to complete a set of representative tasks WHY 1.Through user testing, we identify problems and issues; we then provide a report outlining suggested improvements and changes. 2.User testing is the best way to determine how well people interact with your system. It can provide valuable insights into the problems that real users might have, and offers the greatest potential for improving the usability of your site.

4. Instructor-led workshops Nearly 1,000 per year 10,000+ participants per year 95% Very Good/Good rating

5. Self-paced training downloadable workshop materials downloadable workshop materials

6. Why Usability Testing? Usability testing is essential to ensuring good customer experience. IT Training first tested its site in 2001 when introducing its first dynamic web site. The tests we’ll talk about today were conducted in 2009-2011. Goals were to make it easier for users to find training and to make the site more engaging.

7. Results

8. What Was Working TASK: Find a specific workshop OBSERVATION 1: Specific workshops were easy to find using the search on the IT Training website OBSERVATION 2: Specific workshops were easy to find using Google search Image source: prlog.orgprlog.org Search:

9. What Was Working Registration: TASK: Register for a workshop OBSERVATION: 90% of testers and ~80% of all ‘real’ users were able to register for a class in 2009 2009 2011 EASYHARDFAIL

10. What Was Working User Profile TASK 1: Update your profile TASK 2: Find a list of previously attended workshops OBSERVATION: Both tasks were easily completed. ‘09 ’11 ‘09 ‘11 EASYHARDFAIL

11. User Testing Video Clips 1b.Confusing Navigation - Lack of clear path to main content 1a. Confusing Navigation - Unintuitive modality-centered navigation 2. Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

12. VIDEO CLIP

13. What was NOT working? TASK: Find a specific workshop OBSERVATION: We expected users to use the online listing of workshops to complete this task, but many instead downloaded the PDF catalog 1a. Confusing Navigation – Lack of clear path to main content: 2009 2011 EASYHARDFAIL

14. User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

15. VIDEO CLIP

16. What was NOT working? TASK: Find ALL self-paced training options OBSERVATION: Users either couldn’t find any options, or they found only one and assumed there were no more 1b. Confusing Navigation – Unintuitive modality-centered navigation: 2009 2011 EASYHARDFAIL

17. Old IT Training Home Page

18. Old IT Training Site – Navigation Expanded

19. Old IT Training Site – Instructor Led by Topic

20. Old IT Training Site – Instructor Led by Topic – Excel

21. Old IT Training Site – Tutorials Page

22. User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

23. VIDEO CLIP

24. What was NOT working? TASK: Find the cost of a workshop OBSERVATION: o Users expected the cost to be near the workshop description o Link for the pricing opened the FAQ page, not the specific article 2. Workshop Pricing: 2009 2011 EASYHARDFAIL

25. User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

26. VIDEO CLIP

27. What was NOT working? TASK: Register for a workshop OBSERVATION: The fix for the navigation problem found in the 2009 test resulted in a screen layout change that called for better indication that an item was added to the cart Image source: modmyi.commodmyi.com 3. The Shopping Cart:

28. User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications – Certification comparison table

29. VIDEO CLIP

30. TASK: Find all certification and certificate options. OBSERVATION: o Users couldn’t find all accreditation options, “settling” on one or two o Were not confident if what they found satisfied the task What was NOT working? 4a. Certificates and Certifications – Unclear certification options 2009 2011 EASYHARDFAIL

31. User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications – Unclear certification options 4b.Certificates and Certifications - Certification comparison table

32. VIDEO CLIP

33. TASK: Find all certification and certificate options. OBSERVATION: Comparison table was useful, but was difficult to find What was NOT working? 4b. Certificates and Certifications - Certification comparison table 2009 2011 EASYHARDFAIL

34. Task Success Rates ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 EASYHARDFAIL

35. Next Steps Get user profile information from central systems. Users indicated they didn’t know why they need to have an IT Training profile Will align with IU systems and user expectations Addressed in near term through FAQ

36. Make it easier to find old versions of materials. Current site tells users to contact us. Need to make self-service. Next Steps

37. Events calendar Other new features to come

38. More filtering Other new features to come

39. Deep linking into 3 rd party elearning where possible. Other new features to come

40. We will use usability testing to make sure we build these features the right way! Where do we go from here?

41. Usability testing is a very valuable tool It helps us see what is going on with our changing audience, instructional delivery and business model It has already helped us greatly improve our customer’s experience and we know it will continue to do so We believe in it and we teach it! Conclusions

42. QUESTIONS? User eXperience Group usable@indiana.edu IT Training ittraining@indiana.edu