September 29, 2011 Case Study: Usability Testing for IT Training Website Redesign Project Jae Kim, Jenny Hertel, Greg Hanek, Tom Mason, Chris Payne

The User eXperience Group (UXG) assists IT teams improve the user experience of their products and services through a variety of user-centered design and evaluation methods, including: Expert Reviews Information Architecture Reviews Design Critiques Prototypes and Mockups Focus Groups User Testing

WHAT UXG Consultants observe people using a system as they attempt to complete a set of representative tasks WHY 1.Through user testing, we identify problems and issues; we then provide a report outlining suggested improvements and changes. 2.User testing is the best way to determine how well people interact with your system. It can provide valuable insights into the problems that real users might have, and offers the greatest potential for improving the usability of your site.

Instructor-led workshops Nearly 1,000 per year 10,000+ participants per year 95% Very Good/Good rating

Self-paced training downloadable workshop materials downloadable workshop materials

Why Usability Testing? Usability testing is essential to ensuring good customer experience. IT Training first tested its site in 2001 when introducing its first dynamic web site. The tests we’ll talk about today were conducted in Goals were to make it easier for users to find training and to make the site more engaging.

Results

What Was Working TASK: Find a specific workshop OBSERVATION 1: Specific workshops were easy to find using the search on the IT Training website OBSERVATION 2: Specific workshops were easy to find using Google search Image source: prlog.orgprlog.org Search:

What Was Working Registration: TASK: Register for a workshop OBSERVATION: 90% of testers and ~80% of all ‘real’ users were able to register for a class in EASYHARDFAIL

What Was Working User Profile TASK 1: Update your profile TASK 2: Find a list of previously attended workshops OBSERVATION: Both tasks were easily completed. ‘09 ’11 ‘09 ‘11 EASYHARDFAIL

User Testing Video Clips 1b.Confusing Navigation - Lack of clear path to main content 1a. Confusing Navigation - Unintuitive modality-centered navigation 2. Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

VIDEO CLIP

What was NOT working? TASK: Find a specific workshop OBSERVATION: We expected users to use the online listing of workshops to complete this task, but many instead downloaded the PDF catalog 1a. Confusing Navigation – Lack of clear path to main content: EASYHARDFAIL

User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

VIDEO CLIP

What was NOT working? TASK: Find ALL self-paced training options OBSERVATION: Users either couldn’t find any options, or they found only one and assumed there were no more 1b. Confusing Navigation – Unintuitive modality-centered navigation: EASYHARDFAIL

Old IT Training Home Page

Old IT Training Site – Navigation Expanded

Old IT Training Site – Instructor Led by Topic

Old IT Training Site – Instructor Led by Topic – Excel

Old IT Training Site – Tutorials Page

User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

VIDEO CLIP

What was NOT working? TASK: Find the cost of a workshop OBSERVATION: o Users expected the cost to be near the workshop description o Link for the pricing opened the FAQ page, not the specific article 2. Workshop Pricing: EASYHARDFAIL

User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications - Certification comparison table

VIDEO CLIP

What was NOT working? TASK: Register for a workshop OBSERVATION: The fix for the navigation problem found in the 2009 test resulted in a screen layout change that called for better indication that an item was added to the cart Image source: modmyi.commodmyi.com 3. The Shopping Cart:

User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications - Unclear certification options 4b.Certificates and Certifications – Certification comparison table

VIDEO CLIP

TASK: Find all certification and certificate options. OBSERVATION: o Users couldn’t find all accreditation options, “settling” on one or two o Were not confident if what they found satisfied the task What was NOT working? 4a. Certificates and Certifications – Unclear certification options EASYHARDFAIL

User Testing Video Clips 1a.Confusing Navigation - Lack of clear path to main content 1b.Confusing Navigation - Unintuitive modality-centered navigation 2.Workshop Pricing 3.The Shopping Cart 4a.Certificates and Certifications – Unclear certification options 4b.Certificates and Certifications - Certification comparison table

VIDEO CLIP

TASK: Find all certification and certificate options. OBSERVATION: Comparison table was useful, but was difficult to find What was NOT working? 4b. Certificates and Certifications - Certification comparison table EASYHARDFAIL

Task Success Rates ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 ’09 ’11 EASYHARDFAIL

Next Steps Get user profile information from central systems. Users indicated they didn’t know why they need to have an IT Training profile Will align with IU systems and user expectations Addressed in near term through FAQ

Make it easier to find old versions of materials. Current site tells users to contact us. Need to make self-service. Next Steps

Events calendar Other new features to come

More filtering Other new features to come

Deep linking into 3 rd party elearning where possible. Other new features to come

We will use usability testing to make sure we build these features the right way! Where do we go from here?

Usability testing is a very valuable tool It helps us see what is going on with our changing audience, instructional delivery and business model It has already helped us greatly improve our customer’s experience and we know it will continue to do so We believe in it and we teach it! Conclusions

QUESTIONS? User eXperience Group IT Training