Homegrown Usability Testing-- Will It Provide Results? Library Tech Conference 20112 March15th, 2012 Aurora Jacobsen and Terry Glidden

Different testing models Contract Testing Local companies Costly Contract Initial Testing, SELCO Continues College student option Contract Testing No local company’s the did this kind of testing that we could easily find Costly to have an outside company do ongoing testing Contract Initial Testing Initial search did not turn up testers near Rochester Though probably not as expensive as paying for ongoing testing, but still more than our budget could not afford that either. Even discussed with an instructor at a local college having his class perform the initial test, but the problem was that our site was a little too librarian-centric with our terminology (after all they are our audience) for the students to understand.

Different testing models (continued) Guess Based on Web Clicks Multiple navigation methods on same page Click analysis is after the fact Send Someone to “Usability School” This was partially a joke, but an option. Again, the budget prohibited that Multiple navigation methods: Pages can have multiple links that lead to the same making it harder to tell which navigation item is the one being used the most. We have this issue. Click analysis is after the fact: live testing allows us to see if there is any confusion by the user before picking their link of choice

Different testing models (continued) Steve Krug Method (Rocket Surgery Made Easy) Suggested to us by instructor from above Limited resources required One morning a month or, as we decided, every two months, No special equipment that we didn’t already have in the office

Rocket Surgery Made Easy Test in action: Demo Video  Process One morning a month (every two months) Three Testers Do the same task(s) Enough to identify the big problems Recruit loosely Don’t worry about "representative users" Most usability issues are navigation, not jargon or subject issues. Representative users are more of a factor in later usability tests.

Rocket Surgery Made Easy (continued) Process (continued) List of the top 5-10 things people need to do on our site Spectator Sport Observers are the ones actually making a list of the usability problems Debrief List of problems to be fixed within the next month Focus Ruthlessly on only the most serious problems

Rocket Surgery Made Easy (continued) Other considerations: More important to do regular testing and catch the big stuff than to have a "perfect test“ Pilot test Test other similar sites to get an idea Don't wait until our page is "done" Better to catch early Can't use Search in test

Resources Needed Required Optional Mic Speakers Screen Secording Software Screen Sharing Software Video Recording Equipment Optional Snacks Incentives (gift certificates?)

Pilot Usability Test Experience Recruiting Some of our more members volunteered SELCO staff volunteered Technical Setup Skipped Video recording equipment Incentives - helping us out Have a demographic sheet Ask feelings about the site, instead of initial impressions. Recruiting SELCO staff Some of our more loyal members volunteered just out of frustration with existing site Technical Setup Skipped Video recording equipment we used Skipped video recording test subjects Go2Meeting to record screen movements We did not us incentives our subjects were happy to just help us out Have a demographic sheet for some base questions Ask people about their feelings about the site to begin with, instead of initial impressions.

Pilot Usability Test Experience (continued) Debriefing Session Original proposal recruiting random people Some difficulty with people that we know Need an effective “hall monitor” Web Site Updates Easier to make some small changes Questions about big issues we’re retesting Still doesn’t answer wording questions, just reveals the language problems Debriefing Session Original proposal is recruiting random people Some difficulty with people that we know - Because we are a membership organization, we haven't seem to experience that yet, and in fact people are eager to help. The downside is that because they are members we are more apt to judge the tester instead of the site, even though we promise not to in the introduction. Need an effective “hall monitor” - An effective hall monitor becomes more important in that situation, making sure the focus stays on the site and not tearing down the tester so that we don't take them seriously. Web Site Updates Easier to make some small changes - some small changes have been really helpful for users Questions about big issues we’re retesting Still doesn’t answer wording questions, just reveals the language problems

Pilot Usability Test Experience (continued) Make Modifications to Process Some small updates so far, but it’s been largely successful Schedule Next Testing Day