Web E’s goal is for you to understand how to create an initial interaction design and how to evaluate that design by studying a sample. Web F’s goal is for you to create screens for three specific tasks your user will do from your GUI Project – then start to prepare the material for a usability evaluation of your initial design.

 You are also working in Web F towards creating a usability evaluation for the your paper prototyped designed screens.  In Web E you saw a usability report sample. It had many of the items you need to prepare now.

 Evaluate a prototype of an early design (the system is at the beginning of the design = conceptual).  The scenarios for the tasks are not fully developer nor fully functional – so this is a challenge for the user who wants to “play” with every thing on the screen.  Get the user’s cooperation

Part X =the plan and preparation materials for an evaluation  Set an usability objective and goals.  Deciding on the user tasks.  Create the Pre-Questionnaire 1.Assurance that you have representative users. 2.Obtaining the specific User Profile = characteristics.  Decide on recorded Events & Baseline Criteria 1.What events will you log from the users experience. 2.What formula (baseline) will decide that you redo the interface.  Create the Post-Questionnaire - Subjective Data - Perceptions Part Y= Evaluation with Users - observation  Preparing the Ulab & performing the evaluations Part Z =Evaluating the results of the orientation  Use the results and give recommendations

 You will learn and understand all parts of X, Y, and Z.  But you will only prepare your usability evaluation – you will not conduct your evaluation till next week….   so you will do only part X to hand in.

Before you do any testing, you should take time to figure out what you're testing and what you're not. In other words, determine an objective for your test that focuses on a specific aspect of the product. By limiting the scope of the test, you're more likely to get information that helps you solve a specific problem. Give the target user group and the usability objective for this evaluation and the goal for the prototype. Remember slide 4 of the sample.

 Your test participant will work through specific tasks. These tasks should be real tasks that you expect most users will do when they use your product. The entire user observation should not run over 20 minutes, so you should design tasks that focus on the part of the product you're studying. For example, if you want to know whether your menus are useful, you could design a task that requires the participant to access the menus frequently. After you determine which tasks to use, write them out as short, simple instructions.  The tasks the user will attempt to do need to be printed on separate pages.  Important: Your instructions must be clear and complete, but they should not explain how to do things you're trying to test. For example, if you want to find out whether users can navigate through your program easily, don't give them instructions for navigation. Or, if you want to know whether your interface is self-explanatory, don't describe how it works. This concept is extremely important to remember. If you teach your participants about something you're trying to test, your data will not be useful.

Create a questionnaire that will verify that you have a true user. Also include small profile questions of your user. The Briefer/Debriefer will do this at the start of the evaluation next week.

a. Decide on the events (metrics) that you will log. List the events to be recorded during the evaluation. Give which are quantified and which are qualitative. b. Decide on the Baseline criteria for the evaluation. – Decide on a formula that contains the events (metrics). Give this formula on the Report for Web F. A sample of this is in this presentation.

Prepare the computer to the “Opening scene” for each user by having the same opening scene Test the recording video and audio to insure you will have hard data that you can review later. Practice your conversation with the user to obtain their initial attitude and their pre-questionnaire data. Explain and let user rehearse the “thinking out loud”. Give the task handouts to the user and let them do the tasks alone.

 List the data logged and compile it into lists of pros & cons.  State whether the Baseline criteria was met.  Give recommendations- which are the areas of concerns – not the solutions!.

 I’ve chosen the Barnes and Noble example for my sample for you. 

Scenario 1 – Evaluate the user’s perception of the home page. Ask the user:  About their first impressions of the site  Which link they would use to find specific information  What kind of information they would be able to find by clicking the links Scenario 2 – Want to see the consistency of the site by the screens visited when the user searches for a cookbook. Ask the user:  To search for a cookbook  To determine whether the cookbook meets your requirements  Are they having difficulty determining which cookbooks meet their requirements Scenario 3- Want to evaluate the affordance of the site with the purchasing metaphors. Task: Purchase the cookbook. Ask the user: To add the cookbook that they have chosen to their cart Go through the purchase process Is there anything they dislike about the purchasing process

◦ Do you use the web?  If yes, how often?  If no, thanks so much for coming….. ◦ Do you shop for records, books, gifts, etc. on the web?  If yes, what items have you purchased?  If no, would you consider ever doing it in the future? ◦ Have you ever purchased a book from a website?  If yes, which website?  If no, would you consider ever doing it in the future? ◦ Have you ever been to the Barnes and Noble website?

Your mother’s birthday is coming soon and she has started to be very interested in food and the nutritional information about the meals she cooks. You have decided to look at different cookbooks to see if they have that kind of information and then buy one for your mother. You have decided that the Barnes and Noble website would be able to help you in finding a cookbook for your mother that has nutritional information including calorie and protein counts. Speak out loud and describe all that you see and do. Task 1 – Go to the Barnes and Noble website at Without doing any mouse travel or mouse clicks:  What are your first impressions about the site? Answer out loud.  What links are available? Answer out loud.  What information would you find in those links? Answer out loud. Speak out loud and describe all that you see and do. When you are finish with Task 1, please call the help desk.

Speak out loud and describe all that you see and do.  Search for the cookbook. When you see a possible cookbook - explain if the cookbook meets your criteria. How do you know what cookbooks meet you criteria?  Are you having trouble? Explain  Are you successful? Explain Speak out loud and describe all that you see and do. When you are found the cookbook for your mother, please call the help desk.

Speak out loud and describe all that you see and do.  Go though the purchase process until it asked you to provide a credit card number.  Was it difficult to purchase the book after you found it?  What items were easy in this task? Speak out loud and describe all that you see and do. When you are at the credit card screen, please call the help desk.

 All the team’s usual metrics Smiles, Frowns, etc.  How long it takes the user to figure out how to search for a book.  How long it takes the user to find a book that meets the requirements.  How many different books the user looks at.  How long does it take to add the chosen book to the shopping cart.  How long does it take to go through the checkout process.

 Task 1 Impressions about the site ◦ 4 out of the 5 users Look interested; ◦ All < 3 frowns; ◦ All > 1 smile; ◦ All read screens correctly; ◦ 4 out of the 5 users state that they see something that would take them where they need to go  Task 2 Looking for the book ◦ 4 out of the 5 user’s total Time <3 minutes; ◦ all < 5 frowns; ◦ All < 15 clicks; ◦ 4 out of the 5 book look at > 3 books ◦ 3 out of the 5 user’s finds a book according to their criteria  Task 3 Purchase of book ◦ All user’s total Time<4 minutes; ◦ 4 out of the 5 users < 8 frowns; ◦ 3 out of the 5 users state they understand all of the screens; ◦ 4 out of the 5 users fill out screen successfully; ◦ All users state identifiers to >7 screen items associated to purchasing.

 Identify the intended user GROUP for your evaluation.  Identify the User GOALs for the three specific task of your GUI project.  Give the Hierarchical Task Diagram with your three User Tasks. Make sure these tasks are prototyped completely in your paper system, i.e. don’t ask them to register for class if you don’t have the screen that would do that!  Performing a User Analysis for your three specific tasks - Give the storyboard of interaction  Give usability goals and concerns for your evaluation  Developing two screening questionnaire for the user=PRE ? and POST ? (make sure the Post obtains subjective data from the user).  Creating the 3 task scenarios (make sure you have a setting to your tasks) that you will hand out to your user.  Give all the data to be collected. Determine the quantitative and qualitative measures.  Develop Usability Baseline Criteria – you will have two users for your evalaution.