Usability Testing T/TAC Online Project Design Team Tsehaye Habteselasie Jennifer Okezie Meredith Riddile Aanal Sheth

Usability in Practice Incorporates direct user feedback throughout the development cycle to reduce costs and create products that meet user needs (Usability Professional Association) Learnability Efficiency Five usability attributes are factors that affect the user’s experience with a product or system. (Nielson) Learnability- easy to learn Efficiency- high level of productivity possible Memorability- easy to remember without having the learn everything all over again Errors- low error rate and easy to recover from errors Satisfaction- pleasant to use Errors Satisfaction Memorability

confUsed fruStrated uncertAin Reality of Usability confUsed fruStrated uncertAin emBarrassed anxIous helpLess dissatIsfied disTressed mYstified

T/TAC Online Project- Phase 9 Online performance support tool that allows users to input, view and search Local Improvement Plans Government-funded grants

T/TAC Online

T/TAC Online- School Improvement

Local Improvement Plan Input System Talk about project goals

Level 1 Evaluation Collecting reactions and gauging perceptions Predicting level of user acceptance

Usability Testing Cycle Expert Reviews Redesign Usability Testing Survey Redesign Launch analysis analysis Discuss the importance of expert reviews for our project; justify why expert reviews are part of our usability testing cycle

Usability Goals Usable errors help requests instances of confusion and/or frustration Efficient Likert scale Useful Acceptable expression of satisfaction Reason for goals (align with level 1 evaluation)

Planning Usability Testing Solicited participation from target audience Developed protocols Developed task scenarios based on usability goals Level 1 desired results Reason for goals (align with level 1 evaluation)

Schedule of Activities Evaluation Activity Duration Expert Reviews 2 weeks Analyze & Redesign I 1 week Usability Testing Survey Analyze & Redesign II

Expert Reviews Allowed experts to test system Asked experts a set of predetermined questions Design Experts Content Experts Technical Experts congruency consistency functionality completeness accuracy relevance to job performance potential technical problems

Expert Review Findings Organized comments into categories Calculated frequency of comments Visual Design Interaction Design Navigation Design Information Architecture Directions scrolling font sizes images to be placed throughout process provide feedback after user submits important data reduce number of buttons on pages make buttons more consistent provide more embedded content support reduce length of directions some process directions need to be more explicit

Redesign Redesign based on expert review comments Before After

Redesign Before After

Usability Testing Participants completed task scenarios Used an observation log Measured observations Used Likert scale to measure user perceptions Ease of use Efficiency Usefulness Acceptable

Task Scenarios One example: You began your Local Improvement Plan (LIP) for the 2004-2005 school year earlier and now you have collected more information that you would like to add to this LIP. Add the following information to Section 2 of this LIP: Add the table that corresponds with the description of Goal 3. This table is titled “Practice_Table.doc” and is located on your desktop. Add the following information to Section 3 of this LIP: “There will be a decrease by 10% in the special education teacher turnover rate during the 2003-2004 school year.” Save your work and select continue.

Observation Log H= help request S= satisfaction E= error Usability Test: T/TAC Online LIP Input System Date: Recorder: H= help request S= satisfaction E= error C/F= confusion and/or frustration Task Time H S E C/F Task #1: New LIP Start: Stop: Participant’s Comments Notes Task #1

Debriefing questionnaire Used Likert scale 1 – 6 1. How do you rate the overall ease of use or difficulty of this system? 2. How would you compare using the online system with creating LIPs the way you currently do? 3. How useful is this system? 4. Would you use this system if it was available?

Usability Testing Findings Calculated observations and Likert scale responses ***ADD EXCEL BAR GRAPH***

Likert Scale Results

Likert Scale Results

Usability Reactions Cutting and Much easier than pasting is easy! Much easier than current process! You can glean ideas from other LEAs! Standardized format is great!

Surveys Used in order to reach out to users throughout the state E-mailed one task scenario and questionnaire to 8 participants

Redesign Plans 1. Create explicit directions to explain location of login button 2. Make taskbar active 3. Add spell check feature *** ADD CLIPART***

Redesign Plans 4. Clarify two button features 5. Confirm that Edit page will allow all users to see “edit” button located to the right of each section