EasyChair Project Reviewer sign up and bidding Art Hsieh Jean Huang Norik Davtian Ryan Nissenbaum

Objective Conduct user needs studies and redesign the sign up and paper bidding pages/process for EasyChair website.

Usability Problems General Problems o Is not visually appearance o Instructions are either too wordily or unclear o Error messages are unclear and difficult to read o Terms/words are inconsistent throughout the website

Usability Problems Con't Sign Up Process o Weird sign up steps arrangement o Fields are not well organized o Requires too many steps for simply creating a new user account

Usability Problems Con't Paper Bidding Process o Bidding selection is not clearly marked o Paper bidding list is difficult to read o Current design doesn't put huge paper or user list into consideration

Plan Free-form Interview with Prof. Kobsa Team Evaluation Prototype [alpha] Usability test (Phase I) o Group A Prototype [beta] Interim Presentation Usability test (Phase II) o Group A o Group B Prototype [release candidate] Final Presentation Final Document Done In-Progress Week 5-6 Week 6-7 Week 7 Week 7-8 Week 9-10 Week 10 Week 9-10 T N A,J,R N T A,J,R N T

Participants Professors, graduate students in scientific fields No subgroups Age: in each phase Recruitment: , ask in person.

Team Evaluation Having each individual evaluator inspect the interface alone Evaluator: 4 team members Content: o Sign up process: Form-fill o Paper bidding: Data display Method: each evaluator o Walkthrough both process as a real user o mark down problems encountered through the process o Check HCI design guidelines for form-fill and data display o Meet with other evaluators and make the final decision

Usability Test Having users work with an interactive prototype Two parts: sign up/paper bidding 5-8 participants in total Subgroups: scientific / non-scientific people Screen and voice recording Location: UCI campus, e.g. offices, meeting rooms Duration: 30 minutes

Sign up Scenario: You are interested in the topic of “University in Distress” and want to attend the conference. You get access to EasyChair and know you have to sign up for further information. Method: Think aloud o Walkthrough the sign up process to set up an account, and log in the system. Open-ended questions: o Where an error occurs, do you know what makes it happen and how to fix it? o If the account didn’t set up successfully, do you know how to fix it?

Paper Bidding Scenario: Users are reviewers who want to perform paper bidding process. Sample Tasks and questions: o Enter your reviewing preferences  get the information for papers, such as keywords, author affiliations  read the abstract of certain paper  make your decision among yes/no/maybe/conflict o Change preferences o Read statistics of paper bidding  How many people are willing to review certain paper?  How many papers in total do they want to review?  If you have to assign 5 papers to 10 reviews, what is your decision? Open-ended questions o How do you feel about the instructions and layout?

Demo

Thank you! Questions and comments?