Usability Evaluation The Supreme Court of Texas Sheng-Cheng Huang and Anuj A. Nanavati School of Information University of Texas at Austin November 22, 2004

Overview ► Introduction ► Potential users and information need ► Test objectives and environment ► Test participants ► Test procedure ► Data analysis ► Recommendations  Problem areas  Additional functionalities

The Supreme Court of Texas

Introduction ► Texas Judiciary online  Supreme court of Texas  Court of criminal appeals  Court of appeals  Trial courts ► Use of the Web site  Case related information – access + search + monitor ► Briefs, opinions, orders, deadlines ► Party, attorney, firm, trial courts, judge  Procedural and administrative court rules  Contact information ► Attorney, party, judge

Potential user groups and Information need ► User groups  Attorney ► Trial ► Appellate  Journalist ► Regular monitor ► Occasional monitor ► Watch dogs  Law scholar ► Law student ► Law professor ► Court advisory committee member  Novice user ► Different Information need  Search targets  Known information  Constraints

Test objectives and Environment ► Objectives  Identify distinction between different information needs  Obtain understanding of information categorization and organization  Determine effective information delivery  Identify most frequently used features  Identify overall satisfaction level ► Environment  User’s workplace  Different computing environment ► 4 users – high configuration laptop computer ► 2 users – personal computing environment  Recording software ► Morae

Test participants ► Two user groups ► Total: 6  3 attorneys  3 journalists ► Use of key information  Attorneys ► Case actions ► Orders/opinions ► Rules and procedures  Journalist ► Case briefs ► Orders/opinions ► Individual info ► Diverse age group: ► All experienced and frequent computer/Web users

Test procedure ► Consent form ► Pre-test questionnaire ► 5 tasks: access + search + monitor 1. Search general case information by case number/keywords 2. Search opinion or orders by case number/keywords 3. Search for Texas rules of appellate procedure 4. Track general case development, specific actions or events 5. Open task chosen by participants ► Post-test questionnaire ► Interview – user feedback

Data Analysis - 1 ► Evaluation Measures  Quantitative ► Demographic and computer usage data  Qualitative ► Subjective responses ► Subjective ratings ► Task analysis by user group  Task 1

Data Analysis - 2  Tasks 2 and 4  Tasks 3 and 5

Data Analysis - 3 ► Other issues ► HTML Source Assessment

Recommendations ► Four major problem area  Menu design  Information description  Information organization  Information needs ► Two levels of priority ► Additional functionalities ►

Problem Area: Menu Design ► Level: 1  Appropriate and familiar labels ► Front Desk  FAQ  No sub menus for single entity  Main menu entry for important info ► Contact information, FAQ ► Level: 2  Proper highlighting for current mouse focus  Don’t hide the remaining menu  Menu categorization ► Rules

Problem Area: Information Description ► Level: 1  Current and previous versions or rules  General case info upfront ► Party, trial court  Contact information for inquiries ► Party, attorney, firm ► Level: 2  Rule numbers with current versions  Clear labeling for date field in search forms  Multiple input format for COA number

Problem Area: Information Organization ► Level: 1  Links to opinions & briefs with case info  The folder icon ► Level: 2  Categorized labeling on the home page ► Latest news/releases, weekly orders, events

Problem Area: Information Needs ► Level: 1  Party name in opinion search results  Time deadlines for cases ► Level: 2  Statistical information for a court term ► Case trialed ► Opinions released ► Cases per judge

Additional Functionalities ► Case info in the case mail feature ► Case search by  Law firm  Attorney  Party  Pending cases ► Opinion search by  Keywords  judge ► Search interface  Rules  E briefs

Questions or Comments? Thank you !