SIMians Course Comment Forum (CoCoFo) Linda Duffy Jean-Anne Fitzpatrick Sonia Klemperer-Johnson James Reffell

CoCoFo Overview  Web site for SIMS students to:  Get descriptive information about courses offered at SIMS (and other classes at Berkeley of interest to SIMS students)  Find out about the opinions of other students who've taken a course in the past  Express their opinions about courses they've taken

Paper prototype

First interactive prototype

 Implementation: HTML and Perl/CGI scripts  Large number of static pages  Did not actually save information entered through the "add comment" function

Heuristic Evaluation  Largest issues:  rating scale  forced preview  Many smaller errors (such as clarity of text labels)

Second interactive prototype

 Implementation: Cold Fusion  Small number of dynamic pages  Implemented "add comment" function using Access database  Changes based on heuristic evaluation

Rating scale n Old:

Rating scale n New:

Preview n Old:

Preview n Old:

Preview n New:

Examples of other (visible) changes in second prototype… n Display login status

Examples of other (visible) changes in second prototype… n 'Return to top' links

Examples of other (visible) changes in second prototype… n Prevent invalid submissions

CoCoFo DeMo  I will demo the tasks we assigned to the participants in our informal usability test:  Find the course reviews for IS250.  If the tester browsed to find IS250 as part of the first task, we asked the tester to search for a class on user interface design (or vice versa).  Post a comment to IS206.

Results - positive n Testers liked overall design n Testers liked idea of site n Most testers understood rating system

Results - Negative n Inexact searches got no results n Login button misplaced n Preview didn’t highlight ratings n Course categories less than helpful n Unclear about who has access to site n Text box didn’t wrap

Results - Additions n More ratings n Edit/delete comments n “Rate the rater”

Results - Questions n Anonymity – keep identities public n Rating system – numeric is fine, more description n Threading – negative to lukewarm response

Results - Observations n Preference of browsing over searching n Preference of verb-noun over noun-verb method of adding comment n No search modification n Use of both “preview” and “submit” from “add a comment” page

Formal Experiment - Hypotheses n Presentation of rating scale: combination of both graphical and numeric scales will be understood more quickly and accurately than a strictly graphical scale or a strictly numeric scale. n Span of rating scale: A scale of 1 to 5 will come closer to capturing the user’s “true” rating than either a scale of 1 to 3 or a scale of 1 to 10.

Formal Experiment – Independent Variables n Factor 1: Presentation of the scale 1.Strictly numeric scale 2.Strictly graphical scale 3.Combination of numeric and graphical scale n Factor 2: Span of the scale 1.Scale of 1 to 3 2.Scale of 1 to 5 3.Scale of 1 to 10

Formal Experiment – Response Variables n Amount of time it takes for the user to assign ratings for course difficulty and instructor n Whether the user chooses not to rate course difficulty and/or instructor n Whether user asks for clarification of rating system n User’s satisfaction with the scale n User’s opinion regarding whether ratings assigned accurately reflected user’s opinions

Next Iteration: Plans n Add workload rating n Clarify difficulty rating scale n Add display option to comments n Highlight noun-verb comment option n Make active button focus behavior consistent n Get text box to wrap

Next Iteration: Hopes n Edit/delete functionality for old comments n Implement registration n Improve search n Improve categories n Add multi-year distinctions n Complete course listings n Graphical course rating displays n Rate the rater

Summary n Insightful commentary n Witty banter n Sweeping generalization n Misleading cliché n Feel-good rhetorical flourish n Snappy ending

Further Study n Questionnaire results: –Animated dog preference varies widely –Some prefer cats (!) –Some have multiple-dog preference n Clearly, more research required