the d.r.e.a.m. device Alex Do Ken Langford Therese Peffer Colleen Whitney

Project Goals  Develop a demand response enabled appliance manager… aka the d.r.e.a.m. device!  The device will: –Keep track of electricity use and spending –Provide an electricity “price forecast” –Control heating and cooling (thermostat)

The d.r.e.a.m. device I.Heuristic Evaluation II.Current Design III.Pilot Study IV.What We Plan To Do V.What We Learned

I. Heuristic Evaluation

SylViA’s Evaluation  First interactive prototype: Java applet Java applet  Two marks of Severity 4: usability catastrophe usability catastrophe imperative to fix imperative to fix –Temperature “comfort zones” –Consistency in usage graphs  Eight marks of Severity 3: major usability problem; important to fix major usability problem; important to fix –Clutter of left-hand screen –Use of graphs in right-hand screen –Secondary bottom-tabs

Left screen  Arrows do not work, too close to right side  Comfort zone colors confusing and overwhelming

Usage Graphs  Today vs. Last 7 or Last 30 days – graph formats  Hard to read information and translate

Bottom Tabs  GUI Blooper  3 different heuristics: –H6 Recognition rather than recall; Tabs not obvious –H8 Aesthetics and minimalist design –H2 Match between system and real world; Violate metaphor of tabbed file cards

Settings  Too many radio buttons  Not strong connection between temperature and schedule  Confusing

II. Current Design

2 nd Interactive Prototype  Prototyping language changed to combination of HTML & JavaScript

Comfort Zone & Temp Control Comfort zone eliminated; triangle changes color to match current pricing Arrows moved

Graphs Grouped based on content New graph created

Tabs & Clutter Created nested tabs Date/time info to toolbar

Forecast & Pricing New pricing info

Settings - Temperature

Settings – Schedule

Not Implemented in Second Interactive Prototype  Interactive setup wizards  Help  Videos Help & Videos Wizards

III. Pilot Study

Test Subjects  Who: 3 people who were similar to our personas –Particularly interested in finding someone over 60 “Mabel” 60+, married Lives in condominium Difficulty w/small type “Alison” Mother of 2 Live in fairly large house *Original test subject from low-fidelity testing “Tim” Married w/ 3 children Some college Comfortable with electronics, timers

Main Tasks  Thermostat –Info: Find current temperature –Action: Set temperature preferences (for dynamic prices) –Action: Set schedule of temperature preferences  Dynamic Pricing –Info: Find current price –Decision: When to use appliances?  Budget and Usage –Info: How deep into spending target? –Decision: How to conserve?

Test Measures  How long it took to accomplish each task  Which pathways were taken (how much did they struggle to find what they needed?)  Which pathways were taken (how much did they struggle to find what they needed?)  Level of certainty and satisfaction with task completion (how clear was the feedback?)  Amount of prompting needed

Results: General  Overall, results per task were qualified as significant, slight, or no problem –Of 12 total tasks: Significant Problems Slight Problems “Tim”32 “Alison”05 “Mabel”14  Main problems resulted from using “Away” switch, determining current price, and setting “Away” temperature  Post-test survey results indicated that the device generally performed to the users’ satisfaction and that they were able to understand dynamic pricing by using the device

Results: Hold / Normal / Away  Switch to away mode –2/3 subjects didn’t know what they were looking for, other had already seen it in lo-fi test –Attention focused on right screen rather than left side

Results: Price Indicator  Find current price –None of subjects used price indicator on left side –All used price forecast by looking at “Cost” screen

Results: Temp Setting  Well understood: –Navigation: “Temperature” screen under “Settings” was easy to find –Setpoints: dragging triangles worked well –Profiles: Task specified setting for Away, and all identified correct profile  Problem: –Per-price behavior: All had difficulty setting the temperature specifically for the High price period (specified in task) –Could be interface problem, or could be conceptual problem

IV. What We Plan to Do

Interface Changes  Left side worked well. Minor modifications needed: –Make better visual proxy for “knob” –Better feedback for away/hold/override modes –Visual difference between sets of radio buttons

Interface Changes  Revise tabs to show highlighting more clearly  Incorporate price display into “time bar”

Interface Changes  Price forecast, budget and past usage screens very effective –Reduce visual clutter  Problems with navigation among screens –Card sorting

Interface Changes  Relationship of settings screens, affordances clear –Add wizards –Add help video  Problem with temperature settings: poor understanding of demand response? –Formal testing

Other Issues for Testing  Alternatives for displaying usage information: –pie charts –bar charts –icons  Differences in device use, display preferences by: –age –technical ability –language proficiency –education level  Interaction between radio buttons on left-hand side

V. What We Learned

Lessons Learned  Implementation: required level of fidelity –Easier to modify HTML/Javascript prototype than the initial Java version  Testing: order of questions, minor wording choices matter  Analysis: difficult to tease out problems with comprehension of new pricing model vs. UI problems

Questions?