The Value of USAP in Software Architecture Design Presentation by: David Grizzanti

Introduction Usability is an important quality attribute of interactive systems. Since the 1980’s, usability has been treated as a subset of user interface functionality. Since usability is not considered early in the design process, problems found in user testing are very costly.

Introduction The authors research between usability and software architecture has lead to the development of Usability-Supporting Architectural Patterns (USAPs). Each of these address a usability concern not addressed by separation alone.

Introduction Each USAP consists of: An architecturally sensitive usability scenario. A list of general responsibilities derived from forces inherent in the task and environment, human capabilities and desires, and software state. A sample solution implemented in a larger separation-based design pattern.

Introduction In their paper, the authors described a controlled experiment design to assess the value of using USAP components in modifying a software architecture design. Participants were asked to redesign an existing architecture to allow for users to cancel a long- running command. This experiment measured whether the architectural solutions produced using all the USAP components was more beneficial than using only certain subsets.

Experiment - Participants 18 computer science graduate students at Carnegie Mellon participated in the experiment. The 15 male and 3 female participants ranged in age from 23 to 30, and all had completed work for a master’s degree. Participants also reported spending an average of 22.9 hours per work programming.

Experiment Design & Materials Participants were randomly assigned to one of 3 teams. Participants in each group received a different version of a “Training Document.” All received the same architecture redesign task.

Groups Group 1 received: A usability scenario describing user circumstances. Group 2 received: The usability scenario A list of responsibilities to consider with a cancel command. Group 3 received: The usability scenario List of responsibilities Sample solution for adding cancellation function.

Experiment The task given to participants was to redesign an existing architecture that did not support cancellation, to support cancellation. Chose the Plug-in Architecture for Mobile Devices (PAMD), developed by students at Carnegie Mellon. Developed for the Palm OS4.

Experiment Task Instructions included 7 elements: General description of PAMD architecture Example PAMD scenario List of responsibilities for PAMD operations List of Component Interaction Steps detailing PAMD run- time operation A Component Interaction Diagram (Fig. 1) A Sequence Diagram of PAMD of run-time component interaction (Fig. 2) Final page instructed participant to add the ability to cancel a plug-in to the PAMD architecture design.

Fig. 1

Fig. 2

Experiment Answer Paper was designed so participants could easily and efficiently record the information relevant to their design. Component diagrams were identical to those provided to the Task Instructions, except that run-time steps and assignments were removed. Participants were instructed to use the diagrams as a bases for their designs.

Procedure Participants were randomly assigned to 3 groups and given unlimited time to complete their task. They were told that they were participating in a study about fixing one kind of usability problem in a specific software architecture design. Participants were then given the Training Document and after reading it, received the task instructions.

Results The time to complete the redesign task ranged from 39 to 138 minutes. Participants given only the USAP scenario considered 2-4 cancellation responsibilities. Those who received both the USAP and list of 19 responsibilities considered Participants who were given all 3, considered 5-15 cancellation responsibilities.

Results - Overview Average for 3 groups: 1st group considered an average of nd group considered an average of 7.7 3rd group considered an average of 9.5 An analysis of variance between groups showed that the time on task had no significant effect on the number of cancellation responsibilities considered.

Discussion Experiment showed a strong main effect of providing the full USAP on the number of responsibilities considered. Participants who received only the cancellation scenario considered, on average, a third of the responsibilities of the participants who received the full USAP. This indicates that the full USAP helped the participants in remembering which responsibilities to consider.

Discussion The absence of a correlation between the time on task and performance showed no speed/accuracy tradeoff. However, administering the full USAP allowed participants to create a more complete solution in nearly the same amount of time. Though the group was small, factors such as self- reported experience or familiarity with technology did not interact with experiment conditions.

Discussion For 15 of the 19 Cancellation Responsibilities (CRs), Chi-squared tests revealed no significant difference in training conditions, however the following was observed: CR1, CR5, and CR10 were considered by the majority of participants. CR16 was not considered by a single participant. CR4, CR13, CR17, and CR18 were considered very differently between those who received the cancellation scenario only and those who received the full USAP.

Conclusions Using a full USAP increased the number of responsibilities considered. Participants who used all 3 parts of the USAP were able to identify 3 times as many cancellation responsibilities, in the same amount of time and without having more work experience or formal training prior to the task.

Conclusions Assumption in SE world that programmers know how to implement whatever requirements are given to them. Mostly true, however, not the case for usability. Usability issues have seen to comprise more than 60% of requirements-related defects in some professional software projects. Usability heuristics for software design are not obvious and should be made more explicit to software designers as well as SE students.