1. UCI Knowledge Management for Software Design David F. Redmiles David M. Hilbert, Jason Robbins, Michael Kantor, Shilpa Shukla Information and Computer Science University of California, Irvine Irvine, California 92697-3425 redmiles@ics.uci.edu http://www.ics.uci.edu/~redmiles/ or http://www.ics.uci.edu/eden/

2. UCI Overview Motivation and Problem Statement Approach — Human-Centered Software Development Usage Scenario Systems’ Theory, Implementation, Current Directions –Design: Argo Software Architecture Design –Usage Monitoring: Expectation-Driven Event Monitoring –Review: Knowledge Depot Project Awareness Tool Conclusions

3. UCI Motivation and Problem Statement

4. UCI Motivation Software engineering has become harder because software is being applied to increasingly complex problem situations. Complexity is measured in terms of people and workplace settings.

5. UCI The Challenge of Increased Complexity in Software Changed Nature –Software applications have moved from predictable, static applications to unpredictable, interactive tools. Expanded Scope –Software applications are no longer intended for a single user or small group of highly trained users. New Problem Domains –Software applications are no longer restricted to “simple” problems—mathematical models with a few input variables. Increased User Sophistication –Wide availability of personal computers has increased users’ awareness of computer potential.

6. UCI Changed Nature Software applications have moved from static to interactive tools. Human behavior is “unpredictable”—people do not always use tools as anticipated by the developers. –Incomplete understandings of tools (system model). –Incomplete understanding of problem situations (situation model). –Other cognitive limitations (e.g., attention).

7. UCI Expanded Scope Software applications are no longer intended for a single user or small group of highly trained users. –General-purpose tool sets are applied in a variety of situations in mass markets. –Special-purpose tools cannot assume long training periods because of high personnel turnover and frequent tool updates. –In any tool, the number and variety of features implies that many are infrequently accessed. Moreover, new, cooperative work tools coordinate the activities of several people.

8. UCI New Problem Domains Software applications are no longer restricted to “simple” problems—mathematical models with a few input variables. Technological advances make it possible to put software on every desk. –Tools must be able to coordinate with many dimensions of a workplace: users’ relying on information from other artifacts in the work setting, other computer tools, and other people. –New tasks challenge existing computer representations and algorithms. –Models of tasks must be more rapidly developed.

9. UCI Increased User Sophistication Wide availability of personal computers has increased users’ awareness of computer potential. Attractive interfaces in games and other personal software have raised users’ expectations. Competitive markets for many applications enable users to be “discretionary”—more demanding.

10. UCI New Demands on Design Interactive tools require a human-centered, not artifact-centered, focus. It is more difficult for designers to understand how people will use tools in their workplace than it is for them to understand the domain the tools will support. User evaluation is essential to informing tool development. Fluctuating requirements and other failures should be expected, good “fit” must evolve, new methods for collecting usage data and feedback can make evaluation more tenable and cost-effective.

11. UCI New Demands on Design (cont.) Design is an evolutionary process and continues after delivery. In new problem domains, complexity unfolds incrementally. Moreover, after tools are introduced into a community, the users develop new approaches to their problems, hence new requirements for tools. Users and other stakeholders must share ownership of systems. Acceptance of systems often fails for social and organizational reasons. Stakeholders must feel their intentions have been fairly negotiated.

12. UCI New Demands on Design (cont.) Developers need information about good design in new domains. –Developers require knowledge both about problem domains and potential solutions. Possible solutions grow daily with new software architecture techniques.

13. UCI Approach In a human-centered software development process, feedback comes from all stakeholders. Stakeholders reflect on an artifact that evolves from sketchy designs to partial, and eventually, complete prototypes. Feedback from human stakeholders is facilitated by tools; tools can provide data about actual usage and knowledge evolved from previous development experiences.

14. UCI Human-Centered Software Development

15. UCI Communication and Knowledge in the Lifecycle The human-centered lifecycle interprets software development as a process of communication (of knowledge): –Knowledge about what constitutes a good design is provided to developers. –Knowledge about actual system usage and user commentary is reported. –Knowledge about system usage and other related design knowledge is discussed among stakeholders. Three software systems support these activities: Argo, EDEM, and Knowledge Depot.

16. UCI Usage Scenario

17. UCI Developing a Travel Voucher System A software developer architects a new application, a travel voucher system. Software developers author expectations about the usage of the system; they are deployed with the system. The agents monitor system usage and may solicit user comments. Usage information, comments, and other design information is collected and reviewed. Design knowledge provides feedback for a redesign of the system.

18. UCI Design A software developer architects a new application, a travel voucher system.

19. UCI Design and Usage Software developers author expectations about the usage of the system; they are deployed with the system. This agent fires when a user edits the City or State fields while the Zip field is empty Trigger Guards Actions } } } Authored Agents {

20. UCI Design and Usage Selection of events simplifies agent definition. Agents are deployed independently of applications.

21. UCI Usage The agents monitor system usage.

22. UCI Usage Agent Notification (Optional) Agents may post messages.

23. UCI Usage User Response (Optional) Users may provide feedback.

24. UCI Review Usage information, comments, and other design information is collected and reviewed.

25. UCI Redesign Design knowledge provides feedback for a redesign of the system.

26. UCI Summary The changing nature of software places new demands on software engineering. An empirically-driven, evolutionary and human- centered model is necessary. The human-centered lifecycle interprets software development as a process of communication (of knowledge): –Knowledge about what constitutes a good design is provided to developers. –Knowledge about actual system usage and user commentary is reported. –Knowledge about system usage and other related design knowledge is discussed among stakeholders.

27. UCI Argo Software Design Environments

28. UCI Underlying Theories of Designers’ Cognitive Needs We have tried to identify what makes design challenging for people –Design feedback: Reflection-in-Action [Schoen] –Design process support: Opportunistic Design [Guindon] –Design perspectives: Comprehension and Problem Solving [Kintsch] We have specialized and applied these theories to software architecture

29. UCI Design Feedback — Reflection-in-Action Observation: Designers work through the design –Designers rarely produce a design fully formed –Incrementally frame a partial design; evaluate it; revise Suggested requirements on design environments –Support the designer in framing a partial solution –Support the designer in evaluating it –Integrate manipulation and analysis Support in Argo: Editors, Critics, To Do List

30. UCI Construction with Critical Feedback

31. UCI Design Process Support - Opportunistic Design Observation: Designers deviate from prespecified processes –Emerging issues change the design process –Cognitive costs explain some task reorderings Suggested requirements on design environments –Provide process models as resources –Use process models to keep design feedback timely Support in Argo: First-class decision model, To Do List

32. UCI Process Navigation

33. UCI Design Perspectives — Comprehension and Problem Solving Observation: Designers use multiple mental models –Work through successive levels of refinement –Restructure their thinking to address different issues Suggested requirements on design environments –Present multiple perspectives to match mental models –Help maintain mappings between mental models Support in Argo: Multiple coordinated design perspectives, customizable visualizations

34. UCI Multiple Perspectives

35. UCI Summary Tools for designers should support their cognitive needs. We have examined three theories of cognitive needs and how to support them. –Gives us a checklist of needed design environment features.

36. UCI Argo Future Apply Argo’s infrastructure to two new domains –UML: a new object-oriented design notation –CoRE: a formal state-based representation of behavioral requirements for embedded systems Evaluation of impact of cognitive support on design product and process –Feedback from Collins Avionics on CoRE tool –Planning to use Argo/UML in ugrad class in Spring Evaluation of usefulness of Argo infrastructure –Metrics on effort needed to build Argo/CoRE and Argo/UML –Feedback from other developers using Argo

37. UCI Background UML: The Unified Modeling Language UML unifies –Popular OO design notations: Booch, OMT, OOSE –Many views of the system: object model, state models, use cases, deployment model, etc. –Notations used for analysis, design, and implementation UML is well defined –UML Meta-model defines what is in a UML model –Object Constraint Language specifies system constraints –Meta-model has about 100 classes and 180 constraints (that might make good design critics) There is broad industrial interest in using UML

38. UCI UML Example (self.robot->size) / (self.worker->size) < 0.1 Company Worker Robot « Person » Employee Trainee Course 0..* 1..4 1..*

39. UCI CoRE CoRE is a formal state-based notation for behavioral requirements of embedded systems, based on SCR –Embedded systems monitor some variables and control others. The values of controlled variables depends on the current state. State transitions occur in reaction to changes in monitored variables. Examples –Traffic lights at a simple 3-way intersection Monitors sensing pads, walk buttons, and internal timers Controls color of lights and walk signs About 9 states and 12 transitions –Auto-pilot on a Boeing 777 Monitors buttons and switches in cockpit Controls flight modes and some cockpit displays Thousands of states and transitions

40. UCI Intersection Overhead View Crosswalk button 12 3 4 5 6 7 Cornell Drive Campus Drive

41. UCI Behavioral Specification Diagram Flow Warn Stop Normal Flow Warn Stop Allow2 Flow Warn Stop Walk 1 2 3 4 5 6 7 8 9 10 11

42. UCI CoRE Specifications with Feedback

43. UCI EDEM Usage Data Collection Through Expectation-Driven Event Monitoring

44. UCI Expectations in Development Developers have usage expectations that importantly affect design decisions. Developers' usage expectations are based on their: –knowledge of requirements –knowledge of the application domain –knowledge of specific user tasks, practices, and work environments –past experience developing systems –past experience using applications themselves Developers' usage expectations impact the appropriateness and usability of their designs: –accurate expectations => good designs –inaccurate expectations => poor designs

45. UCI Characteristics of Expectations Some usage expectations are represented explicitly. –e.g. those specified as requirements or in "use cases" Most usage expectations are implicit. –e.g. those encoded in window layout, toolbar and menu design, key assignments, and user interface libraries. Example usage expectations: –users complete forms from left to right and top to bottom. –frequently used features are easy to access, recognize, and use. Because most usage expectations are not represented explicitly, they often: –fail to be tested adequately –fail to be explicitly recognized by developers

46. UCI Resolving Mismatches Detecting and resolving mismatches between developers' expectations and actual usage can help improve: –design, automation, on-line help, training, and use. Once mismatches are detected, they may be resolved in one of two ways: –Developers may modify their expectations to better match actual use, thus refining system requirements and eventually improving the design. e.g., features expected to be used rarely but used often in practice can be made easier to access. –Users may learn about developers' expectations, thus learning how to use the existing system more effectively. e.g., learning they are not expected to type full URL's in Netscape's Communicator can lead users to omit characters such as "http://".

47. UCI Expectation-Driven Event Monitoring (EDEM)

48. UCI Agent Representation Agents are instances of a simple Java class w/ the following members: –Trigger: patterns of user interface (or agent) events –Guard: boolean expression involving user interface (or agent) state –Actions: pre-supplied actions or arbitrary code Triggers are continually checked as users interact w/ the application. Guards are checked if an agent trigger has been activated. Actions are performed if the guard evaluates to true.

49. UCI Triggers Triggers specified in terms of the following patterns: (1) "A or B or... " (2) "A and B and... " (3) "A then B then... " (4) "(A and B) with no interleaving C" (5) "(A then B) with no interleaving C" Where variables A,B,C are filled in by specifying: (1) a Component from the UI plus an Awt or EDEM event on that component (e.g. "TextField1:LOST_EDIT" which occurs when TextField1 is edited and then input focus shifts and editing begins in another component) (2) another Agent (e.g. "AddressDone" which occurs when another agent detects that the address section has been completed)

50. UCI Guards Guards specified in terms of the following patterns: (1) "A or B or... " (2) "A and B and... " Where variables A,B are filled in by specifying: (1) a Component from the UI and some expression involving its properties (e.g. "TextField1:value='Married'" or "Button1:count>100") (2) another Agent and some expression involving its properties (e.g. "AddressStarted:enabled=true" or "AddressDone:count>1")

51. UCI Actions Actions may include arbitrary code, but usually involve pre-supplied actions such as: –generating higher level events for further hierarchical event processing –interacting with users to provide suggestions and/or collect feedback, and –reporting data back to developers

52. UCI Integrating w/ EDEM void initialize(boolean obtrusive) –load agents, post EDEM message window if obtrusive void addMonitors(Object obj) –recursively add monitors to this component and all subcomponents void setName(Object obj, String name) –name components to be monitored that don’t have a unique label void processEvent(Event evt) –pass events to EDEM for processing void finalize() –remove monitors and send log & summary

53. UCI Summary Usage expectations: –focus data collection –raise awareness of implications of design decisions Event monitoring architecture: –distributed event filtering and abstraction –reduction of data to be reported and analyzed –independent evolution of instrumentation/application Extensible event model: –data collected/analyzed at multiple levels of abstraction –events of interest unavailable in a given event model are added by defining them in terms of available events

54. UCI Global Transportation Network (GTN) Application

55. UCI Action Sequence Dependency

56. UCI Knowledge Depot Information Review and Project Awareness

57. UCI Communication and Group Memory The Knowledge Depot, an enhanced version of GIMMe (Zimmermann, Lindstaedt, Girgensohn, 1997), is a group memory that captures email conversations and documents submitted over the web. Users may browse through the information to learn or to rediscover why different decisions were made (or are being made), allowing the user to gain some of the context in which those decisions were made..

58. UCI UCI Knowledge Depot

59. UCI Characterizing Individual and Group Awareness

60. UCI Project Awareness Project Awareness is an approach to providing communications links between individuals and all of the different aspects of the project that affect those individuals. The strength of those communication links should depend on the relevance to each individual.

61. UCI Types of Information Changes to component’s design or interface Changes to completion dates Changes to higher level design Changes to management Changes to customer requirements The knowledge that the above changes are being discussed The concerns that lead to the discussions

62. UCI Types of Links

63. UCI From Group Memory to Project Awareness Group Memories can enhance Project Awareness by allowing users to request periodic summaries of new information in topics of interest. This gives a group memory three key attributes for creating an awareness system: –Information Capture: Information captured or added to the group memory –Information Distribution: Emailed to interested users –Information Display: A list of documents added to the group memory, accompanied by brief descriptions

64. UCI Future Work How well does Knowledge Depot enhance project awareness relative to existing approaches? How accurate are our predictions on the benefits of Project Awareness? How can Knowledge Depot more effectively support project awareness? How effective are categorization techniques on personal/frequently used mailboxes as well as the less frequently used archives.

65. UCI Java Depot: Integration between Project and Individual Depots

66. UCI Conclusions

67. UCI Research Goals To develop a model of software development as a process of on-going communication; To support this model through active mechanisms in software tools; and To improve the accessibility and acceptance of usability methods by software practitioners.

68. UCI Research Approach To focus on a human-centered software development process, in which feedback comes from all stakeholders. Stakeholders reflect on an artifact that evolves from sketchy designs to partial, and eventually, complete prototypes. Feedback is not limited to human stakeholders: tools can provide data about actual usage and knowledge evolved from previous development experiences.