1. End User Architecting Vishal Dwivedi Institute for Software Research Carnegie Mellon University vdwivedi@cs.cmu.edu Advanced User Interface Software 1

2. Carnegie Mellon University, School of Computer Science If you build it, they will come… 2 “Hundreds of end user programming tools have been created in this hope. Not every tool succeeds. Many of them are built at a high cost, and are impoverished…” Life may have been much better if everyone did not start from scratch…

3. Carnegie Mellon University, School of Computer Science Agenda Motivation/Definitions End User Architecting Examples Abstractions for dataflow, pub-sub, … Style and the user interface elements Analyses Composition of user interfaces (and why it is a hard problem) A framework for End User Architecting 3

4. Carnegie Mellon University, School of Computer Science Definitions “Software Architecture” ‘‘The software architecture of a computing system is the set of structures needed to reason about the system, which comprise software elements, relations among them and properties of both.’’ – Garlan and Shaw 4 High level of design abstractions and analysis Emergent properties Reuse of design styles, patterns, and frameworks

5. Carnegie Mellon University, School of Computer Science Definitions “Architecture Style” A style defines the computation model, vocabulary consisting of elements such as components and connectors, their interfaces, and the constraints that they must obey. Examples: dataflow, pub-sub, client-server, pipe and filter 5 Gregory R. Abowd, Robert Allen, David Garlan, Using Style to Understand Descriptions of Software Architecture,” SIGSOFT ’94:

6. Carnegie Mellon University, School of Computer Science 6 “Architecture Style” Factors out common infrastructure. Allows configuration with additional details. Captures the domain elements and constraints on their usage. Are translated into required technology-specific vocabulary. Definitions

7. Carnegie Mellon University, School of Computer Science Recalling Brad’s Lecture “Layers of UI software” 7 Application Higher Level Tools Toolkit Windowing System Operating System

8. Carnegie Mellon University, School of Computer Science Recalling Brad’s Lecture For example: SUIT’s architecture 8 Pausch, R., Conway, M., & DeLine, R. (1992). Lesson Learned from SUIT, the Simple User Interface Toolkit. ACM Transactions on Information Systems, 10(4), 320-344.

9. Carnegie Mellon University, School of Computer Science Architectural Layer Adding an additional architecture Layer 9 Execution Platform Architectural Layer User interface Visual Language Domain-specific architectural styles Platforms like SOA, SCA etc Guidance Translation MonitoringAdaptation Analysis Code Generation

10. Carnegie Mellon University, School of Computer Science End User Architecting Similar to end-user programming, it recognizes up front that the key issue for end user programming is bridging the gap between available computational resources and the skill set of the users who must harness them [Nardi,93] But unlike end-user programming, it seeks to find higher-level abstractions that leverage architectural techniques for defining the domain, supporting compositions, and aiding in tradeoffs and analyses. Three key elements to the approach An architecture layer between the user interface and execution environment supports explicit representation of end ‐ user compositions A reusable style that can be specialized for specific domains A graphical front end for composition and for analyzing and executing compositions 10

11. Carnegie Mellon University, School of Computer Science Architecture layer between user interface and execution environment Raises level of abstraction, making it easier for users to understand the vocabulary Provides a basis for architectural analysis, guidance, composition Architectural styles help domain specialization Styles define the vocabulary for compositions in a domain. Domain-specific specializations for each domain further refine vocabulary, rules, and concerns 11 End User Architecture Approach

12. Carnegie Mellon University, School of Computer Science Example Pub-Sub Style (Ozone Widgets) 12 https://www.owfgoss.org

13. Carnegie Mellon University, School of Computer Science Example Pub-Sub Style (Ozone Widgets) 13 Ozone provides a framework for creation and integration of HTML widgets Widgets can publish and subscribe to channels to communicate messages. End users can specify which pairs of widgets can send/receive messages Widget A Widget C Widget B Publish Subscribe

14. Carnegie Mellon University, School of Computer Science Example Pub-Sub Style (Ozone Widgets) 14 Key features of End User Architecting: Architectural Style: Compositions are defined using a variant of a publish-subscribe style that takes into account the idea of restrictions. Architecture Representation: Ozone widget configurations are represented as explicit architectural models, that indicate which widgets are involved in a composition and the communication topology. Analysis: such as which widgets are communicating, whether there are data mismatches over publish-subscribe channels, how to restrict communication to minimize event messaging… Reuse: Component based reuse User interface: Additional widgets to inform the user about restrictions, widget communications and other analyses.

15. Carnegie Mellon University, School of Computer Science Example Dataflow based compositions 15 Sequence Receive Assign Invoke Assign Wait Assign 0 Invoke Wait Invoke Sequence While Sequence Assign Catch Assign Invoke Reply Throw Sequence X X X X Assign Wait Assign 0 Invoke Wait Invoke Sequence While Sequence Assign Catch Assign Invoke Reply Throw Sequence X X X X Assign Wait Assign 0 Invoke Wait Invoke Sequence While Sequence Assign Catch Assign Invoke Reply Throw Sequence X X X X Assign Wait Assign 0 Invoke Wait Invoke Sequence While Sequence Assign Catch Assign Invoke Reply Throw Sequence X X X X Assign Invoke Reply

16. Carnegie Mellon University, School of Computer Science 16 Root Style Specialization for Neuroscience Specialization for Intelligence Analysis Example Dataflow style vocabulary (created by refinement) Vishal Dwivedi et al. An Architectural Approach to End User Orchestrations. ECSA 2011: 370-378

17. Carnegie Mellon University, School of Computer Science Example Dataflow UI elements 17 Workflow Editor YUI Layout Perspective (Compose, Execute, Analyze) YUI Layout Palette Module Canvas Service (Container) Data Service Multi Port Service YUI framework defines the GUI widgets, events and controls, utilities A style maps these compositions to an architectural representation This is analyzed for various quality attributes.

18. Carnegie Mellon University, School of Computer Science Example Dataflow analyses 18 Architectural specification allows domain-specific analyses, such as: Data mismatch Security Ordering Domain-specific constraints (e.g. above diagram before a brain image is visualized it should be registered and converted in a specific format.) Resolving Data Mismatches in End-User Compositions, ISEUD 2013

19. Carnegie Mellon University, School of Computer Science Example Dataflow Style (Widgets) 19 Key features of End User Architecting: Architectural Style: Compositions are defined using a variant of a dataflow style that is specialized for a domain Architecture Representation: Workflows are instances of the style. Analysis: custom analyses for data-mismatch, security/privacy violations, ordering analyses, etc. Reuse: Reuse of component vocabularies, compositions, etc. User interface: A dataflow based interface having element vocabulary defined in a domain-specific style.

20. Carnegie Mellon University, School of Computer Science The Bigger Problem 20 Creating compositions today is difficult for end users due to: Complexity due to low-level details For example, parameters, file systems, paths, operating systems, etc Lack of support for error resolution For example, Analyses for quality attributes such as security and privacy and other syntactic and semantic problems. Conceptual mismatch For example, “Remove Image Noise” as opposed to calling the specific program(s) to perform this function. Composing UI components is even harder!

21. Carnegie Mellon University, School of Computer Science Composing User interfaces with Interviews 21 The premise: Object oriented User interface code Separate interactors Iconic interfaces Tradeoff at the level of GUI consistency. Mark A. Linton, John M. Vlissides, Paul R. Calder: Composing User Interfaces with InterViews. IEEE Computer 22(2): 8-22 (1989) But what if GUI objects have semantic meanings?

22. Carnegie Mellon University, School of Computer Science Building blocks for reusability 22 Anders I. Mørch: Application Units: Basic Building Blocks of Tailorable Applications. EWHCI 1995: 45-62 Tailoring by: Extension and refinement Specialization of components and virtual binding But who does that?

23. Carnegie Mellon University, School of Computer Science 23 But certain communities have handled this well E-sciences (e.g. LONI pipeline, Taverna) Things they did well: Componentization of functionalities as services Pluggable visual components, with search and discovery And most importantly, creating an ecosystem with different developer roles. Loni Pipeline demo

24. Carnegie Mellon University, School of Computer Science HALO: a framework for End User Architecting 24

25. Carnegie Mellon University, School of Computer Science Current work HALO SDK (and plugins) for: Architectural representation, Packaging and reuse Data reference Multiple analyses Component registry Execution-support Hypotheses : Associating architectural specifications with user interfaces will allow a greater level of analysis and reuse. Many tool capabilities will be provided by the framework. So this will lower cost of tool development and improve quality. 25

26. Carnegie Mellon University, School of Computer Science Questions? 26

27. Carnegie Mellon University, School of Computer Science References Anders I. Mørch: Application Units: Basic Building Blocks of Tailorable Applications. EWHCI 1995: 45-62 David Garlan,Vishal Dwivedi,Ivan Ruchkin and Bradley Schmerl. Foundations and Tools for End- User Architecting. In Proc. 17th Monterey Workshop, Oxford, 2012, UK David Garlan, Bradley Schmerl, Vishal Dwivedi, Aparup Banerjee, Laura Glendenning, Mai Nakayama, and Nina Patel. Swift: A tool for constructing workflows for dynamic network analysis. http://acme.able.cs.cmu.edu/pubs/show.php?id=333, 2011. Gregory R. Abowd, Robert Allen, David Garlan, Using Style to Understand Descriptions of Software Architecture,” SIGSOFT ’94: Mark A. Linton, John M. Vlissides, Paul R. Calder: Composing User Interfaces with InterViews. IEEE Computer 22(2): 8-22 (1989) Ozone Widget Framework https://www.owfgoss.org Pausch, R., Conway, M., & DeLine, R. (1992). Lesson Learned from SUIT, the Simple User Interface Toolkit. ACM Transactions on Information Systems, 10(4), 320-344. Perla Velasco-Elizondo, Vishal Dwivedi, David Garlan, Bradley Schmerl and Jose Maria Fernandes. Resolving Data Mismatches in End-User Compositions, In Fourth International Symposium on End-User Development (ISEUD), 2013, Denmark. Vishal Dwivedi et al. An Architectural Approach to End User Orchestrations. ECSA 2011: 370- 378 27