1. Defense Of Research Ian Smith

2. Overview Motivational scenario The problem Approach Tool explanation (example) Local state problem Contribution Summary Related work

3. Scenario Alice & Bob want to change the org chart together They worked together before but now Alice travels more often... Alice is remote, Bob is at his desk Alice has a pilot, Bob has a workstation.

4. Workstation Org Chart App

5. Org Chart App On A Pilot? (Bad Idea 1) Pilot Display?

6. Org Chart App On A Pilot? (Bad Idea 2) Pilot Display

7. Pilot Org Chart Application Org ChartReports To President (John) VP of Eng. (Jane) Principal Sci. (Mary) Staff Eng. (Bill) VP of Finance (Fred)

8. The Problem? People want to work together, but have different devices Two structurally different views of same data –UIs have different underlying structures Views must be synchronized

9. Org Chart President (John) VP of Eng. (Jane) Principal Sci. (Mary) Staff Eng. (Bill) VP of Finance (Fred) PilotWorkstation Window Graphical Formulation Reports To PaletteScroller LineRect... Scroller Tree Control Tree Node Pilot App

10. Org ChartReports To President (John) VP of Eng. (Jane) Principal Sci. (Mary) Staff Eng. (Bill) VP of Finance (Fred) Pilot My World Scroller Tree Control Tree Node Pilot App Workstation PaletteScroller LineRect... Window

11. Given a UI tree A, compute a new UI tree B from A If A is computed from a model, compute B from the same model Problem Generalization AB Ultraman Model AB Ultraman

12. Approach End-programmer declares constraints on the structure of an interface –“Constraint” here is a function of a tree which computes a tree (not values!) Constraints apply over an entire input tree –“When you see a node of type foo, create a node of type bar and attach to this structure…” Constraints are applied dynamically, as the application runs

13. Using Ultraman “Ultraman” is a tool and a runtime Tool allows you to declare patterns and transformations Pattern is a structural relationship of nodes in the source tree Transformation is a computation to run when a pattern is found

17. Ultraman Tool Summary Several ways to specify computations and structural patterns Tool generates a lexer/parser specification which implements the pattern matching –Lexer/parser pair is built on ANTLR 2.20 Generated parser calls into user code to implement transformations Transformations are run after each event

18. Example App

20. Transformations? But…. Pull-Down menus have state: the currently displayed item. If we regenerate the menu how do we get the right item displayed in the Pull-Down? Column RadioButton* Pull-Down Menu MenuItem... Ultraman

21. Local State Problem Tree T1 produces tree T2 by some transformation F. User manipulates T2 in some way, modifying T2’s state. We make a change to T1 and rerun transformation F producing tree T3. How do we reconcile the freshly created T3 with the old, stateful tree T2?

22. Value Numbering Nodes created by the transformations are given a “label” called a value number The value number (VN) is a representation of “where and when” in the transformation node was created. Nodes with the same VN are from the same “spot” in the transformation process.

23. Shadow Objects Shadow objects are objects which “represent” interior nodes in the UI tree. Shadow objects mimic the API of an interactor, but actually buffer calls End programmers build dynamic structures underneath shadow objects. Effectively, shadow objects build a “shadow” version of the interactor hierarchy during the transformation.

24. Grafting Algorithm After the transformation is complete... Walk the interface on the screen’s tree (T2) collecting pairs (VN,node). Walk the shadow interface’s tree (T3) and compare VNs of its nodes to the table. If a VN is in the table, exchange the corresponding nodes. Replace the interface on the screen with the shadow.

25. Grafting Algorithm Example N1 (A) N3 (C)N2 (B) Currently On Screen N1 N6N2 Result Nodes To VN Table N2 N1 N3 B A C N4 (A) N6 (D)N5 (B) Shadow Version

26. Issues With Grafting Value numbering is a sensitive and bottom- up change metric. Changes at the “top” of a tree can be missed due to shadow objects. End-programmer can be surprised when objects which are created never appear! Garbage collector can be a problem...

27. Contribution Summary Developed a taxonomy for multi-viewed transformation (4 categories) & identified salient characteristics of multi-viewed transformation system (7 properties) Implemented a specification tool for composing patterns and transformations

28. Contributions (cont’d) Developed and implemented algorithms for converting specifications into grammars Developed and implemented algorithms for handling the local state problem Implemented 2 demo applications –Cards: Transform to a very different device –Contact: Transform someone else’s data model

29. Related Work UI Software (excluding toolkits) –Patterson & Hill, Foley/Szekely, Olsen Constraints –Borning, Hudson, Myers, Vander Zanden Compilers/Parsing –Early, Parr, Graham & Glanville, Brotsky/Wills, Alpern Transformation Systems –TXL, REFINE

30. Characteristics Of A Multi- Viewed Transformation System Semantic Predicates Aggregation Multiple & recursive patterns Input & output trees have different shapes Control the progress of the pattern match Multi-tree input Take input from a model

31. Taxonomy Of Transformations Filtering - projection (N to 1) –Aggregating - Computing a function of the entire input Reordering - order change only (1 to 1) Decorating - adding nodes to the input (1 to N)