1. UofC Large Display Framework
Part I: Refactoring
Fabrício Anastácio
September, 2007

2. Outline Part II: Part I: Toolkit Layer Basic Concepts
Interface with the Framework
Extending the Framework Classes
Types of Strategies
Some Toolkit Strategies
Creating an Application
Conclusion
Part I:
Basic Concepts
Objectives
New Structure
I-Buffer Layer
Framework Layer
Framework Classes

3. The Problem Have responsive interaction in large display applications
Provide better application performance since higher resolution means more pixels to be processed
Simultaneously display a large number of objects that interact with the user and among themselves
Have a reusable code base dealing with the common issues that need to be addressed in this type of applications

4. The Solution
Use buffers as intermediate data structures for communication among objects
Stacks of buffers keep positional information that can be quickly accessed by the objects
Local awareness: objects know about the values in the buffer stack under their current position
Buffers can be used to steer objects and store their properties without the need of complex operations among objects
Encapsulation of this structure in a framework

5. Definitions
Visualization Objects: carry the information to be communicated [1]
Control Structures or Interface Components: guide the behavior of the visualization objects and are manipulated by user interactions [1]
For our context, the term Visualization Component (or just Component) is used to refer to both cases
Components are organized in a Composite Pattern, so that they can be composed of other components (becoming a Container) while being treated uniformly
[1] A. Miede, Realizing Responsive Interaction for Tabletop Interaction Metaphors, 2006

6. Interaction Buffers (I-Buffers)
Data structure that holds positional data relative to some property or steering information of a visualization object
Active buffers: buffers owned by an interface component that influence other interface components and visualization objects [1]
Passive buffers: buffers that a visualization object or interface component looks into for specific information (usually, a passive buffer of a component is the active buffer of another) [1]
Private buffers: buffers that hold information that only their owner uses and should not be made visible to other components
[1] A. Miede, Realizing Responsive Interaction for Tabletop Interaction Metaphors, 2006

7. Why refactor?
Applications, specialized components, and framework code were mixed together
New applications required changing/adjusting the framework to add new components
The more applications used the framework, the more complex it became
Code became too long, complex, and interdependent
Compiling time became unnecessarily too long

8. Objectives Keep the same functionality
Make the framework independent of the applications that use it
Make the buffer data structure independent of the framework
Separate specialized components from the framework and individual applications
Make large display applications easier to create
Keep a good performance
Add some new features (maybe)

9. Large Display Framework
The New Structure
Specific Application
Component Toolkit
Large Display Framework
I-Buffer

10. I-Buffer Layer Separated DLL
Composed of a single template class: IBuffer
Holds the buffer data + the operations over the data:
Number of channels;
Real dimensions (width & height);
Virtual dimensions (width & height); (...)
Resizing (empty, interpolation, copy, fill)
Clearing
Filling whole buffer & quad filling
Checking if point is inside (actual & virtual)
Accessor methods (actual & virtual)
Image output (debugging)
IBuffer

11. Framework Layer Separated DLL, dependent on the IBuffer DLL
Encapsulates the mechanism of component interaction via the buffer paradigm
Should be independent of OS, compiler, and rendering, GUI, and device APIs

12. VisComponent Main element of an application
Represents visualization objects as well as interface components
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Manages a component’s appearance and behavior
It is a CompositeNode and has a state, a state stack, a collection of active buffers, a collection of passive buffers, and a chain of strategies

13. CompositeNode
Composite Pattern: “Composes objects into tree structures to represent part-whole hierarchies. Composites let clients treat individual objects and composition of objects uniformly” [2]
Holds a reference to its parent and a list of references to its children
The order in the list of children is important when drawing the composition (children are rendered in reversed order)
Allows moving a node from one parent to another, bring a node to the front, add and remove children
A leaf node is indicated by a flag
The VisComponent class is a CompositeNode
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Provides the functionality for the implementation of a Composite Pattern
Composite Pattern: “Composes objects into tree structures to represent part-whole hierarchies. Composites let clients treat individual objects and composition of objects uniformly” [2]
Holds a reference to the parent and the collection of references to the children of a node
A leaf node is indicated by a flag
The children are kept in a list. The order of the list is important when drawing the composition (children are rendered in reversed order)
Allows moving a node from one parent to another, bring a node to the front, add and remove children
The VisComponent class is a CompositeNode (i. e., it is a subclass of CompositeNode)
[2] E. Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software. 1994

14. VisComponentState Transitory data of a VisComponent object:
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Transitory data of a VisComponent object:
3D position;
Rotation angle (in radians);
Dimensions (width & height);
Scale factor along the x-, y-, and z-axis.
Linear interpolation between the data of two VisComponentState objects
A VisComponent object has a current state and an auxiliary stack of states. This allows saving and retrieving previous/alternate states for a single component in a simple way
State change animation is done using the state stack of a component and the interpolation method
Holds the transitory data of a VisComponent object:
3D position;
Rotation angle (in radians);
Dimensions (width & height);
Scale factor along the x-, y-, and z-axis.
Provides linear interpolation between the data of two VisComponentState objects
A VisComponent object has a current state and an auxiliary stack of states. This allows saving and retrieving previous/alternate states for a single component in a simple way
State change animation is done using the state stack of a component and the interpolation method

15. IBufferProxy
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Since the IBuffer class is a template, this provides a safe abstract reference to it without knowing its actual data type
Holds a reference to the IBuffer object (void*), the type of the buffer (user defined constant), and a reference to the owner of this buffer (needed when calculating buffer coords)
A VisComponent has a list of IBufferProxy objects for its active buffers and another for its passive buffers

16. VisComponentStrategy
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Determines how a component should behave and draw itself
Strategy + Decorator Patterns
Strategy Pattern: “Defines a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it” [2]
Decorator Pattern: “Attaches additional responsibilities to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality” [2]
Replaces component subclasses from the original framework
Follows the OOD principle: “Favor object composition over class inheritance” [2]
[2] E. Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software. 1994

17. VisComponentStrategy
Different aspects of a component are encapsulated in different strategies, allowing them to be used by other components without establishing a rigid hierarchical relationship
It can provide a component with strategies for:
Buffer initialization;
Drawing;
Drawing for picking;
Device pressed (inside & outside);
Device dragged;
Device released;
Device pointing/hovering (inside & outside);
Resizing;
Dropping;
Reading passive buffers (for drawing & picking).

18. VisComponentStrategy
Strategies can be linked together to form chains (or stacks), adding functionality to a component
Each strategy executes a given method and passes the same call to the following strategy (actually, the method in the last strategy in the chain is executed first and propagation goes from back to front)
VisComponent
RNTStrategy
BorderStrategy
ImageStrategy
execution

19. LargeDisplayManager Manages all the components in an application
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Manages all the components in an application
Holds a collection of all the current components (hash map)
Holds the dimensions (width & height) of the display
Has a list of components that should be updated (i. e., checked for changes in the composition tree; a.k.a. dropping)
Keeps track of how many users are using the application simultaneously, their touch indicators, and currently selected components
Distributes IDs to the components using an auxiliary stack for freed IDs

20. LargeDisplayManager
Keeps the components organized as a single composition tree, holding the tree root (the “background”)
Manages component rendering, picking, and event forwarding
Keeps track of the frame rate via a FrameTimer object
Must be extended for rendering API-specific methods: initialization of touch indicators, picking for a component, and picking for a container
The subclass should also implement the component initialization, where all the initially available components are added and initialized
Manager
Root
Leaf1
Container
Leaf2
Leaf3

21. FrameTimer
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
It is used by the LargeDisplayManager to measure the time past between rendering intervals
Keeps the intervals in a queue to calculate the average frame rate
Since obtaining the system time is platform dependent, this class should be extended by a platform specific class that provides access to the current system time and calculates the frame rate based on the unit provided by the system

22. TouchIndicator
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
Draws a touch indicator at the point of the user interaction
Defines a color, position, scale factor, width, height, and a flag indicating if an extent should be draw around the indicator
The actual drawing for the indicator and its extent should be defined by a subclass using a rendering API

23. Utils A Point3f class is defined to hold 3D point data
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f
A Point3f class is defined to hold 3D point data
A VectorTemplate class is provided for simple vector math. Common types are already defined, such as: Vector2f, Vector3f, Vector2i, Vector3i, etc.

24. Large Display Framework
IBuffer
TouchIndicator
LargeDisplayManager
VisComponentStrategy
VisComponentState
CompositeNode
VisComponent
IBufferProxy
FrameTimer
VectorTemplate
Point3f

25. End of Part I