Building Objects and Interactors for Collaborative Interactions with GASP Thierry Duval and David Margery IRISA - Siames

Our Background

GASP : overview n An abstract Object Oriented framework êObjects in GASP = simulation objects n Its implementation is C++ based n A run-time platform which deals with : êNetwork programming ê3D visualization ê3D interactions

Our Specifics : GASP is not user-centered n Collaboration could also happen with or affect autonomous agents n The number of workstations involved is not limited to the number of users n Because of autonomous agents, a common notion of time has to be maintained across all workstations êTimestamps are the basis of our consistence mechanism

Outline n A short presentation of GASP n How does GASP deal with distribution issues n How GASP enables shared interactions within 3D virtual environments n Conclusion and ongoing work

Framework - Overview n An object in GASP is publicly seen as: êA number of public data fields we call outputs êA frequency at which its behavior must be calculated n And privately it also has: êA number of private data fields called inputs êA calculation object holding all the necessary data and member functions (especially calculate) for the calculation of its behavior

FC2:FollowerCalculFC1:FollowerCalcul Framework - Calculation Objects n Manage the creation and connection of the inputs of their simulation object n Read the inputs of their simulation object n Calculate and produce the outputs of their simulation object position F1:Follower Followed positionposition F2:Follower Followed position

Framework - Frequencies n Each simulation object has its own frequency êThe connection between input and outputs is an abstract one for the programmer êInterpolation or extrapolation is fundamental to the framework êWhether the connection is local or uses a network is of no importance when the entity is written FC2:FollowerCalculFC1:FollowerCalcul position F1:Follower position F2:Follower Followed position 50hz25hz Followed position

GASP’s Distributed Run-Time Platform n Communication êHow is the data connection between simulation objects implemented ? n Synchronization êHow is the notion of time kept meaningful across a network of workstations ? n Consistency êHow is the virtual environment kept coherent ?

Distribution - Communications FC2:FollowerCalcul FC1:FollowerCalcul position F1:Follower Followed position position F2:Follower Followed position 50hz 25hz Process A Process B position F1:Follower (Mirror)

Distribution - Synchronization n Algorithm (parameterized by latency) êProceed to simulation step of date T if all the update data of simulation step of date T-dt-latency from the other controllers is available (dt is the simulated time spent during one simulation step) êSend update values and repeat êFast controllers have to wait for slower ones n Simulation step T takes place while update values of T - dt transit through the network êControllers do not have to wait for update values

Distribution - Consistency n Extrapolation avoids waiting for exact values êGood approximations are available for date T êAnd for date T - dt if needed depending on latency n Timestamp of update values is in the past êValues used locally are still extrapolated, but from a more exact base êNo disruption in the nature of the signal : no artifacts are introduced with this method êBut, a constant delay (latency) is introduced in the propagation of signal changes

Sharing 3D Virtual Environments n Simulation objects can have 3D representations n The visualization of the virtual environment and its contained objects is also a simulation object n We can use several visualization simulation objects êDistributed on several workstations using the GASP run-time kernel, thus sharing VEs

Interacting within VEs n By using « Interactors » simulation objects êEntities that provide outputs for interaction ê2 kinds: Low-Level and Advanced Interactors n These interactors act upon interactive objects êExisting entities ready for interaction êSpecialized entities that have been made interactive using inheritance

The Interactors n Low-Level Interactors (Drivers) êUsed for hardware device encapsulation êProvide Raw data n Advanced Interactors êBuilt upon Low-Level Interactors êProvide Higher-level data

Low-Level Interactors n A “Driver” is a simulation object: êTypically with no inputs êThe calculation part deals with the hardware êWhich provides raw outputs such as: any DoF (3D positions, 3D orientations, …) any meaningful output (blood pressure, …)

A Led-Tracker Driver n Tracks 4 LEDs (for the example) n Gets these 3D positions from a serial link to the physical device tracking the LEDs n Provides the 3D positional outputs of the LEDs Calculation (RS 232) position1 position2 position3 position4 LED Driver

Using the Led-Tracker Driver n To drive some geometrical components... Calculation (RS 232) position1 position2 position3 position4 LED Driver CC1:CornerCalculation position C1:Corner CC2:CornerCalculation position C2:Corner CC3:CornerCalculation position C3:Corner CC4:CornerCalculation position C4:Corner position

Using the Led-Tracker Driver

Advanced Interactors n Are pure software n Make a higher-level software interpretation of the drivers outputs n Are able to send messages to interactive objects: êTo take control of them … êTo release them...

A 3D LEDs-Based Interactor Calculation (RS 232) position1 position2 position3 position4 LED Driver CC1:CornerCalculation position C1:Corner CC8:CornerCalculation position C8:Corner 3D Mouse Calculation positionD1 positionD2 positionC1 positionC2 3D Mouse positionC3 positionC4 positionC6 positionC5 positionC8 positionC7 position... message for control taking proposal Grouping the LEDs Analysis of the positions

CC8:CornerCalculation CCC8:ControlledCornerCalculation A 3D LEDs-Based Interactor Calculation (RS 232) position1 position2 position3 position4 LED Driver CC1:CornerCalculation position C1:Corner position C8:Corner 3D Mouse Calculation positionD1 positionD2 positionC1 positionC2 3D Mouse positionC3 positionC4 positionC6 positionC5 positionC8 positionC7 position... position

Using the 3D Mouse (1/3)

Using the 3D Mouse (2/3)

Using the 3D Mouse (3/3)

Sharing interaction n The visualization objects also provide interactors for keyboard and mouse n We share interaction: êEach user can act upon a different object of the environment êSeveral users can act upon the same shared object if the interactive object is able to handle multiple interactors…

Building Interactive Objects (1/2) n From scratch êIt is the hard way to do this... n Providing Interactive classes to inherit from êWhat about already existing simulation objects ? n Modifying existing simulation objects êWhat about software engineering point of view ? n Inheriting manually from existing objects êQuite cumbersome but possible...

Building Interactive Objects (2/2) n Parameterized inheritance: the best solution ! êC++ template inheritance êAllows effective code reuse n Template interactive classes (adapters) with 2 parameters : êReady to inherit from the first parameter class êAble to create a new calculation object inheriting from the second parameter class

The “MouseFollower” adapter n Is able to force the position of a simulation object to the one of the Mouse n Knows how to handle the “asking for interaction” message class Corner ; class CornerCalculation ; template class MouseFollower : public initialObject { …} MouseFollower anInteractiveObj ;

The “MouseFollowerCalculation” n Inherits from the calculation it has to replace n Redefines the effective calculation: êCalls the inherited calculation (position and orientation for example) êEnforces the position of the object to the proposed position êThus, allows the orientation to be up to date...

Example of interaction

Reuse of adapters for interaction n The “MouseFollower” class and its associated calculation: ê Have been used for several simulation objects followers geometrical elements (cube’s sub-elements) targets of the museum’s visitors êHave been refined to be able to manage any parameter of a simulation object according to its type and name

Conclusion and ongoing work n We believe GASP enables varied forms of collaborations in Virtual Environments between several end-users: êon the same machine or not êon the same simulation object or not n Effective code reuse to obtain interactive objects using our adapters n We still have to learn on the specifics of human-human collaboration (awareness)