Assembly Simulation on Collaborative Haptic Virtual Environments Rosa Iglesias, Elisa Prada Sara Casado, Teresa Gutierrez Ainhoa Uribe, Alejandro Garcia-Alonso Fundacion Labein Universidad del Pais Vasco (UPV-EHU)

Outline Objectives Haptic devices, haptic rendering Haptic Assembly Simulation Collaborative Haptic Assembly Simulation Network Topologies Client/Server (two implementations) P2P Conclusions

GOAL: Create a Distributed Virtual Environment for collaboration, whereby users can carry out assembly and maintenance tasks VE Assembly + simultaneously + haptic devices to design and evaluate computer generated mock-ups prior to building any physical prototype. Collaborative Haptic V Environment for Assembly Tasks

Haptic devices

Haptic rendering “Haptic rendering is the process of computing and generating forces in response to user interactions with virtual objects” (Salisbury, 1995). “Haptic rendering allows users to “feel” virtual objects in a simulated environment” (Salisbury, 2004) –To touch objects, move them,… –To create different haptic effects: texture perception, deformable elements or collision impacts. Haptic devices require an update frequency of 1000 Hz.

Haptic loop Haptic rendering An update frequency of 1 KHz means:

Haptic assembly simulation (HAS) Description video

Introduction: HAS Examples of assembly constraints: –Assembly along an axis –Assembly along a common plane

Haptics require new architectures Classical simulation loop : –Process input events (move objects) –Simulation –Image rendering – Haptic rendering ???  it is not possible Due to high frequency of haptic rendering: required two independent processes (threads).

HAS : two processes Get last position Simulation (collisions, constraints, …) Image rendering Send “haptic requirements” Read last position (from haptic device) Send last position “haptic requirements” received ? Compute force (haptic rendering) Send force to haptic device Simulation (20-60Hz)Haptic (1000 Hz) Problems : (1) haptic cycles without simulation updates, (2) updates mean brusque changes (haptic feedback instability)

Objectives & Challenges -Broad objective: Several users can work simultaneously to achieve a common goal using haptic devices in assembly operations. -Challenges: Consistency (virtual scene synchronization) must be guaranteed, because users simultaneously interact with the same scene Effective and stable haptic feedback when users collide or assemble their grasped objects. Scalability: number of users, virtual objects and so on.

Objectives & Challenges KEY FACTOR: Network conditions –Delay –Variation of delay (jitter) –Message Loss,…. They affect: VE consistency and user perception –Visually: Sluggish scene update of the remote object –Haptically Unstable haptic feedback

Objectives & Challenges in CHAS GOAL: Collaborative Haptic Assembly Simulator (CHAS),GOAL: Collaborative Haptic Assembly Simulator (CHAS), whereby two users can simultaneously carry out assembly tasks using haptic devices. Two types of interaction: –INDEPENDENT INTERACTION: interacting with static objects (not grasped). –DEPENDENT INTERACTION: depends on other user’s action.

Objectives & Challenges in CHAS Two types of dependent interaction: –Dependent collision: when both grasped objects collide –Remote assembly: when a grasped object is being assembled into other grasped object when a grasped object is being assembled into other grasped object CHALLENGES in CHAS In the case of a remote assembly + dep. coll.: consistency must be guaranteed. each user must receive adequate haptic feedback

To build a distributed VE: Client/Server (C/S) Peer-to-Peer (P2P) Mixture of them C/S P2P Network topology Multiple Servers

Client/Server (A1) PROS: Consistency is automatically satisfied, since data are validated centrally, and then distributed to clients. CONS: computations for simulation are made sequentially A1 Simulation or validation: check if new object position is colliding or computes the assembly constraints Simulation

Client/Server (A3) Observing its disadvantage: sequential computation. A new C/S architecture was designed. PROS: well-balanced distribution of the workload and parallel computation. CONS: efficiency comes from the worst computer conditions A2 Consistency: server checks if there is any inter-object penetration. In that case, synchronise user’s scenes Consistency Simulation

Network topology: Analysis using C/S RESULTS using C/S architectures: With A1 and A3: several users can simultaneously interact using keyboard/mouse. A3 seemed to have better performance when > 2 users A user can haptically interact while others watch with A3. After experiments, it was shown that simulation/validation must be placed at each user for avoiding unstable haptic feedback, even with independent interaction. A1 and A3 are very sensitive to the network delay.

Network topology: Analysis using C/S They address: –collaboration taking turns (any delay) –cooperative manipulation = a carrying task Two users grasping the same object. Only low-delay interaction between both users, because haptic interaction is affected. Two goals still not achieved : –simultaneous assembly tasks –high delay

Network topology: Analysis C/S vs P2P Comparing C/S and P2P: C/S: Consistency  is more easily achieved. Local scene update after a round-trip delay to the server. P2P: Consistency  challenge. Update delay is only one-way  - Updates are less dependent on network conditions. - Permit a better performance with haptic interaction. - Potential to scale to a larger number of users. Then, a P2P architecture is chosen

Consistency-maintenance scheme CH ASCHASIn our research, to build the Collaborative Haptic Assembly Simulator (CHAS), whereby two users can simultaneously carry out assembly tasks using haptic devices. A new consistency-maintenance scheme has been designed and tested.

Consistency-maintenance scheme Main features: Gives priority to the validation of interactions between objects grasped by users for fast haptic response. Small messages size: < 200 bytes Not need to store and manage a history of movements. Not specific messages to deal with inconsistency because each user manages consistency locally.

Experimental set-up Many experiments were carried out varying end-to- end delays: less than 1 ms, 50 ms, 100 ms, 200 ms and with jitter (0 ms – 100 ms). The experimental platform:

Experimental set-up The experimental platform: –Experimental platform: 3 computers via a 100Mbps links connected by two high-speed switches. –Two computers are each connected to an OMNI device. –The other computer to simulate the different values of delay and jitter.

Results between Labein & QUB: VE & tasks Virtual scene: Electrical box for an aircraft engine User Labein grasps the handle User Queen’s grasps the screw Perform collisions Assemble both grasped objects CAD design: Electrical box for an aircraft engine by SENER

Results between Labein & QUB: Consistency Consistency after a dependent collision

Results between Labein & QUB: Forces Collision force magnitude after a dependent collision

Results between Labein & QUB: Forces Force magnitudes during a remote assembly

Results between Labein & QUB: Conclusions With the consistency-maintenance scheme in CHAS, users had a realistic sensation of the VE and the performance was satisfactory. CHAS provided an adequate haptic interaction when both users perform remote assemblies or dependent collisions. Such haptic feedback showed that the sense of co-presence between the users continue to improve, with regard to only visual feedback.

THE END QUESTIONS ? THANK YOU VERY MUCH ! MUCHAS GRACIAS ! ESKERRIK ASKO !

DONOSTIA-SAN SEBASTIAN