1. Performance Evaluation of a Multi-Threaded Distributed Telerobotic Framework Mayez Al-Mouhamed, Onur Toker, and Asif Iqbal College of Computer Science and Engineering King Fahd University of petroleum and Minerals Dhahran 31261 Kingdom of Saudi Arabia

2. OUTLINE Background on telerobotics Background on telerobotics A multi-threaded distributed framework A multi-threaded distributed framework Server module Server module – Distributed framework Client module Client module – Distributed framework Performance Performance – Distribution of inter-arrival times of force packets only – Distribution of inter-arrival times of force packets with active video – Distribution of inter-arrival times of force packets during transfer of a video frame – Distribution of inter-arrival times of video packets with active force thread – Thread engineering Conclusion Conclusion

3. Telerobotics is an approach that extends a human operator’s sensing and manipulating capabilities to a remote environment. Telerobotics is an approach that extends a human operator’s sensing and manipulating capabilities to a remote environment. Telerobotics has numerous applications in hazardous or hostile environments. Telerobotics has numerous applications in hazardous or hostile environments. The purpose of this work is to design a reliable and efficient man-machine interface between a slave arm (robot) and a master arm over a network which preserves a large degree of operator dexterity. The purpose of this work is to design a reliable and efficient man-machine interface between a slave arm (robot) and a master arm over a network which preserves a large degree of operator dexterity. Computer vision and force feed techniques are implemented to enhance the maneuverability of the operator by providing him force feeling. Computer vision and force feed techniques are implemented to enhance the maneuverability of the operator by providing him force feeling. Background on Telerobotics

4. To avoid communication delays in frequently interacting telerobotic systems, the operator will have a library of local automation tasks. Only coarse operator supervision is required. For example, picking up a tool, moving tool to a certain location, returning tool to initial position, etc. To avoid communication delays in frequently interacting telerobotic systems, the operator will have a library of local automation tasks. Only coarse operator supervision is required. For example, picking up a tool, moving tool to a certain location, returning tool to initial position, etc. Telerobotics is a multidisciplinary area integrating knowledge from robotics, real- time operating systems and programming, network programming, 3D stereo vision, mechanical linkage engineering, ergonomics, etc. Telerobotics is a multidisciplinary area integrating knowledge from robotics, real- time operating systems and programming, network programming, 3D stereo vision, mechanical linkage engineering, ergonomics, etc. Background on Telerobotics

5. Client-Server distributed component telerobotic system. Client-Server distributed component telerobotic system. A telrobotic server has components (PUMA, Force Sensor, and Decision Server) and interfaces (Proxy Robot, Force Sensor, and DecisionServer). A telrobotic server has components (PUMA, Force Sensor, and Decision Server) and interfaces (Proxy Robot, Force Sensor, and DecisionServer). One or more telerobotic client components One or more telerobotic client components An integrated scheme of client-server components An integrated scheme of client-server components A multi-threaded distributed telerobotic system A multi-threaded distributed telerobotic system A Multi-Threaded Distributed Framework

6. PUMA Component PUMA Component Acts as a software proxy (thread) of the slave robot arm diagram Acts as a software proxy (thread) of the slave robot arm diagram –Public methods (Initialize, incremental motion, etc) –Public properties (robot states) –Public Events (motion commands and communication) Force Sensor Component (wrist sensor) Force Sensor Component (wrist sensor) –Public propertie Real-time streaming of force data from server to client Real-time streaming of force data from server to client Server module

7. Distributed framework at server

8. Client GUI (Graphic User Interface) for remote testing and maintenance operations Client GUI (Graphic User Interface) for remote testing and maintenance operations IDecisionServerto interface to server through.NET Remoting IDecisionServerto interface to server through.NET Remoting All the definitions to execute methods on PUMA and ForceSensor components All the definitions to execute methods on PUMA and ForceSensor components After initialization, the client carries an empty un- referenced copy of IDecisionServer After initialization, the client carries an empty un- referenced copy of IDecisionServer Following the network connection, the client can reference any instance of DecisionServer Following the network connection, the client can reference any instance of DecisionServer Client module

9. Distributed framework at client

10. Distribution of inter-arrival times of force packets only

11. Distribution of inter-arrival times of force packets with active video

12. Magnified plot of inter-arrival times of force packets under active video

13. Distribution of inter-arrival times of force packets during transfer of a video frame

14. Distribution of inter-arrival times of video packets with active force thread

15. Thread engineering Break-down independent operations into separate threads Break-down independent operations into separate threads Implement threads to allow maximum thread concurrency Implement threads to allow maximum thread concurrency –Thread-1: transfer video from camera to buffer-1 –Thread-2: transfer of video over network from Buffer-2 –Pipeline threads 1 and 2 using time overlapping execution

16. Distribution of inter-arrival times of video packets with active force thread

17. Conclusion An integrated scheme of client-server components An integrated scheme of client-server components A multi-threaded distributed telerobotic system A multi-threaded distributed telerobotic system Thread engineering (partitioning, concurrency, and priority) enables optimizing real-time processing Thread engineering (partitioning, concurrency, and priority) enables optimizing real-time processing –Picture (0.5x2 MB) copy thread takes 24 ms –Video transfer is 86.5 ms (single buffer) and 56 ms (double buffer) which gives a rate of 17 stereo frames per second –Worst inter-arrival times for reflected force is 9 ms. Implemented system provides reasonable QoS for telerobotics operating over a LAN Implemented system provides reasonable QoS for telerobotics operating over a LAN