U NIVERSITY OF B RIDGEPORT C OMPUTER S CIENCE AND E NGINEERING D EPARTMENT By: Tamer Abukhalil Under the Supervision of: Dr. Tarek Sobh UBSwarm: Design of a Software Environment to Deploy Multiple Decentralized Robots 1

P RESENTATION OUTLINE Introduction Review of previous work Research scope Research Plan and System Architecture Experiments and Result Conclusion and Remarks 2

I NTRODUCTION This research work is the development of a software application that facilitates the rapid deployment of a multiple robotic agents. The software deploys these robots using its GUI command window and uploads program which are integrated with an embedded middleware. UBSwarm environment is the name given to the deployment interface. Figure 1: System hierarchy Robot Deployment Environment Operator Robot Agents Middleware 3

S WARM SYSTEMS Figure 2: Flying in patterns Quadrotors [2012] Biologically Inspired Functionally Inspired Modular Design Reconfigurable Systems Extendibility 4

S WARM S YSTEMS Figure 3: pulling a child using Swarm- bot [2006] 5

P RESENTATION OUTLINE Introduction Review of previous work Research scope Research Plan and System Architecture Experiments and Result Conclusion and Remarks 6

R OBOTICS D EPLOYMENT E NVIRONMENTS E-Puck [Trifa V. et al.]. DCF [Kulis et al.] MAJIC [Gregory P. Ball G. et al.] Pyro [Blank, D., et al.] Player/Stage [Gerkey et. al.] 7

R OBOTICS D EPLOYMENT E NV. The Player/Stage Its two main products are the Player robot server, a networked interface to a collection of hardware device drivers, and Stage, a graphical, two-dimensional device simulator. running this software requires variety of prerequisite libraries and each library requires another set of libraries. It was never been easy to understand how the system communicates with the actual robots. Pyro, Programming robot behaviors in Pyro is accomplished by programming high-level general-purpose programs. The Pyro language allows robot control programs written for specific robots to be used to control another types of robots. However, robot programs were written for specific motor controllers, sensors, communications protocols. Figure 4: Stage screenshot 8

R OBOTICS D EPLOYMENT E NVIRONMENTS, Separate XML files are used to define the capabilities of the various robot agents and to configure the sensors, actuators, and controllers used by the robot agents in the system. The DCF is specifically designed to control interacting heterogeneous agents. DCF uses a high-level platform-independent programming language for hybrid control called MDLE. They say the DCF is available in multiple version for commercial or academic use on the following link Figure 5: The DCF human interface (© 2008 IEEE) 1 9

R OBOTICS D EPLOYMENT E NVIRONMENTS The system provides basic components for user interaction that enables the user to add/remove robots change the robotic swarm configuration, load java scripts into robots and so on as shown in Fig 4. Authors did not explain how their formal language is translated to program specific to different robots. FIGURE 6 : The MAJIC Control Platform (© 2008 IEEE) 2 10

P RESENTATION OUTLINE Introduction Review of previous work Research scope Research Plan and System Architecture Experiments and Result Conclusion and Remarks 11

R ESEARCH S COPE The first major aspect of our problem is inspired by the recent advances in sensor miniaturization and the increasing computational power and capability of microcontrollers in the past few years. The proposed application offers customization for robotic platforms by simply defining the available sensing devices, actuation devices, and the required tasks. 12

P RESENTATION OUTLINE Introduction Review of previous work Research scope Research Plan and System Architecture Experiments and Result Conclusion and Remarks 13

R ESEARCH P LAN UBSwarm environment is a high end interface used for distributing algorithms to heterogeneous robotic agents. We are developing an environment to utilize robots that have different modular design and configuration of sensory modules, and actuators. We built 5 different robots and we ran 2 experiments to perform 2 different tasks. The system is implemented as a GUI interface to reduce efforts in controlling swarm robotic agents. 14

R ESEARCH PLAN C ONT. UBSwarm environment is a high end interface used for distributing algorithms to heterogeneous robotic agents. One of the key features of UBSwarm is configuring special programs which act as middleware that gain control over the agent’s parameters and devices. The middleware consequently allows auto-detection of the attached standardized components according to current system configurations. Figure 7: UBSwarm System Overview Robot Deployment Environment Robot Control Middleware User interface Face Detect ion Obstacle Avoidanc e Navigatio n Coordination Agent HAL Knowled ge base Device Agent Polling routine Hardware Components XbeeSonarGPS Robot N Robot 1 Device Library Figure 8: System Architecture Robot agents 15

S YSTEM A RCHITECTURE UBSwarm is an interactive Java-based application designed for extensibility and platform independence. The system establishes communications with embedded robot modules via various mediums. At the time of startup the system will expect to operator to:  Configure the system by picking the available agents, their onboard features (sensors, motors, etc.) and the services needed to accomplish each task  Or simply run the system using the last executed configurations The system is divided into two main subsystems, a robot deployment system and a robot control and translation system. 16

R OBOT D EPLOYMENT S YSTEM deployment system takes the responsibility of running actions according to the definition parameters and integrates the heterogeneous robots. Each application is implemented as a software module to perform a number of specific tasks used for sensing, decision-making, and autonomous action. The deployment system contains a developer interface, coordination agent, dynamic interpreter, and knowledge base. 1. Operator Interface: The user can interact with the computer through interaction tools which provides a list of actions/tasks and the available robotic agents. In some other parts of the interface, the user will be prompted to input the required system parameters for all sensors incorporated on robots such as the PIN location that each sensor/actuator is connected to on each robot. When connecting the robot to the USB, UBSwarm will detect the COM port automatically. The interface provides a number of tasks to be assigned the group of robots as can be seen in figure 9 (a). 17

1) O PERATOR I NTERFACE The user interface also provides a mechanism to allow users to update, remove, or add a robot application. After clicking on a particular task, the user will be prompted to decide the number of robots displayed in a list of the available robot types by manipulating the arrow buttons as shown in figure 9 (b). (a)(b) Figure 9: The interface 18

O PERATOR I NTERFACE The user will be then be asked to enter each agent’s pin locations (once for each type of robot) associated with various hardware components such as ultrasonic sensors, scan servo motors, and the n pin locations for the n-Dof arm if any is attached on the robot. A value of -1 will be assigned to pin locations of components that does not exist on the particular robot. Different sensors have different programs. The particular function will be activated only of the pin number associated with the intended sensor/hardware component. Example: In main function: Void loop() { If (trigpin>-1 && echopin>-1) Ping(); I f (URPWM>-1) PWM_Mode();. } 19

URM V3.2 U LTRASONIC S ENSOR int PWM_Mode() { // a low pull on pin COMP/TRIG triggering a sensor reading digitalWrite(URTRIG, LOW); digitalWrite(URTRIG, HIGH); // reading Pin PWM will output pulses unsigned long DistanceMeasured=pulseIn(URPWM,LOW); if(DistanceMeasured==50000) { // the reading is invalid. Serial.print("Invalid"); } else{ distance=DistanceMeasured/50; // every 50us low level stands for 1cm } return (distance); } void PWM_Mode_Setup() { pinMode(URTRIG,OUTPUT); // A low pull on pin COMP/TRIG digitalWrite(URTRIG,HIGH); // Set to HIGH pinMode(URPWM, INPUT); // Sending Enable PWM mode command for(int i=0;i<4;i++){ Serial.write(EnPwmCmd[i]); }} 20

HC-SR04 AND PING U LTRASONIC R ANGE F INDER int ping(){ long duration, inches, cm; //Send Pulse pinMode(trigPin, OUTPUT); digitalWrite(trigPin, LOW); delayMicroseconds(2); digitalWrite(trigPin, HIGH); delayMicroseconds(10); digitalWrite(trigPin, LOW); //Read Echo pinMode(echoPin, INPUT); duration = pulseIn(echoPin, HIGH); // convert the time into a distance inches = microsecondsToInches(duration); cm = microsecondsToCentimeters(duration); Serial.print("Ping: "); Serial.println(inches); return round(cm); } 21

O PERATOR I NTERFACE The computer will ask the user to connect each robot for the purpose of uploading the program on each robot as shown in figure 10 Figure10: prompt 22

2 ) R UNTIME I NTERPRETER When new devices are plugged in, system developers can upload new platform software packages specific for the use of the newly added devices (components). system developers can extend the system’s functionality by adding new service modules to the list of available modules. When new service is added to the system, the dynamic interpreter manages flow of information between these services by monitoring the creation and removal of all services and the associated static registries. The Dynamic interpreter maintains state information regarding possible & running local services. The Dynamic interpreter will be the first service created which in turn will wrap the real JVM Runtime objects. Applications Application objects Get new instance Service Service Manager Runtime agent Send service directory update (SDU) SDU = getLocalServices Program command flow Coordination Agent IfIf if service is not registered on name Get new instance Get list of registered services Get new service instant Update registry New task is added. Send msg (data) New service is added Add service(name) Fig.ure 11: Adding services in runtime 23

P RESENTATION OUTLINE Introduction Review of previous work Research scope Research Plan and System Architecture Experiments and Result Conclusion and Remarks 24

R ESULTS The most important advantage of the new system is that it uses the most up-to-date programming language to program the robotic agents. UBSwarm uses C# programs which utilize the strong C# built-in libraries to interface with vast types of microcontrollers. Unlike the player/stage system software, UBSwarm is not a real-time simulator although it can monitor live data being sent by the multiple robots to the serial ports. Our system explains very clearly how programs are constructed and how function blocks of code are being fetched as the user feeds his/her inputs to the system interface. 25

T HE ROBOTIC AGENTS The platforms as shown in figure 22 is built using Arduino UNO, Arduino Due, and Digilent PIC baords, Figure 12: The robots with showing different configurations 26

H UMAN RESCUE ( PULLING ) Figure 13 shows a human dummy being pulled by four robots. Table 1 below shows the distances achieved by the different number of robots with respect to different weights for the object being transported. Figure 13: Human rescue using four robots Object weight Pulling distance (m) 1 robot2 robots3 robots4 robots5 robots 10 Kilograms Kilograms Kilograms00221 Table 1: successful Pulling distances

W ALL P AINTING The end effecter is the basic 1-Dof gripper attached to a 2-Dof arm that controls the position of the end effecter in two movements; up, down, and 360 degree rotation of the gripper around its own center. Painting with that particular type of end-effecter creates multiple adjacent rectangular coating sectors Figure 14: painting action, wall sectors, and arm movements

P RESENTATION OUTLINE Introduction Review of previous work Research scope Research Plan and System Architecture Experiments and Result Conclusion and Remarks 29

C ONCLUSIONS AND REMARKS As the experimental results show, the sensing and the overall task-specific capabilities of the platforms can be easily upgraded by adding another and not necessarily sophisticated sensors, e.g., laser range finders and that is how the name extendible robots come from. From the software environment prospective, UBSwarm makes it easier to program robotic systems that use the actually available microcontrollers in the market. UBSwarm environment generates programs that cope with changes of the robots configurations. Running experiments has been easier using UBSwarm. Conducting multiple tests will eventually lead to the optimal configuration of the swarm system. 30

T AMER A BUKHALIL Thank you, Question? 31