Investigation of antnet routing algorithm by employing multiple ant colonies for packet switched networks to overcome the stagnation problem Firat Tekiner (Phd Student) Z. Ghassemlooy Optical Communications Research Group, The University of Northumbria, Newcastle upon Tyne S. Alkhayatt Faculty of ACES, Sheffield Hallam University LCS 2004

2 Contents  Background Information  Reinforcement Learning  Behaviour of ants in real life  Antnet routing algorithm  Antnet with Multiple Ant Colonies  Antnet with Evaporation  Simulation Environment and Results  Concluding Remarks

3 Routing Problem “In internetworking, the process of moving a packet of data from source to destination.” A routing algorithm is necessary to find the optimal path (or the shortest path) from source to destination. Problems:  Existing algorithms are mostly Table-Based (high cost)  Congestion and contention (requires traffic distribution)  Requires human intelligence  The routing algorithms that are in use are all static algorithms

4 Reinforcement Learning - Routing  Q-Learning  Q-routing (Boyan et al, 94) (Tekiner et al. 1, 04)  Dual reinforcement Q-routing (Kumar et al., 97 & 01)  Ant (software agent) based Routing Algorithms  ABC routing (Schoonderwoerd et al., 96)  Regular and Uniform ant routing (Subramanian et al., 97)  Antnet (Dorigo et al., 98)  Antnet++ (Dorigo et al., 02)  Improved Antnet (Boyan et al., 02)  Modified Antnet (Tekiner et al. 2, 04)  Antnet with evaporation (Tekiner et al. 3, 04)  Agent Distance Vector Routing (ADVR) (Amin et al., 01 & 02)

5 Comparison of Algorithms Antnet uses probabilistic routing tables whereas in well known Link State and Distance Vector algorithms routing table entries are deterministic Antnet uses less resources on the nodes Antnet is dynamic and self organising whereas Distance Vector and Link State algorithms require human supervision Q-Routing does not guarantee on finding the shortest path always. Moreover, they can only find a single path, they cannot explore multiple paths In antnet stagnation is the main problem (routing table freezes due to selecting same paths continously)

6 “unsophisticated and simple” Ants In Nature - “unsophisticated and simple” Builds and protects their nests Sorts brood and food items Explore particular areas for food, and preferentially exploits the richest available food source Cooperates in carrying large items Migrates as colonies Leaves pheromones on their way back Stores information in the nature (uses world as a memory) Make decision in a stochastic way Always finds the shortest paths to their nests or food source Are blind, can not foresee future, and has very limited memory

7 How do they find their way ? Ants – How do they find their way ? Notion of Stimergy: Indirect communication via nature. Ants don’t know where to go initially, and choose paths randomly i.Ants taking the “shorter path” will reach the destinations before the those taking a long route. The path is marked with pheromone. ii.There after the number of ants using the shorter path will keep increasing, since more pheromone is laid on the path.

8 Ants in Antnet Software Agents (Ants) communicates with each other by using: Probabilistic routing tables An array which represents statistical local traffic experienced by every node. Two types of software agents (ants): Forward Ants (collects information) Backward Ants (updates prob. table entries) Two types of queues: Low priority queue (data packets and forward ants) High priority queue (backward ants and forward ants)

9 Antnet Algorithm Overview At regular intervals every node creates a forward ant to randomly selected destinations. Forward ants uses probabilistic routing tables together with queue status at every intermediate node to choose its output port from unvisited list of nodes. Time elapsed and node identifier is pushed to ants stack. If cycle is detected, cycle is deleted from ants memory. When a forward ant reaches to its destination: It transforms itself to a backwad ant, Visits the list of the nodes in its stack in a reverse order, Updates corresponding entries in the routing tables and array on its way back to source by using its values stored on its stack. Ants reinforces the solution by the reinforcement parameter which is calculated by using trip times that it has experienced.

10 Antnet - Example Forward - Ants Stack 0 t0 Node Id Time 01 t0t1 Node Id Time 013 t0t1t2 Node Id Time Backward - Ants Stack Node Id Time Routing Table for 3 Routing Table for 1 Routing Table for 0 Lets assume that: P fd = 14 and P nd = 7 Lets assume that: P fd = 7 and P nd = 7

11 Antnet With Multiple Ant Colonies

12 Antnet with Evaporation Evaporation is a real life scenario where pheromone laid by real ants evaporates in time due to natural circumstances. Link usage statistics are used to evaporate (e(x)) the phernome laid by the ants. It is the proportion of number of forward ants destined to the node x over the total ants received by the current node in the given time window. Frequency of evaporation is defined by the programmer. Amount of probability to be evaporated is subtracted from the associated link. Then this amount is equally distributed over the other links.

13 Simulation Parameters Parallel Virtual Machine (PVM) together with C Language is used to simulate the algorithm Every network node is assigned to a different process Poisson traffic distribution 2500 packets created per node Average of 15 simulation runs is used for accuracy No packet loss due to node/link failures Ant creation rate is set to 5sec per node and Evaporation rate is set to 0.5 sec per link. 29 Node Random Network is used.

14 Results Table Multi Antnet Evaporation Improved Antnet Average Delay (sec.) Throughput (kbits/sec) Agent Overhead (%) Antnet with multiple ant colonies performed the best in terms of throughput. Antnet with evaporation performed best in terms of average packet delay.

15 Results - Throughput Throughput vs. Simulation Time

16 Concluding Remarks Stagnation Stagnation is a major problem but solution exists. Multiple Ant Colonies applied to the antnet routing algorithm for the packet switched networks. Multiple Ant Colonies has increased the throughput of the network whereas there is no improvement observed on the average delay experienced per packet. No interaction among the different colonies has been considered. This can be taken into account in the future. By evaporating the links probabilities in a predefined rate, average delay experienced per packet is reduced by 7%. No mathematical formula only constant variables are used in antnet. optimise There is a need for a second meta-heruistic to optimise antnet’s parameters

17 Any Questions? Thanks!