1. Adaptive CPU Allocation for Software based Router Systems Puneet Zaroo

2. Software based routers Implement packet forwarding/processing in software. E.g a PC with multiple NICs. Provide value added services like encryption, network address translation esp. at the network edge. Issues Software architecture. Per flow threads / per-packet threads Division of input, forwarding and output functions CPU scheduling. How to determine CPU shares How to enforce CPU shares.

3. Objective Leverage the advantages of a component based software router system. Flexibility in designing routers Reusability of software components Dynamic addition of element modules Overlay a QoS provisioning mechanism on top of the component based system. Develop an adaptive QoS system Adaptive to varying input rate and per-packet processing costs.

4. Some Software Router Systems Router Plugins : ETH Zurich, Uwash St. Louis Per flow code modules or plugins. Implemented in the NetBSD kernel. Click Modular router : MIT Routers made of elements composed into a flow graph. ANTS Programmable and customizable networks. Customizable applications acting on packets / packets carrying code as well as data. X-kernel : University of Arizona Object oriented interface to protocols. Can be used on end systems as well as routers. Scout : University if Arizona, Princeton University Communication oriented OS based on x-kernel. Path based abstraction. Advanced CPU scheduling.

5. OS support for CPU scheduling Scout Proportional scheduling. CPU balance (extension of work on livelock) Resource Containers : Rice University Decoupling of protection domain/resource domain. Proper accounting of resources to processes. Resources include threads as well as kernel data structures and memory, bound to containers. E.g a web server serving multiple connections. Processor Capacity reserves : CMU Provides support for both time-sharing and real-time systems. The OS enforces the reservations (cpu share, time period). Applications free to change their reservations subject to admission control. Nemesis : Cambridge OS does low level resource multiplexing. Avoiding QoS cross-talk Support for I/O in user level libraries.

6. Click Composable flow-graphs from router elements Packets travel along graph edges Element based processing (push/pull). Element based scheduling. Multithreaded SMP Click Issues in flow level QoS on top of an element based architecture Flow level accounting and scheduling. CPU balance b/w input, output and processing. CPU conservation of idle elements.

7. CROSS/Linux – Resource reservation with containers Containers Group of related elements Elements doing per flow processing. Container – CPU resource reservation unit. Why use containers and not flows ? Types of Containers Input Output Forwarding Best Effort QoS - Packet rate reservations

8. Example Router Configuration

9. CROSS/Linux - CPU scheduling Three level scheduler Linux schedules CROSS Linux process scheduler CROSS schedules Containers Proportional (Dynamic stride scheduling) Containers schedule Elements Simple Round Robin scheduling

10. CROSS/Linux – Architectural Enhancements to Click CPU conservation through sleep/wakeup Elements tested for scheduling eligibility Containers tested for scheduling eligibilty Notifier Queues - wake up elements (make eligible for scheduling) Delayed wakeup Network interface Input Element Switching between polling and interrupt Based on a threshold packet input rate to reduce programmed I/O overhead Topology discovery Discovering input/output queues for a container

11. CROSS/Linux – Enhancements to Click virtual Interface queues – especially for interface statistics gathering Linux /proc interface – One directory for each container Directory provides information about Container tickets CPU cycles consumed Packet rate/drop rate Elements Input/Output queues Set container tickets

12. CROSS/Linux – Share adaptation Why ? Inability to do a-priori CPU share calculation Variations in packet input rate Variations in per-packet processing cost How ? Scheduler for each container keeps track of Packet input rate. Packet drop rate. CPU cycles used. Recomputes container shares to remove packet drops.

13. CROSS/Linux – Share adaptation Statistics maintained by Queues Packet rates Packet drop rates Queues used to connect containers Packet pass/drop rates at Queues indicate the difference between the required and the actual CPU shares for the container

14. Share adaptation Algorithm Invoked every 1 second Notation used T – Ticket share C – Current CPU share p – Input packet rate d – packet drop rate m – maximum input rate General idea Increase ticket share of a container so that the drop rate is removed at all the containers

15. Input Container share adaptation (Issues) Pass as many packets as possible upto a maximum. How to arrive at this maximum? Forwarding more than the maximum adversely affects the effective router throughput. Reduce share on observing over allocation.

16. Input Container – Share adaptation (Algorithm) if p > m /* Input rate too high */ /* reduce share */ T = C * (m/p) else if d > 0 /* Increase share to */ /*remove packet drops */ drate = min(d + p,m) T = C * (drate/p) else if (T – C) >= delta /* Over allocation */ /* reduce share */ T = T – eps

17. QoS container – Share adaptation (Issues) Always forward till reserved rate. Target a forwarding rate range. Reduce share in case of over allocation

18. QoS container – Share adaptation (Algorithm) If p ε [ R – Dt, R + Dt] /* No change */ return if p > R + Dt /* Reduce share */ T = C * (R/p) else if d > 0 /* Increase share */ drate = min(p + d,R) T = C * (drate/p) else if (T-C) >= delta /* Reduce share */ T = T – eps

19. Output Container – Share adaptation (Issues) Try to forward all that is received Throttling if any has happened upstream Reduce share in case of over allocation

20. Output Container – Share adaptation (Algorithm) if d > 0 /*Increase share */ T = C * (1 + d/p) else if (T – C) >= delta / * Reduce Share */ T = T - eps

21. Best Effort Container – Share adaptation No action taken System makes no guarantees

22. Discussion Packet rate based reservation Reservations based on packet rates more intuitive CPU shares may vary for the same packet rates C (Actual share) - How is it calculated? Input container Only include CPU cycles used in packet processing as opposed to idle polling. Other containers Easy to calculate since no idle polling. m (Maximum forwarding rate) Constant determined at router initialization Evaluated at each iteration

23. Evaluation Using a simulator Calculates the forwarding rate, drop rate based on the CPU shares. Mimics the actions of the adaptive algorithm Eases loading the “router” and testing of diverse workloads Using a real implementation CROSS/Linux on 866 Mhz Pentium III CPU.

24. Adaptive vs. Non Adaptive (Experimental setup) Input (2 µs), Output (2 µs), Best Effort Container (6 µs). Router – 1 MHz CPU => max forwarding = 100,000 packets/s Static ticket assignment = 1:1:1 Input varied for 0 to 110,000 packets/s in increments of 10,000 packet/s every 10s.

25. Adaptive vs. Non Adaptive (Variation with time)

26. Adaptive vs. Non Adaptive (Maximum loss free forwarding rate)

27. Variable packet processing time (Experimental Setup) Input (2µs), Best Effort/QoS (6µs), Output Container (2µs) Observe different convergence behavior for QoS / Best Effort Router – 1 MHz CPU => max forwarding rate initially = 100,000 packets/s Constant input = 50,000 packets/s Per packet processing cost increased by 2 µs every 10 secs. Max. forwarding rate = 50,000 packets/s at t=50s.

28. Variable packet processing time (Adaptive vs. Non Adaptive)

29. Variable packet processing time- (Best Effort vs. QoS)

30. Adaptation in m Hard to determine m at router initialization May vary with variations in per packet processing costs. m = max i (TOTAL_CPU_CPS/cpu_cpp(c i )) where c i ε C TOTAL_CPU_CPS - Total CPU cycles per second available to the router cpu_cpp(c i ) - cycles/packet being used by the flow serviced by container c i cpu_cpp(c i ) = cpu_cpi() + cpu_cycles(c i )/num_packets(c i ) + cpu_cpo() C - The set of containers servicing active flows

31. Fixed vs adaptive m - (Experimental setup) Input (8µs), Best Effort/QoS (1µs), Output Container (1µs) Router – 1 MHz CPU => max forwarding rate, initially = 100,000 packets/s Constant input = 50,000 packets/s Per packet processing cost increased by 2 µs every 5 secs Max forwarding rate = 50,000 packets/s at t=30 s.

32. Fixed vs adaptive m - (Effective Best Effort Forwarding)

33. Fixed vs. adaptive m (Effective QoS forwarding)

34. Fixed vs. Adaptive m ( Best Effort, QoS, Theoretical maximum)

35. Advanced Adaptation in m Previous algorithm gives too much stress to the least expensive flow. Fine if all packets destined for that flow. The packet rate to different flows can be variable. m =(TOTAL_CPU_CPS/weighted_cpu_cpp) weighted_cpu_cpp = Σ (cpu_cpp(c i ) * rate(c i ))/ (Σ rate(c i )) where c i ε C

36. Adaptive m vs. advanced adaptive m (Experimental Setup) Input container (5 µs), Output Container(5 µs) Router (1 MHz CPU) 2 flows QoS container (50,000 p/s,30 µs) => max forwarding rate achievable = 25,000 packets/s Best Effort container (3 µs) => max forwarding rate achievable = 77,000 packets/s Input rate to best effort container = 500 packets/s Input rate to QoS container varied from 15,000 packets/s to 50,000 packets/s in increments of 5,000 packets/s every 5 s.

37. Adaptive m vs. advanced adaptive m (Forwarding rate vs. time)

38. Evaluation on a Router CROSS/Linux software router platform P III 866 MHZ pc. 3 network interface cards.

39. QoS Forwarding (Experimental setup) 866 MHz, PIII router Input Container(4.5 µs), Best Effort Container(3 µs),QoS container (32,000 packets/s), Output Container (4.9 µs) 3 different per – packet processing costs for the QoS container 3, 9.7 and 15.2 µs Input to QoS => 32,000 packets/ Input to Best Effort => 27,000 packets/s

40. QoS Forwarding (Forwarding rate)

41. QoS Forwarding (Ticket Share)

42. QoS forwarding (Ticket Shares) CaseInputOutputBest Effort QoS 3 µs0.290.236 9.7 µs0.270.2820.1530.293 15.2 µs0.2130.2450.0680.47

43. QoS forwarding (CPU Shares) CaseInputOutputBest Effort QoS 3 µs0.510.290.080.10 9.7 µs0.310.2990.0870.30 15.2 µs0.210.240.0660.48

44. Effective Forwarding rate (Experimental setup) Input (4.5 µs), best effort (8.3 µs) and output (4.9 µs) Maximum forwarding rate = 57,000 p/s 3 different scenarios No Adaptation CPU share Adaptation and m = 65000 packets/s CPU share Adaptation and m = 110000 packets/s

45. Effective Forwarding rate

46. Future Work Conjoint CPU – Buffer Allocation Insufficient CPU share => always packet drops Once sufficient CPU shares, more buffering => more efficiency More buffering => higher packet delays and packets getting dropped at line cards. Share adaptation between Linux/CROSS Can use the SFQ scheduler already implemented

47. Conclusion Provide a QoS provisioning layer on top of a component based system. Adaptive in response to variable packet input and processing costs.

48. THANK YOU