1. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 1 Router Software Architecture – Now and Going Forward Michael Beesley, Engineering Director, Routing Technology Group, Cisco Systems mbeesley@cisco.com

2. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 2 Agenda  Overview of Router software components IO Plane Forwarding Plane Control Plane  Constraints and requirements  Software component evolution  Summary

3. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 3 Overview of Router Software Components

4. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 4 IO Plane  Device drivers to manage IO hardware  Handles configuration, status reporting and statistics  Typical embedded application Requires modest CPU and memory resources Modest amount of software, with no major stability or maintenance concerns Fairly low entropy in requirements – IO hardware evolution has slowed (possible exception is ring technology, RPR etc)  Will usually have dedicated CPU resources  Modularity at the per driver/per IO slot level

5. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 5 Forwarding Plane - Control  Embedded software to manage the forwarding/data plane hardware  Builds tables, trees, classification tables/TCAM entries etc required by the forwarding plane to process packets and apply features  Gathers statistics and state from forwarding plane hardware  Can require significant CPU and memory resources  Over time is a continually growing piece of software in size and complexity as hardware and feature set evolves  Will usually have dedicated CPU resources  Scale, performance and real time response are important

6. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 6 Forwarding Plane – Packet Processing  Depends on system and hardware architecture May not exist – all hard coded silicon Can be small amounts of custom microcode Can be larger amounts of software in a higher level language (typically C with asm extenions)  Main job is to process packets and apply all configured features to packet flows  For stateful features where first packet processing is done in the forwarding plane, handles state creation, destruction and update of control plane  Some house keeping with regard to statistics, resource management may be required  As forwarding plane feature set expands, increases in size and complexity  Processing power and memory resources depends greatly on scale and performance of router being built  Hardware assists will be included to offload heaviest work: QoS scheduling, crypto, tree lookups, statistics management, classification etc

7. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 7 Control Plane  Largest and most challenging software component in a router: CLI and other external management functions (SNMP, XML etc) Routing protocols Link layer protocols Gateway to off box services (DHCP, SIP, Radius etc)  Must scale in terms of: Physical and logical (subscriber) interface count Routing protocol peers Route and prefix counts Off box services transactions per second  Very rich feature set with high feature velocity  Some real time(ish) requirements  Can easily be 20M~40M lines of source code

8. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 8 Constraints and Requirements

9. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 9 Constraints and Requirements  Reliable 5 9’s up time (3 minutes a year outage) Must be very reliable and must have good redundancy/upgrade mechanisms  Scalable In both enterprise and service provider, scale is going up in terms of peers, interfaces/subscribers and configured feature sets  Functional Very rich feature set with some features (BFD, APS, Fast Reroute) pushing real timeliness of the system as networks converge onto packet based infrastructure  Long hardware lifetime with reluctance to upgrade  Due to power/cooling/lifetime, highest end CPUs typically not usable  Existing control plane software architectures can make using multi-core CPUs problematic

10. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 10 Software Component Evolution

11. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 11 IO Plane Evolution  Software redundancy to facilitate bug fixing and system software upgrade Dual hw/sw complexes Minimum Disruptive Restart of software/CPU  Scales with router size and IO, as such could offload some simple, repetitive but expensive tasks from control plane Link layer keepalives Media management (OAM, LMI etc)  Per IO module software modularity and separation Allows different versions of the same driver to be running on the same line card, controlling different IO modules

12. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 12 Forwarding Plane Evolution  Redundancy and version translation between control software and packet processing code/microcode Minimum Disruptive Restart of software/CPU Fast restart of packet processing forwarding engine  Ample processing power, as such could offload some tasks from control plane Routing protocol keepalives, BFD, etc  Resource exhaustion management (classification, prefixes, QoS queues etc)

13. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 13 Control Plane Evolution  Good hardware and software redundancy must be built in to the infrastructure to achieve carrier class  Clean separation between OS and application  Simple but expensive tasks must be offloaded: To the IO plane (media/link layer protocol keepalives) To the forwarding plane (routing protocol/BFD keepalives) To dedicated hardware (key generation etc)  Multi core CPU usage (either SMP or master/slave) To scale compute resources of control plane  All algorithms and data structures must be designed for robustness and scale

14. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 14 Control Plane Evolution (continued)  Consider separating some functions and moving them off box: Configuration Provisioning Element management/CLI  Some degree of modularity needed: Coarse grain modularity Fine grain modularity Monolithic application separated from an underlying OS  Scheduler choices: Pre-emptive Co-operative Two level scheduler, Co-operative on top of Pre-emptive  Must consider deprecating features to control code size/quality  Might consider using more advanced programming languages

15. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 15 Summary

16. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 16 Summary  Modern routers are very complex hardware and software systems with demanding requirements and constraints  To achieve carrier class converged networks, router software and hardware architecture is going to have to evolve to better achieve scale and reliability  Some functions may be better implemented off box in a management or provisioning layer  Some efforts to limit feature sets and obsolete older less used features would improve system characteristics and reliability

17. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 17 Questions?

18. © 2006 Cisco Systems, Inc. All rights reserved.Cisco Public mbeesley@cisco.com 18 Thank You