Architecture for the 21st Century Network Avici Company Confidential Avici TSR – An overview “True scalable routing” Ides Vanneuville Systems Engineer - EMEA

Best generation networking Agenda Introduction Next (best) generation networking Software features Hardware features Summary

Best generation networking Who is Avici ? Founded: 1996 Billerica, MA Mission: To market scalable core routers that delivers service providers reduced capital and operational expenditures and simplified operations without service disruptions Public Company (Nasdaq AVCI), $ 240 million raised employees worldwide, 11 in Europe Gained market share (in last 12 months) 2% in core router market (+1G links) 6.5% in core router market (10G links)

Best generation networking Explosive Demand for Data Services Peak-Hour Internet Bandwidth Demand Source: Pioneer Consulting LLC 120% CAGR (terabits per second)

Best generation networking Access Routers Backbone Routers Connected Router Infrastructure Business Problems Floorspace Management Power Equipment Costs Upgrade Granularity Technical Problems Blocking POP Limited redundancy Complex redundancy Configuration complexity Routing complexity Routing Protocol Scaling Management System Scaling

Best generation networking Access Routers Access Interfaces Avici’s Solution Blocking Limited hardware resilience Excessive software resilience Complex Failure Modes Coarse scalability Excessive network topology Non-blocking Rich hardware resilience Appropriate software resilience Simplified failure modes Granular scalability Efficient network topology Fabric

Best generation networking Customer Premise Equipment (CE) Provider Edge (PE) Next generation network Provider IP Core (P) Optical Core Avici

Best generation networking Why next generation network? Increase revenue deploying new services High-speed Internet access IP transit services – IP bandwidth whole sale Virtual Private Networks (L2 & L3) Voice over IP - Multimedia Application Service Provider connectivity Decrease cost converging networks Optical SDH ATM services IP services Voice Frame relay services Leased line services ATM services IP services

Best generation networking Building Blocks for CN The converged network must encompass the service levels of predecessor or legacy networks Composite links, QoS and MPLS is the enabling technology! Legacy NetworkCN EquivalentPE RoleCore Role TDM & Circuit EMMPLS “Virtual Circuit” LDP with QoS Markings LDP over “QoS Aware TE Tunnels” ATM ServicesMPLS Cell Relay or Cross connect VCI/VPI mapped to LSPs LDP or RSVP-TE forwarding with QoS Frame Relay Services MPLS Layer 2 or Layer 3 VPNs DLCI mapped to LSPs LDP or RSVP-TE forwarding with QoS Broadband Multimedia Distribution IP Multicast and IP Differentiated Services Video CODEC and Media Gateway Multicast Routing with PIM MSDP and MBGP Carrier’s Carrier Bandwidth Services MPLS Layer 2 (IGP Tunneling) or Layer 3 VPNs Sub-I/f based on MPLS label LDP or RSVP-TE forwarding with QoS

Best generation networking Virtual Private networks Layer 3 VPNs For IP based traffic Point-to-point Point-to-multipoint Can be combined with other services. E.g. Internet access. Layer 2 VPNs Transparent LAN services Transport over IP network Virtual Leased Line (VLL) IPSec Encrypted data Transport over IP network VPN requirements VPN edge equipment Scalable IP core Grow without pain Service differentiation (QoS) MPLS & TE VPN provisioning tool Source: Telechoice 1999

Best generation networking More services… Voice over IP (VoIP) Controlled bandwidth Low latency Controlled jitter Network resilience IP transit – IP wholesale Scalable capacity Scalable connectivity VPN services Network resilience Multimedia Controlled bandwidth Low latency Controlled jitter Robust multicast environment Scalability IP core SP1 SP2

Best generation networking Role Differentiation Is Key Requirements for Converged NetworksPECore Subscriber Aggregation over copper ( DSL,T-1,E-1,T-3,E-3 )  Deep Packet Classification (beyond layer 4)  Support for termination of thousands of subscribers  Transport for millions for subscribers  Adaptation of IPv4, FR, ATM, TDM, VoIP into MPLS  LDP/MPLS support for CN transport services  RSVP-TE/MPLS support for TE Core with BW reservation  Class based traffic policing to enforce SLAs  Class Based queuing for aggregating subscriber traffic into SLA managed services  Class based traffic conditioning to meet low jitter SLAs  Accounting statistics for billing applications 

Best generation networking Carrier design principles Architected to suit to Carrier design principles Design network Establish and stabilize topology Acceptance tests Transition to operations Add customers, expand capacity cycle

Architecture for the 21st Century Network Avici Company Confidential Software features

Best generation networking About IPriori Unicast Routing Fully featured BGP-4 Implementation –Confederates, Route Reflectors, damping, policy, Route Redistribution, Aggregation, dynamic configuration, peer security IGP Support IS-IS –Level 1 and 2 support, Route Redistribution, Database overflow support, passive interface support OSPF –Stub Area support, Virtual links, Router Summarization, Passive Interface support, Route Redistribution Multicast Routing PIM Sparse Intermediate Point PIM Dense/Sparse, Auto-RP, MSDP, MBGP for mcast NLRI

Best generation networking server-id 2 upper hostname bay_TWO_UPPER_server system-password 7 S9bQQdb9Sd interface Loopback 0 ip address no shutdown interface Ethernet 0 ip address no shutdown module 1/15 4xoc3c no shutdown interface pos 1/15/1 ip address crc 16 no keepalive sonet scramble-atm no shutdown module 1/17 1xoc48c no shutdown interface pos 1/17/1 clock source internal ip address no keepalive sonet scramble-atm no shutdown router ospf 100 network area ip route null0 ip community-list 1 deny ip community-list 1 deny ip community-list 1 deny ip as-path access-list 1 permit ^$ ip as-path access-list 100 deny _1_ ip as-path access-list 101 permit _65010_ ip as-path access-list 286 permit _286_ ip as-path access-list 286 permit _3561_ ip as-path access-list 286 permit _701_ route-map EBGPMAP permit 10 set local-preference 25 set metric 150 route-map IBGPMAP deny 10 match as-path route-map IBGPMAP permit 20 set community set local-preference 80 set metric 10 route-map connected-bgp permit 10 set community set origin igp route-map static-bgp permit 10 set community set origin igp bay 1/1 logging console platform all system bgp end router bgp 4293 bgp router-id bgp dampening route-map 1 redistribute connected route-map connected-bgp redistribute static route-map static-bgp neighbor ebgp peer-group neighbor ebgp send-community neighbor remote-as 200 neighbor peer-group ebgp neighbor timers 6 neighbor ibgp peer-group neighbor ibgp remote-as 4293 neighbor ibgp send-community neighbor ibgp route-map IBGPMAP out neighbor remote-as 4293 neighbor next-hop-self neighbor route-reflector-client neighbor timers 180 neighbor remote-as 4293 neighbor next-hop-self neighbor route-reflector-client neighbor send-community neighbor timers 180 neighbor route-map IBGPMAP out neighbor remote-as 100 neighbor timers 180 Easy Migration - Industry CLI

Best generation networking Management and Security SNMP Standard and vendor-specific MIBS RMON Events and Alarms Group 64bit counters System event log for all events Protocol activity (tracing) Local and remote SYSLOG Log all CLI commands Standby Route Servers Dual Servers can be deployed for additional redundancy Traffic separation All ForMe traffic is classified into 24 application-specific queues serviced by WRR to avoid starvation of control traffic and ensure against DoS attack Configurable for ForMe Traffic Filters Security TACACS+ Secure Shell

Best generation networking Network and Traffic Engineering Leverage emerging D-WDM Technology Get ahead of the serial bandwidth curve Stabilize topology Enable Rapid Provisioning Simplify the IP routing mesh Provide Protection options Provide TE network for the transport of Diff-serv traffic Enable Circuit Emulation/Private Line Service Enable 45ms restoration

Best generation networking OC-48c 10G vOC192 ! interface pos 2/29/1 no ip address no keepalive sonet scramble-atm no shutdown ! interface composite-link TSR1_CL ip address peer default ip address member-link POS 1/22/1 member-link POS 1/37/1 member-link POS 1/39/1 member-link POS 2/29/1 no shutdown Composite Links Combines up to 64 physical connections to a single logical connection Removes serial bandwidth limitations Granular bandwidth scaling 4-1 speed mismatch Preserves packet sequence Add/remove trunks dynamically Recovery in 45ms Compartmentalizes link failure Routing table stability

Best generation networking Composite Link 3 λ Composite Link 4 λ Optical Switches Label Switch Router Composite Link 4 λ Composite Link 3 λ Optical Switches Label Switch Router TSR monitors load on composite links If total premium traffic exceeds a limit, shifts ports from other links TSR signals optical switch to reassign wavelength to high traffic link Dynamic reallocation of total network Composite Links & intelligent optics Demonstrated interoperability

Best generation networking SuperCOM 2001 demo

Best generation networking MPLS Traffic Engineering Ingress, Midpoint, and Egress Penultimate Hop Popping is NOT mandatory Ingress behavior allows unparalleled tunnel scaling Constraint Based Routing with Avici enhancements Enhanced RSVP signaling for Label Distribution Enhanced OSPF and IS-IS to support TE LDP End to end TE instrumentation for network mngt Tight Integration with Composite Links Hardware label switching on OC-3, OC-12, OC- 48, OC-192 and Gigabit Ethernet

Best generation networking Traffic Engineering E B K C Short Cut Tunnel 1 A Shortest Path to ISPs 1, 2, 3, and 4 INGRESS LSR Subscriber ISP 4 M Router G's Next Hop For Routes in ISP 4 EGRESS LSR EGRESS LSR

Best generation networking Optimized Path Layout Intelligent MPLS Reflood Timing Improve responsiveness of Ingress path layout Resilience Optimization Recover from MPLS failures with minimal impact to core topology Adaptivity Optimization Take full advantage of positive changes in the core topology Rebuild LSPs, ordered by required bandwidth 1 99% loaded, 3 sec 3 90% loaded, 20sec 4 50% loaded, 180sec 5 0% loaded, 600sec 5 sec 15 sec 100 sec 200 sec Percentage of link loading T = % loaded, 10sec D C B A TE Reflood Timing

Best generation networking Intelligent Reflood Timing IGP Flooding for TE If midpoint cannot honor bandwidth reservation at the requested priority level the midpoint will immediately queue a BW TLV to IGP This keeps the ingress up-to-date Otherwise MPLS reflood timer is provided Immediate Pacing Piecewise Linear Model –99% 3s 95%10s 90% 20s 50% 180s 0%600s –Insures that higher bandwidth demands are flooded more frequently

Best generation networking Resilience Optimization Controls response to failures in LSP path Triggered by link loss, LSR loss, preemption Three methods to enact repairs Immediate Response Pacing Sorted-Pacing Sequence LSP creation based on their demand for bandwidth at an associated priority level Piecewise Linear Model –Insures that LSPs with higher bandwidth demands are signaled first

Best generation networking Adaptivity Optimization Controls re-optimization of LSP layout when the topology changes Responsible for two activities Timing of new CR-SPF Timing and sequencing of LSP re-layout New SPFs might be necessitated based on IGP and IGP-TE flooding Adaptivity Methods Disabled Pinned Periodic Piecewise Linear –Applies to both SPF and Reroute

Best generation networking Head-end Reroute IGP or RSVP signaling indicates that a reservation along the primary path can no longer be sustained: Administrative Change to LSP entity Congestion along a given LSP Failed Link Failed Node Headend LSR constructs a new constrained SPF to Egress Utilizes a “Make-Before-Break” algorithm for new path selection Creates a new primary LSP Restoration time is a function of path complexity! Restoration is restricted to the entire path

Best generation networking Integration with Composite Trunks Subnet 1 Subnet 2 Subnet 1 Subnet 2 Compose Trunk Higher Pre-emption value affects tunnel reroute Re-route time affected by reservable bandwidth attribute

Best generation networking MPLS Fast-Reroute Protection Router switches traffic to pre-configured backup Allows identification of which traffic to protect Traffic switched to backup and traffic normally on link contend for resources using QoS – no traffic is discarded if link is not over-utilized Works for POS and gigabit Ethernet

Best generation networking MPLS Fast-Reroute Drafts draft-atlas-rsvp-local-protect-interop-01.txt (Avici) draft-swallow-rsvp-bypass-label-01.txt (Cisco) draft-gan-fast-reroute-00.txt (Juniper)

Best generation networking Future tunnel resilience options Composite Path Leverages same mechanism as composite trunks Establish multiple tunnels for same prefixes Unequal tunnel size support Fast headend re-route Fast Reroute Midpoint repair mechanism Failure affinity groups Backup bandwidth pre-reserved Midpoint reroute in 45ms

Best generation networking VPN Support: RFC 2547 VPN-P Support Support for Multi-Protocol Extensions to BGP-4 to support route-distinguisher Compatible with Cisco, Juniper, Unisphere, Riverstone, Alcatel, etc. Traffic Engineering/VPN tunnel integration Provide a TE core for Virtual Private Networks Avici TSR PE Node Optical Switch Avici TSR Optical Switch Avici TSR Optical Switch Peering Node Customer

Best generation networking QoS Functional Summary Combination of WFQ, WRR, WRED and GPRA (CBR) Programmable ASIC Pipeline Multiple hierarchy of Algorithms Many iterations per algorithm implemented Minimum packet data handling Functionality increased for Multiservice module Counters maintained per interface,VC,class traffic management and accounting information

Best generation networking REG CTL PR BE Group Policer 8 Class Policer Classifier Fabric Queues Direct Connect Fabric 8 Class RED 512 Class Shaper Group Shaper QoS Treatment per node

Best generation networking MPLS TE/Composite Trunks/QoS Subnet 1 Subnet 2 Subnet 1 Subnet 2 Compose Trunk QoS operates on Diff-serv inferred MPLS exp marking WRED discards tunnel two or hop-by-hop traffic during tunnel reroute

Best generation networking Tunnel establishment Filtering on next-hop and DSCP Combined QoS packet treatment Aggregate or per-class tunnels Dynamic Tunnel mapping to QoS traffic classes Classes configured on a per node basis Signaling interaction Label inferred QoS packet treatment assignment Configurable packet treatment protected forwarding table Class-based traffic tunnels

Best generation networking Traffic Engineering with QoS Enables no-loss, low delay variance, low latency services Virtual Leased Line Voice over IP Storage Area Networks Add new properties to Service Level Agreements Improve reliability at reduced cost Deliver protection appropriate for each service, up-to SONET APS levels Improve utilization in network backbone

Architecture for the 21st Century Network Avici Company Confidential Hardware features

Best generation networking The Avici Systems TSR Carrier class features Carrier class design Unprecedented scaling Future proof

Best generation networking Patent-pending Velociti™ Fabric Toroidal direct connect fabric Scales to 560 active modules Each element adds switching & forwarding capacity Generation Gbps Generation Gbps Each module connects to 6 other modules Path diversity through fabric Resilient Non-blocking

Best generation networking Velociti™ - Non-uniform 3-ary n-cube Alternate Minimum Paths Axis Wrapped Generation 1 Generation 2 Generation 3

Best generation networking Y X Z Physical Axis 2. Virtual Channel Overlay Path Diversity Link State Fabric Routing Flow invariant path selection byte fabric unit Wormhole forwarding Fabric Forwarding 6x10G Axis - Generation 1 6x40G Axis - Generation 2

Best generation networking Backplane Mesh Topology Shelf 1 Shelf 2 Shelf 3 Shelf 4 Slots 1-5Slots 6-10 Y X Z

Best generation networking Growing the TSR System Expanding the interconnect Two physical backplanes in each bay Connected top-to-bottom Connected side-to-side Connected over-the-top Power Distribution -48VDC 60A per shelf redundant feed Environment Controllers Two per bay interconnected between bays

Best generation networking Avici Product Scalability 6 TSR Bay 2.4 Tbps 4 TSR Bay 1600 Gbps 2 SSR 400 Gbps SSR/TSR 600 Gbps 1 SSR 200 Gbps 1 TSR 400 Gbps 2 TSR 800 Gbps

Best generation networking TSR™ Interface Modules Packet over SONET/SDH Concurrent IP and MPLS 1 port OC-192c/STM-64 1 port OC48c/STM-16 2 port OC48c/STM-16 4 port OC12c/STM-4 4 port OC3c/STM-1 16 port OC3c/STM-1 Ethernet 2 port GbE 8 port GbE Q2/02 10GbE 2H/02

Best generation networking O/E Conversion SONET Termination Packet QoS Multicast Expander & Packet Scheduler Forwarding Engine Control Processor FE Interface PPP/MPLS Processing E/O Conversion SONET Termination PPP/MPLS Processing Switch Fabric BACKPLANE O/E Conversion Packet Memory 64 MB Module I/O Architecture

Best generation networking PCMCIA PowerPC Processor Console Fast Ethernet HDD Server Memory 320Mb OSPFBGP4IS-IS MBGPPIMMSDP RSVPCLI SNMP FTP Control Processor FE Interface SONET Termination Packet QoS Multicast Expander & Packet Scheduler Forwarding Engine Control Processor FE Interface PPP/MPLS Processing E/O Conversion SONET Termination PPP/MPLS Processing Switch Fabric O/E Conversion SONET Termination Packet QoS Multicast Expander & Packet Scheduler Forwarding Engine Control Processor FE Interface PPP/MPLS Processing E/O Conversion SONET Termination PPP/MPLS Processing Switch Fabric O/E Conversion SONET Termination Packet QoS Multicast Expander & Packet Scheduler Forwarding Engine Control Processor FE Interface PPP/MPLS Processing E/O Conversion SONET Termination PPP/MPLS Processing Switch Fabric O/E Conversion Bay & Environment Controller Routing and Configuration tasks Forwarding and Fabric Routing tasks PCMCIA PowerPC Processor Console Fast Ethernet SSD Server Memory 580Mb OSPFBGP4IS-IS MBGPPIMMSDP RSVPCLI SNMP FTP Control Processor FE Interface SONET Termination Packet QoS Multicast Expander & Packet Scheduler Forwarding Engine Control Processor Server Interconnect PPP/MPLS Processing E/O Conversion SONET Termination PPP/MPLS Processing Switch Fabric O/E Conversion Packet Memory 64 Mb Module Control Software Software & System Architecture

Best generation networking System Assurance Testing Regression Fully Automated conformance testing Over 2000 scripts for link layer/protocol/function testing Stress and large topology Highly accelerated software live cycle testing Tests combination forwarding, IGP/EGP topology, features and operator interaction Topology Testing Fully connected TSRs Automated customer topology testing Custom inhouse testing tools Topology Simulation Packet Forwarding Industry leading QoS test tools Strategic Co-development relationship with Netcom/Spirent ADTECH AX4000

Best generation networking Field proven operation First systems deployed in NTON (DARPA Supercomputer research network) Operational in the Internet in May 1999 (BGP) Currently deployed in one of the worlds largest carrier IP networks Support OC-192 links Operating as BGP Route Reflector in one of the worlds largest RR configurations

Best generation networking AT&T network

Architecture for the 21st Century Network Avici Company Confidential Summary

Best generation networking Summary - Carrier imperatives Cost Lower the cost of building and operating the network Velocity Decrease the time to add or change services Profit Deliver value added services and quality of service enhancements

Best generation networking Lower Operational Costs Backbone Interfaces Customer Facing Interfaces Low Speed (electrical) High Speed (Optical) Backbone Interfaces Customer Facing Interfaces Low Speed (electrical) IP VPN / Low Speed Transit High Speed (Optical) IP Transit VS. Current best practice Avici alternative

Best generation networking Provisioning - Months to Minutes Backbone capacity increased without truck-rolls Market share not impacted by provisioning Immediate provisioning for all bandwidth products up-to physical network capacity 10λ 6λ 14λ Immediate fulfillment of customer demands

Best generation networking Velocity – Composite Links Avoid hitting the transmission capacity wall Backbone links with 64 times the capacity of our competitor on the same transmission network No impact to customer or network stability Backbone Capacity added in days not months

Best generation networking Profit Use IP core for converging all services Scale services past competitors offerings Spend your time creating new services… Make €€€€ at the edge of the network Save €€€€ at the core of the network

Best generation networking Conclusion Next generation public network switch Scalability Non-stop system and software architecture Quality of Service Network and Traffic Engineering Benefits Reduce capital costs Stable service velocity Increased equipment life cycle Faster, stable network expansion & dramatic reduction in cost per bit delivered

Architecture for the 21st Century Network Avici Company Confidential Thank You