Page 1 John Morgan Infrastructure Processor Division September 2004 Intel® IXP2XXX Network Processor Architecture Overview

Agenda  IXP2400 External Features  IXP2800 External Features  Comparison of IXP2400 and IXP2800  IXP2XXX Resource Overviews –MEv2 Overview –QDR SRAM Overview –DDR Overview –RDRAM Overview –PCI Overview –MSF Overview –Miscellaneous

Customer ASICs IXP2400 External Features Utopia 1/2/3 or POS-PL2/3 Interface PCI 64-bit / 66 MHz IXP2400 (Ingress) Host CPU ( Optional ) ATM / POS PHY or Ethernet MAC Flash Classification Accelerator CoProc Bus Micro- Engine Clusters Slow Port Switch Fabric Port Interface Utopia 1,2,3 SPI – 3 (POS-PL3) CSIX IXP2400 (Egress) Flow Control Bus External Interfaces  MSF Interface supports UTOPIA 1/2/3, SPI-3 (POS-PL3), and CSIX.  Four independent, configurable, 8-bit channels with the ability to aggregate channels for wider interfaces.  Media interface can support channelized media on RX and 32-bit connect to Switch Fabric over SPI-3 on TX (and vice versa) to support Switch Fabric option.  2 Quad Data Rate SRAM channels.  A QDR SRAM channel can interface to Co-Processors.  1 DDR SDRAM channel.  PCI 64/66 Host CPU interface.  Flash and PHY Mgmt interface.  Dedicated inter-IXP channel to communicate fabric flow control information from egress to ingress for dual chip solution. DDR DRAM 2 GByte QDR SRAM 1.6 GBs 64 M Byte IXA SW

SDRAM IXP2400 Full-Duplex OC-48 System Implementation IXF6048 Framer IXP2400 Ingress Processor IXP2400 Egress Processor Switch Fabric Gasket SDRAM QDRQDRQDRQDR Q QQDRDRQQDRDR DDR SDRAM Packet Memory QDR SRAM Queues & Tables DDR SDRAM Packet Memory QDR SRAM Queues & Tables 1x OC-48 or 4x OC-12 OC-48OC48 QDRQDRQDRQDR QDRQDRQDRQDR TCAM Classification Accelerator TCAM Host CPU (IOP or iA) SAR’ing Classification Metering Policing Initial Congestion Management Ingress Processor Traffic Shaping Flexible Choices diff serve TM 4.0 … Egress Processor

IXP2400 Chaining PCI 64/66 2.5Gbs CSIX-L1 IXP2400 Processor DDR Packet Memory IXP2400 Processor QDR SRAM Queues & Tables DRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDR DRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDR DDR Packet Memory 2.5 Gbs CSIX-L1 IXP2400 Processor QDR SRAM Queues & Tables DRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDR DDR Packet Memory Glueless Interface between IXP2400 Devices using CSIX-L1 Control Plane Processor 2.5Gbs CSIX-L1 2.5Gbs SPI3

MEv2 6 MEv2 7 MEv2 5 MEv2 8 Intel® XScale™ Core 32K IC 32K DC Rbuf 128B Tbuf 128B Hash 64/48/128 Scratch 16KB QDR SRAM 1 QDR SRAM 2 DDRAM GASKETGASKET PCI (64b) 66 MHz 32b 32b b S P I 3 or C S I X E/D Q MEv2 2 MEv2 3 MEv2 1 MEv2 4 CSRs -Fast_wr-UART -Timers-GPIO -BootROM/Slow Port IXP2400

IXP2400 Bandwidths  600 MHz Operation 4.8+ GOPs  2.5 Gb/s Full Duplex Media Interface –POS-PHY –Utopia –CSIX-L1  2.4 GBs DDR Memory Bandwidth at 300 MTs  1.6 GBs QDR Memory Bandwidth with 200 MHz QDRII devices

IXP2400 Resources Summary  Half Duplex OC-48 / 2.5 Gb/sec Network Processor  (8) Multi-Threaded Microengines  Intel® XScale™ Core  Media / Switch Fabric Interface  PCI interface  2 QDR SRAM interface controllers  1 DDR SDRAM interface controller  8 bit asynchronous port –Flash and CPU bus  Additional integrated feature –Hardware Hash Unit –16 KByte Scratchpad Memory,Serial UART port –8 general purpose I/O pins –Four 32-bit timers –JTAG Support

Agenda  IXP2400 External Features  IXP2800 External Features  Comparison of IXP2400 and IXP2800  IXP2XXX Resource Overviews –MEv2 Overview –QDR SRAM Overview –DDR Overview –RDRAM Overview –PCI Overview –MSF Overview –Miscellaneous

IXP2800 External Features Customer ASICs SPI-4 or CSIX- L1 PCI 64-bit / 66 MHz IXP2800 (Ingress) Host CPU ( Optional ) ATM / POS PHY or Ethernet MAC Flash Classification Accelerator CoProc Bus Micro- Engine Clusters Slow Port Switch Fabric Port Interface SPI – 4, CSIX-L1 IXP2800 (Egress) Flow Control Bus External Interfaces  Media Interface supports both SPI-4 and CSIX  4 Quad Data Rate (QDR) SRAM channels  Each channel can interface to Co- processors  3 RDRAM Channels  PCI 64/66 Host CPU interface  Flash and PHY Management interface  Dedicated inter-IXP channel to communicate fabric flow control information from egress to ingress for dual chip solution RDR DRAM 50+Gbps 2 Gbyte total for 3 channels QDR SRAM 12.8 Gbps x 4 64 M Byte x 4 channels IXA SW

10Gb/s SONET Line Card 10GbE OC-192c SPI I/F Fabric CSIX I/F RDR Packet Memory IXF Gbs 15Gbs 10Gbs Control Plane Processor PCI 64/66 Fabric Interface Chip (FIC) SAR’ing Classification Metering Policing Initial Congestion Management Ingress Processor Traffic Shaping Flexible Choices diff serve TM 4.1 … Egress Processor IXP2800 Egress Processor DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR QDR SRAM Queues & Tables Q QQDRDRQQDRDR IXP2800 Ingress Processor DRAMDRAMDRAM QDRQDRQ QQDRDRQQDRDR RDR Packet Memory QDR SRAM Queues & Tables Q QQDRDRQQDRDR 10 GbE WAN / PPP/ ATM/ OTN / SONET/ SDH CDR, DEMUX CDR, DEMUX Flow Ctl

IXP2800 System with SPI gasket Utopia3 SPI gasket 10Gbs PCI 64/66 Dual CSIX 10Gbs IXP2800 Ingress Processor DRAMDRAMDRAM QDRQDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR RDR Packet Memory QDR SRAM Queues & Tables Control Plane Processor x x SPI4 2 U3 SPI 2 U3 SPI4 2 U3

IXP2800 Chaining PCI 64/66 10Gbs SPI-4 IXP2800 Processor RDR Packet Memory IXP2800 Processor QDR SRAM Queues & Tables DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR RDR Packet Memory 10Gbs SPI-4 IXP2800 Processor QDR SRAM Queues & Tables DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR RDR Packet Memory Glueless interface between IXP2800 devices using SPI-4.2Glueless interface between IXP2800 devices using SPI-4.2 Control Plane Processor 10Gbs SPI-4 10Gbs SPI-4

Page 14 Intel® XScale™ Core 32K IC 32K DC MEv2 10 MEv2 11 MEv2 12 MEv2 15 MEv2 14 MEv2 13 Rbuf 128B Tbuf 128B Hash 48/64/128 Scratch 16KB QDR SRAM 2 QDR SRAM 1 RDRAM 1 RDRAM 3 RDRAM 2 GASKETGASKET PCI (64b) 66 MHz IXP b 16b b S P I 4 or C S I X Stripe E/D Q QDR SRAM 3 E/D Q 1818 MEv2 9 MEv2 16 MEv2 2 MEv2 3 MEv2 4 MEv2 7 MEv2 6 MEv2 5 MEv2 1 MEv2 8 CSRs -Fast_wr-UART -Timers-GPIO -BootROM/SlowPort QDR SRAM 4 E/D Q 1818

IXP2800 Bandwidths  1.4 GHz Operation 20+ GOPs  10Gbs Full Duplex Media Interface –SPI-4.2 –CSIX-L1  1.9 GB/s QDR SRAM Memory Bandwidth/Channel  2.1 GB/s RDRAM Memory Bandwidth/Channel

IXP2800 Resources Summary  Half Duplex OC-192 / 10 Gb/sec Network Processor  (16) Multi-Threaded Microengines  Intel® XScale™ Core  Media / Switch Fabric Interface  PCI interface  4 QDR SRAM Interface Controllers  3 Rambus* DRAM Interface Controllers  8 bit asynchronous port –Flash and CPU bus  Additional integrated features –Hardware Hash Unit for generating of 48-, 64-, or 128-bit adaptive polynomial hash keys –16 KByte Scratchpad Memory –Serial UART port for debug –8 general purpose I/O pins –Four 32-bit timers –JTAG Support

Agenda  IXP2400 External Features  IXP2800 External Features  Comparison of IXP2400 and IXP2800  IXP2XXX Resource Overviews –MEv2 Overview –QDR SRAM Overview –DDR Overview –RDRAM Overview –PCI Overview –MSF Overview –Miscellaneous

IXP2800 and IXP2400 Comparison Dual chip full duplex OC48 Dual chip full duplex OC192 Performance 8 (MEv2) 16 (MEv2) Number of MicroEngines Separate 32 bit Tx & Rx configurable to SPI-3, UTOPIA 3 or CSIX_L1 Separate 16 bit Tx & Rx configurable to SPI-4 P2 or CSIX_L1 Media Interface 2 channels QDR (or co- processor) 4 channels QDR (or co- processor) SRAM Memory 1 channel DDR DRAM - 150MHz; Up to 2GB 3 channels RDRAM 800/1066MHz; Up to 2GB DRAM Memory 600/400MHz 1.4/1.0 GHz/ 650 MHz Frequency IXP2400IXP2800

Agenda  IXP2400 External Features  IXP2800 External Features  Comparison of IXP2400 and IXP2800  IXP2XXX Resource Overviews –MEv2 Overview –QDR SRAM Overview –DDR Overview –RDRAM Overview –PCI Overview –MSF Overview –Miscellaneous

128 GPR Control Store 4K/8K Instructions 128 GPR Local Memory 640 words 128 Next Neighbor 128 S Xfer Out 128 D Xfer Out Other Local CSRs CRC Unit 128 S Xfer In 128 D Xfer In LM Addr 1 LM Addr 0 D-Push Bus S-Push Bus D-Pull BusS-Pull Bus To Next Neighbor From Next Neighbor A_Operand B_Operand ALU_Out P-Random # 32-bit Execution Data Path Multiply Find first bit Add, shift, logical 2 per CTX CRC remain Lock 0-15 Status and LRU Logic (6-bit) TAGs 0-15 Status Entry# CAM Timers Timestamp Prev B B_op Prev A A_op MicroEngine v2

 Clock Rates –IXP2400 – 600/400 MHz –IXP /1.0 GHz/ 650 MHz  Control Store –IXP2400 – 4K Instruction store –IXP2800 – 8K Instruction store  Configurable to 4 or 8 threads –Each thread has its own program counter, registers, signal and wakeup events –Generalized Thread Signaling (15 signals per thread)  Local Storage Options –256 GPRs –256 Transfer Registers –128 Next Neighbor Registers – bit words of local memory Microengine v2 Features – Part 1

 CAM (Content Addressable Memory) –Performs parallel lookup on bit entries –Reports a 9-bit lookup result –4 State bits (software controlled, no impact to hardware) –Hit – entry number that hit; Miss – LRU entry –4-bit index of Cam entry (Hit) or LRU (Miss) –Improves usage of multiple threads on same data  CRC hardware –IXP Provides CRC_16, CRC_32 –IXP Provides CRC_16, CRC_32, iSCSI, CRC_10 and CRC_5 –Accelerates CRC computation for ATM AAL/SAR, ATM OAM and Storage applications  Multiply hardware –Supports 8x24, 16x16 and 32x32 –Accelerates metering in QoS algorithms –DiffServ, MPLS  Pseudo Random Number generation –Accelerates RED, WRED algorithms  64-bit Time-stamp and 16-bit Profile count Microengine v2 Features – Part 2

Intel® XScale™ Core Overview  High-performance, Low-power, 32-bit Embedded RISC processor  Clock rate –IXP MHz –IXP /500/325 MHz  32 Kbyte instruction cache  32 Kbyte data cache  2 Kbyte mini-data cache  Write buffer  Memory management unit

QDR SRAM Overview  Controller Configuration –IXP channels –IXP channels –Optional parity (support for x16 or x18 parts)  Address up to 64 Mbytes of SRAM per channel  Pin design supports up to 4 SRAM loads  Supports Burst of 2 QDR Devices  Supports byte parity bits [8], [17] for byte 0/1  Parity can be enabled/disabled per channel in SRAM_control CSR

QDR SRAM Overview  Peak bandwidth of 1.6 GBytes/sec per channel –Using 200 MHz SRAMs  Specialized SRAM operations: –Atomic swap, bit set, bit clear, add, subtract –Hardware support for ring, queue and journal operations –64 Q_Array registers per channel  Interface to QDR compatible TCAMs and CoProcessors –Network Processor Forum LA-1 Co-Processor Standard Compliant –“Clamshell” topology enables both Memory and Co- processor to share the same channel

IXP2400 DDR DRAM Overview  1 64-bit (72-bit with ECC) SDRAM channel  DRAM sizes of 64Mb, 128Mb, 256Mb, 512Mb, or 1Gb –Max capacity is 2GB (using 1Gb parts) –Support x8 or x16 devices, DIMM or direct soldered –Support devices with 4 banks –Support 1 or 2 sided DIMM –Optional ECC  200/300 MTS, 100MHz/150MHz respectively  Hardware Interleaving spreads contiguous addresses across multiple banks

IXP2800 RDRAM Overview  3 Independent Rambus* DRAM Channels which operate concurrently  1.6 GBytes/s (12.8Gbps) per channel at 800 MHz  Maximum total of 2 GBytes –768 MBytes each if 3 channels are populated –1 GBytes each if only 2 channels are populated –2 GBytes if only 1 channel is populated  Supports 64Mb, 128Mb, 256Mb, 512Mb and 1 Gb devices  Supports RDRAMS with 1x16, 2x16 dependent and 4 independent Banks  Optional ECC and Parity Support  Interleaving implemented in HW provides balanced access across all channels –Interleave size is 128 bytes

PCI Interface Overview  PCI 2.2 compliant  PCI Bus Target –SRAM –DRAM –Control and Status Registers  PCI Bus Master to other devices  DMA channels –IXP2400 – 3 Channels –IXP2800 – 2 Channels  Doorbell and Mailbox Registers  Loads: –4 loads at 66MHz –8 loads at 33MHz

IXP2400 Media Switch Fabric Interface  Protocols –POS-PHY Levels 2 and 3 –Utopia Levels 1, 2 and 3 –CSIX-L1 for Switch Fabric Interface  LVTTL IO (3.3V)  32-bit receive, 32-bit transmit  25–133 MHz  8KB receive buffer and 8KB transmit buffer

IXP2800 Media Switch Fabric Interface  Protocols –SPI-4 Phase 2 for Network Device –CSIX-L1 for Switch Fabric Interface  LVDS IO (IEEE , ANSI/TIA/EIA-644)  16-bit receive, 16-bit transmit  311–500 MHz  8KB receive buffer and 8KB transmit buffer

Miscellaneous  UART –Standard RS232 primarily for debugging  TIMER – bit timers –Timer 4 can be used as Watchdog Timer  GPIO –8 General Purpose IO pins –Can be used as interrupt source to XScale core or clock to timers  Interrupt Controller –Provides the ability to enable or mask interrupts from a number of chip wide sources like timers, PCI devices, DRAM ECC errors, etc.  Slow Port –Used for Flash ROM access and 8, 16, or 32-bit asynchronous device access –Allows XScale do read/ write data transfers to these slave devices

Backup

IXP2400 Target Application   WAN Edge/Access Aggregation – –Includes IP Service Switches, Multiservice Switches, DSLAM, Cable Head End   Wireless Infrastructure   Layer 4-7 Switches – –Includes Firewall, Server Offload, Content-Based Load Balancing IXP2400 will provide IXP 1200 customers a performance upgrade for OC-12 applications and enable multi Gigabit Ethernet platforms up to OC-48

IXP2800 Target Application  Metropolitan Area Network (MAN) switches and routers  Internet core access switches and routers  Multi-service switches  10 Gbs enterprise switches and routers supporting tomorrow’s data centers,  Storage area networks (SAN)  Content aware server off-load/web switches.  Security/VPN solutions  Wireless base stations  Digital Subscriber Line Access Multiplexers (DSLAMs).

Edge Multi-Service Switch - WAN/LAN Solutions Edge Multi-Service Switch - WAN/LAN Solutions OC-48c ATM SAR & Traffic Manager Optical Ring WAN Backbone (ATM, SONET) Vallejo 4 x 1G Ethernet MAC IXP2400 IXP IXP2400 IXP Oahu Quad Gig Phy 1 Gig LAN Backbone or Server Farm 1 Gig LAN or Server Farm Phy Interface Utopia3 SPI-3 (Utopia3 Packet) Amazon IXF6048 OC-48c ATM & POS Framer SPI-3 (Utopia3 Packet) OR CSIX Switch Fabric 80 Gig – 1+ Terabit Switch Fabric

Edge Server Offload Edge Server Offload Vallejo 4 x 1G Ethernet MAC IXP2400 IXP IXP2400 IXP Oahu Quad Gig Phy 1 Gig LAN Backbone or Server Farm Server Farm Phy Interface CSIX Level 1 SPI-3 (Utopia3 Packet) CSIX Switch Fabric 80 Gig – 1+ Terabit Switch Fabric Host CPU (IOP or iA) PCI Bus

IXP2400 Media Configurations IXP2400 DDR QDRQDR TCAM DDR QDRQDR TCAM DDR QDRQDR TCAM 32bit Utopia 1/2/3 Or SPI-3 (POS-PHY 2/3) Or CSIX_L1B 16bit Utopia 1/2/3 Or SPI-3 (POS-PHY 2/3) 8bit Rx & Tx paths each have 2 separate clock domains for asynchronous traffic IXP2400 DDR QDRQDR TCAM 8bit16bit RxTxRxTxRxTx RxTx Each Rx & Tx path may be configured to be single 32bit, quad 8bit, dual 16bit or combination of 8 & 16bit wide buses Xscale

10Port 1Gb/s Ethernet Line Card 10x1GbE SPI I/F Fabric CSIX I/F RDR Packet Memory Ben Nevis 10Gbs 15Gbs 10Gbs PCI 64/66 SAR’ing Classification Metering Policing Initial Congestion Management Ingress Processor Traffic Shaping Flexible Choices diff serve TM 4.1 … Egress Processor IXP2800 Egress Processor DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR QDR SRAM Queues & Tables Q QQDRDRQQDRDR IXP2800 Ingress Processor DRAMDRAMDRAM QDRQDRQ QQDRDRQQDRDR RDR Packet Memory QDR SRAM Queues & Tables Q QQDRDRQQDRDR 10 x 1 GbE LAN Control Plane Processor Flow Ctl Fabric Interface Chip (FIC)

10Gb/s to Infiniband 10GbE 10x1Gb OC-192c SPI I/F IXP2800 Ingress Processor IXP2800 Egress Processor Fabric CSIX I/F DRAMDRAMDRAM QDRQDRQ QQDRDRQQDRDR DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR RDR Packet Memory QDR SRAM Queues & Tables RDR Packet Memory QDR SRAM Queues & Tables Calypso Ben Nevis Loch Lomond 10Gbs 15Gbs 10Gbs PCI 64/66 Infiniband Fabric 2.5Gbps Q QQDRDRQQDRDR Q QQDRDRQQDRDR Control Plane Processor Flow Ctl

10Gbs Ethernet to SONET 10GbE 10x1Gb SPI I/F OC-192 4xOC48 SPI I/F QDR SRAM Queues & Tables Loch Lomond Ben Nevis 10Gbs PCI 64/66 Calypso Server or Disk Farms MetroOrWAN IXP2800 Egress Processor DRAMDRAMDRAMQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR IXP2800 Ingress Processor DRAMDRAMDRAM QDRQDRQ QQDRDRQQDRDRQ QQDRDRQQDRDR RDR Packet Memory QDR SRAM Queues & Tables RDR Packet Memory Control Plane Processor Flow Ctl

Media / Fabric Receive Logic: packet Rbuf 64/128 Elements 128/64B each Status Word Per element Pkt ctrlPkt payload aPkt ctrlCell payloadPkt ctrlPkt payload b ATM Cell Idle bucket Port APort B buffer Media Device Rbuf Freelist Thread Freelist Receive State Machine Bit vector Media Switch Fabric Unit AutoPush Status to Thread Get Free element # Data Arrives Create Status Assign thread # 7 Thread pushes ID onto Freelist SPI-4.2 Frame 6 Thread moves data Discarded if idle packet