SPP V1 Memory Map John DeHart Applied Research Laboratory Computer Science and Engineering Department.

SPP V1 Memory Map John DeHart Applied Research Laboratory Computer Science and Engineering Department

Revision History 9/11/2007 (JDD): Created

Counters Two sets of counters maintained for each successful lookup:
Pre-Queue: Pkt counter and Byte counter Post-Queue: Pkt counter and Byte counter Each counter is 32-bits Organization of Counters: Addr of a pre-q pckt ctr: BASE+(4 * 4 * Index) + (0*4) Addr of a pre-q byte ctr: BASE+(4 * 4 * Index) + (1*4) Addr of a post-q pckt ctr: BASE+(4 * 4 * Index) + (2*4) Addr of a post-q byte ctr: BASE+(4 * 4 * Index) + (3*4) Counter Index 0 BASE Pre-Queue Pkt Ctr Pre-Queue Byte Ctr Post-Queue Pkt Ctr Post-Queue Byte Ctr Counter Index 1 Pre-Queue Pkt Ctr Pre-Queue Byte Ctr Post-Queue Pkt Ctr Post-Queue Byte Ctr . . . Counter Index N Pre-Queue Pkt Ctr Pre-Queue Byte Ctr Post-Queue Pkt Ctr Post-Queue Byte Ctr

LC Ingress SRAM Channel 3
0x000000 PreQ/PostQ Cntrs MI Counters 0x100000 0x100000 0x17FFFF HF Init 0x180000 0x1801ff 0x180200 Counter Error Cnts 0x200000 Unallocated 0x180220 0x180221 Unallocated 0x1FFFFF 0x300000 Unallocated 0x400000 Unallocated 0x700000 Unallocated 0x500000 Unallocated 0x7FFDFF Rx Pkt Counters (turned off for Perf.) 0x7FFE00 0x600000 Unallocated 0x7FFE9F Tx Pkt Counters (turned off for Perf.) 0x7FFEA0 0x700000 0x7FFF3F Unallocated 0x7FFF40 0x7FFFFF

0x000000 Buffer Descriptors (0x38000 * 32B) ( * 32B) (7MB) 0x700000 QM1 Sched0 QDs (2048 * 16B) 0x708000 QM1 Sched1 QDs (2048 * 16B) 0x710000 QM1 Sched2 QDs (2048 * 16B) 0x700000 QM1 Queue Desc Array (16384 * 16B = 256KB) 0x718000 QM1 Sched3 QDs (2048 * 16B) 0x740000 QM1 Queue Desc Array (16384 * 16B = 256KB) 0x720000 QM0 Sched4 QDs (2048 * 16B) 0x728000 0x780000 Unused QDs (2048 * 16B) QM2 Queue Desc Array (16384 * 16B = 256KB) 0x730000 Unused QDs (2048 * 16B) 0x7C0000 QM3 Queue Desc Array (16384 * 16B = 256KB) 0x738000 Unused QDs (2048 * 16B) 0x740000

0x000000 QM QParams (No longer pre-assigned to a QM) (32768 * 16B) 0x040000 0x080000 0x0C0000 0x080000 QM0 Sched (3278 * 44B= 0x23368) 0x0A3368 QM1 Sched (3278 * 44B= 0x23368) 0x0C66D0 QM2 Sched (3278 * 44B= 0x23368) 0x0E9A38 QM3 Sched (3278 * 44B= 0x23368) 0x10CDA0 QM0 Freelist (3278 * 4B= 0x3338) 0x1100D8 0x119A80 QM1 Freelist (3278 * 4B= 0x3338) QSCHED Params (Rate and Intfc) (128B) 0x113410 0x119AFF QM2 Freelist (3278 * 4B= 0x3338) 0x119B00 Unallocated 0x116748 QM3 Freelist (3278 * 4B= 0x3338) 0x7FFFFF

LC Egress SRAM Channel 3 0x000000 PreQ/PostQ Cntrs MI Counters
0x17FFFF 0x100000 0x180000 HF Init 0x1801FF 0x180200 Counter Error Cnts 0x18021F 0x200000 0x180220 Flow Stats Hash Table (4MB) Unallocated 0x1FFFFF 0x300000 0x700000 FS to XScale SRAM Ring 0x400000 0x73FFFF 0x740000 FS Freelist SRAM Ring 0x77FFFF 0x780000 Unallocated 0x500000 Rx Pkt Counters (turned off for Perf.) 0x7FFE00 0x600000 Unallocated 0x7FFE9F Tx Pkt Counters (turned off for Perf.) 0x7FFEA0 0x700000 0x7FFF3F Unallocated 0x7FFF40 0x7FFFFF

LC Egress SRAM Channel 2 0x000000 0x100000 Buffer Descriptors
(7MB) 0x100000 0x200000 0x300000 0x400000 0x500000 0x600000 0x700000 Queue Desc Array (65536 * 16B = 1MB)

(No longer pre-assigned
LC Egress SRAM Channel 1 0x000000 QM QParams (No longer pre-assigned to a QM) (32768 * 16B) 0x040000 0x080000 0x0C0000 0x080000 QM0 Sched (3278 * 44B= 0x23368) 0x0A3368 QM1 Sched (3278 * 44B= 0x23368) 0x0C66D0 QM2 Sched (3278 * 44B= 0x23368) 0x0E9A38 QM3 Sched (3278 * 44B= 0x23368) 0x10CDA0 QM0 Freelist (3278 * 4B= 0x3338) 0x1100D8 0x119A80 QM1 Freelist (3278 * 4B= 0x3338) QSCHED Params (Rate and Intfc) (128B) 0x113410 0x119AFF QM2 Freelist (3278 * 4B= 0x3338) 0x119B00 Unallocated 0x116748 QM3 Freelist (3278 * 4B= 0x3338) 0x7FFFFF

Egress Scratch Memory (16 KB)
0x0000 0x0000 PACKET_TX_AVAIL_TBUF_ELE_SCR_ADDR (4B) 0x0400 0x0100 SRAM RING OCCUPANCY COUNTERS 0x0800 0x0200 0x0C00 0x0300 0x1000 TO_XSCALE_RING (Ring 1) 0x1400 FROM_XSCALE_RING (Ring 2) 0x1800 COUNTER_RING (Ring 3) 0x1C00 PS_TO_QM_RING_1 (Ring 4) 0x2000 PS_TO_QM_RING_2 (Ring 5) 0x2400 QM_TO_FS_RING_1 (Ring 6) 0x2800 QM_TO_FS_RING_2 (Ring 7) 0x2C00 FS1_TO_FS1_RING (Ring 8) 0x3000 FS_TO_TX_RING_1 (Ring 9) 0x3400 FS_TO_TX_RING_2 (Ring 10) 0x3800 0x3C00

SE Per Slice Control Tbl
NPE/MR SRAM Channel 3 0x000000 0x000000 SD Init (8B) 0x000007 HF Init (40B) 0x000008 0x00002F 0x100000 PreQ and PostQ Ctrs 0x000030 SE Init (16B) 0x00003F 0x000040 SE Src IP Tbl (128B) 0x0000BF 0x0000C0 0x200000 SD VLAN Code Opt Tbl (0x1000 Entries x 24B) Unallocated 0x000FFF 0x001000 0x218000 Rx Pkt Counters (2560B) (turned off for Perf.) Unallocated 0x00127F 0x300000 SE Per Slice Control Tbl (0x1000 Entries x 20B) 0x001280 Tx Pkt Counters (2560B) (turned off for Perf.) 0x0014FF 0x314000 0x001500 Unallocated SD Counters (24B) 0x001517 0x400000 Unallocated 0x001518 Error Ctrs (32B) 0x001537 0x001538 Slice Specific Mem 0x011537 0x500000 Unallocated 0x011538 IPv4/I3 specific data (0x1000B) 0x012537 0x600000 Unallocated 0x012538 Unallocated 0x0FFFFF 0x700000 Unallocated

NPE/MR SRAM Channel 2 0x000000 0x100000 Buffer Descriptors
(7MB) 0x100000 0x200000 0x300000 0x400000 0x500000 0x600000 0x700000 Queue Desc Array (65536 * 16B = 1MB)

(No longer pre-assigned
NPE/MR SRAM Channel 1 0x000000 QM QParams (No longer pre-assigned to a QM) (32768 * 16B) 0x040000 0x080000 0x0C0000 0x080000 QM0 Sched (3278 * 44B= 0x23368) 0x0A3368 QM1 Sched (3278 * 44B= 0x23368) 0x0C66D0 QM2 Sched (3278 * 44B= 0x23368) 0x0E9A38 QM3 Sched (3278 * 44B= 0x23368) 0x10CDA0 QM0 Freelist (3278 * 4B= 0x3338) 0x1100D8 0x119A80 QM1 Freelist (3278 * 4B= 0x3338) QSCHED Params (Rate and Intfc) (128B) 0x113410 0x119AFF QM2 Freelist (3278 * 4B= 0x3338) 0x119B00 Unallocated 0x116748 QM3 Freelist (3278 * 4B= 0x3338) 0x7FFFFF

Extra Slides The rest of the slides are old or here for extra info.

LC Egress SRAM Channel 1 0x000000 QM0 QParams 0x100000 QM1 QParams
(65536 * 16B) 0x100000 QM1 QParams (65536 * 16B) 0x200000 QM2 QParams (65536 * 16B) 0x300000 QM3 QParams (65536 * 16B) 0x400000 QM0 Sched (13109 * 44B= 0x8CD1C) 0x48CD1C QM1 Sched (13109 * 44B= 0x8CD1C) 0x519A38 QM2 Sched (13109 * 44B= 0x8CD1C) 0x5A6754 QM3 Sched (13109 * 44B= 0x8CD1C) 0x633470 QM0 Freelist (13109 * 4B= 0xCCD4) 0x640344 QM1 Freelist (13109 * 4B= 0xCCD4) 0x64D018 QM2 Freelist (13109 * 4B= 0xCCD4) 0x6669C0 Unallocated 0x659CEC QM3 Freelist (13109 * 4B= 0xCCD4) 0x7FFFFF

0x000000 QM0 QParams (65536 * 16B) 0x100000 QM1 QParams (65536 * 16B) 0x200000 QM2 QParams (65536 * 16B) 0x300000 QM3 QParams (65536 * 16B) 0x400000 QM0 Sched (13109 * 44B= 0x8CD1C) 0x48CD1C QM1 Sched (13109 * 44B= 0x8CD1C) 0x519A38 QM2 Sched (13109 * 44B= 0x8CD1C) 0x5A6754 QM3 Sched (13109 * 44B= 0x8CD1C) 0x633470 QM0 Freelist (13109 * 4B= 0xCCD4) 0x640344 QM1 Freelist (13109 * 4B= 0xCCD4) 0x64D018 QM2 Freelist (13109 * 4B= 0xCCD4) 0x6669C0 Unallocated 0x659CEC QM3 Freelist (13109 * 4B= 0xCCD4) 0x7FFFFF

NPE/MR SRAM Channel 1 0x000000 QM0 QParams 0x100000 QM1 QParams

Egress Scratch Memory (16 KB)
0x0000 PACKET_TX_AVAIL_TBUF_ELE_SCR_ADDR TO_SCALE_RING FROM_XSCALE_RING COUNTER_RING PS_TO_QM_RING_1 PS_TO_QM_RING_2 QM_TO_TX_RING_1 QM_TO_TX_RING_2 0x1F00 0x1F04 … 0x2400 0x2800 0x2C00 0x3000 0x3400 0x3800 0x3c00 0x3FFC

dl_system #define BUF_SRAM_BASE SRAM_CHANNEL_1_BASE_ADDRESS #define NUM_IN_PORTS 10 ; number of input ports #define NUM_OUT_PORTS 10 ; number of output ports #define_eval PACKET_COUNTERS_SRAM_BASE SRAM_CHANNEL_3_BASE_ADDRESS #define RX_PACKET_COUNTERS_SRAM_SIZE 16 * NUM_IN_PORTS #define_eval PACKET_TX_COUNTER_BASE PACKET_COUNTERS_SRAM_BASE + RX_PACKET_COUNTERS_SRAM_SIZE #define PACKET_TX_COUNTER_SIZE 16 * NUM_OUT_PORTS #define PS_TO_QM_RING_ #define PS_TO_QM_RING_1_SIZE #define PS_TO_QM_RING_1_BASE 0x3000 #define PS_TO_QM_RING_ #define PS_TO_QM_RING_2_SIZE #define PS_TO_QM_RING_2_BASE PS_TO_QM_RING_1_BASE + 4 * PS_TO_QM_RING_1_SIZE #define QM_TO_TX_RING_ #define QM_TO_TX_RING_1_SIZE #define QM_TO_TX_RING_1_BASE PS_TO_QM_RING_2_BASE + 4 * PS_TO_QM_RING_2_SIZE #define QM_TO_TX_RING_ #define QM_TO_TX_RING_2_SIZE #define QM_TO_TX_RING_2_BASE QM_TO_TX_RING_1_BASE + 4 * QM_TO_TX_RING_1_SIZE Um, shouldn’t this be “4” for the four bytes per queue entry? And… Should the rings be 256 or 512?

Planet Lab Memory Map The following SRAM is used in every design:
Buffer Descriptors in Channel 1 (8 MB): 0x RX PACKET COUNTERS (debug) in Channel 3 (640 bytes) TX PACKET COUNTERS (debug) in Channel 3 (640 bytes) Scratch Memory usage: TO_XSCALE_RING: 0x2800 – 0x2bff FROM_XSCALE_RING: 0x2c00 - 0x2cff PS_TO_QM_RING_1: 0x3000 – 0x33ff PS_TO_QM_RING_2: 0x x37ff QM_TO_TX_RING_1: 0x3800 – 0x3bff QM_TO_TX_RING_2: 0x3c00 – 0x3fff TX1 - PACKET_TX_AVAIL_TBUF_ELE_SCR_ADDR: SCRATCH_BASE + 0x1f00

Substrate Decap (IPv4) Memory Map
Initialization data Enet MAC Addr MR blade (48b) Dynamic Data VLAN-to-CodeOption index table 2^12 = 4096 VLANs 4b Code Option field 1LW for minimal cycles (16KB req’d: SRAM) or, 4b for minimum memory (2KB req’d: SRAM, Scratch, LM) Counters 2 LWs total for packets received & sent 4 LWs total for Ethernet VLAN validation 6 LWs per VLAN for UDP/IP validation using power-of-2 sizes saves cycles; multiply becomes shift now 32B per VLAN x 4096 VLANs = 128KB SRAM req’d

Parse (IPv4) Memory Map Initialization data Dynamic Data: counters
none; only compile-time constants and ring data used Dynamic Data: counters 2 LWs total for packets received and sent 9 LWs per VLAN for UDP/IP validation using power-of-2 sizes saves cycles; multiply becomes shift now 64B per VLAN x 4096 VLANs = 256KB SRAM req’d

Lookup (IPv4) Memory Map
Initialization data Dynamic Data

HF(IPv4) Memory Map Initialization data (88b + #GP*88b)
Enet Addr MR blade (48b) IP Addr MR blade (32b) #GPs (8b) Table[#GP] Enet Addr (8b) Upper 40 bits are same as Enet Addr MR Blade above IP Addr (8b) Upper 24 bits are same as IP Addr MR Blade above Local Delivery UDP dest (16b) QID (20b) Exception Dynamic Data (None)

QM (IPv4/LCI/LCE) Memory Map
Constants #Q – 128K (total for all QM) QM_NUM_QUEUES QD_BASE_ADDR 0x000000 QD_SIZE 0x200000 QPARAMS_BASE_ADDR 0x200000 QPARAMS_SIZE 0x200000 QSCHED_BASE_ADDR 0x400000 QSCHED_SIZE 0x140000 PORT_RATES_BASE_ADDR 0x540000 PORT_RATES_SIZE 0x28 Initialization data (none) Dynamic Data (SRAM) Port Rates (5 * 32b) Per Queue Queue Parameters (per queue) Discard Threshold (32b) Quantum (32b) Qlen (32b) Rsvd (32b) Queue Descriptor (128b) Scheduling Structure/Freelist (SRAM) (12W/5Q)

KeyExtractor (LC) Memory Map
Ingress Initialization data (none) Dynamic Data (none) Egress

Lookup (LC) Memory Map Ingress Egress Initialization
Dynamic Data (none) Egress Initialization (none)

HF (LC) Memory Map Egress Initialization Dynamic Data (none)
Ingress Initialization Enet Addr (48b) Dynamic Data (none) Egress Initialization (none)

Port Splitter (LC) Memory Map
Ingress Initialization (none) Dynamic Data (none) Egress

SRAM Channel 1 (8 MB) – 0x40000000 0x000000 0x100000 0x219A38 0x2333E0
Queue Parameters - Each is 4, 32-bit words (16 bytes) So room for 64K Queue Scheduling Structure (*2) - Each is 11, 32-bit words (44 bytes) Allocated 13109 Queue Free List (*2) - Each is 1, 32-bit words (4 bytes) Allocated (same as # of QSS) Port Rates Each is 1, 32-bit words (4 bytes) Ten ports 0x100000 0x219A38 0x2333E0 0x233408 . 0x7FFFFC

SRAM Channel 2 (8 MB) – 0x80000000 BUF_SRAM_BASE - Buffer Descriptors
- Each is 8 32-bit words (32 bytes) - So room for 224K Queue Descriptors Each is 4 32-bit words (16 bytes) - So room for 64K 0x6FFFFF 0x7FFFFC

IPv4 SRAM Channel 3 (8 MB) – 0xC0000000
Packet/Byte Counters (4 words * 16K) - PRE_Q_PKT_CNT PRE_Q_BYTE_CNT POST_Q_PKT_CNT POST_Q_BYTE_CNT MI Counters (512K) Initialization Data 0x010000 0x090000 0x190000 0x7FFFFC

Per Block Initialization Memory RX: Base = 0x000000, Size = 0 SD: Base = 0x000000, Size = 8 PR: Base = 0x000008, Size = 0 LK: Base = 0x000008, Size = 0 HF: Base = 0x000008, Size = 8 PS: Base = 0x000010, Size = 0 QM: Base = 0x000010, Size = 0 TX: Base = 0x000010, Size = 0 0x000010 0x001000

Per Block Dynamic Memory RX: Base = 0x001000, Size = 0x280 TX: Base = 0x001280, Size = 0x280 SD: Base = 0x001500, Size = 0x24018 PR: Base = 0x025018, Size = 0x40008 IPv4 POST_Lookup Counters (PreQ, PostQ): Base: 0x025018, Size 0x 0x074520 0x100000

BUF_SRAM_BASE - Buffer Descriptors - Each is 8 32-bit words (32 bytes) - (0x – 0x100000)/32B = - So room for Buffer Descriptors 0x100000 0x7FFFFC

LC SRAM Channel 3 (8 MB) – 0xC0000000
PACKET_COUNTERS_SRAM_BASE - 16 words per port, 16 ports (1024 bytes, total) - Used in RX for debugging PACKET_TX_COUNTER_BASE Per Lookup Result Counters (pre-Q and post-Q) - 16 MR, 256 indices, 4 32-bit counters per index = 64KB (See next slide for ctr details) Per MI Counters - 64K * 2 * 4 = 512KB 0x000400 0x000800 0x010800 0x090800 0x100000 0x7FFFFC

0x000000 Q Params (65536 * 16B) 0x100000 QM0 Sched (13109 * 44B= 0x8CD1C) 0x18CD1C QM1 Sched (13109 * 44B= 0x8CD1C) 0x219A38 QM0 Freelist (13109 * 4B = 0xCCD4) 0x22670C QM1 Freelist (13109 * 4B = 0xCCD4) 0x2333E0 QM0 & QM1 Port Rates (10 * 4B) 0x233418 Unallocated 0x600000 Unallocated 0x7FFFFF

LC Egress SRAM Channel 1 0x000000 Q Params (65536 * 16B) 0x100000
QM0 Sched (13109 * 44B= 0x8CD1C) 0x18CD1C QM1 Sched (13109 * 44B= 0x8CD1C) 0x219A38 QM0 Freelist (13109 * 4B = 0xCCD4) 0x22670C QM1 Freelist (13109 * 4B = 0xCCD4) 0x2333E0 QM0 & QM1 Port Rates (10 * 4B) 0x233418 Unallocated 0x600000 Unallocated 0x7FFFFF

NPE/MR SRAM Channel 1 0x000000 Q Params (65536 * 16B) 0x100000
QM0 Sched (13109 * 44B= 0x8CD1C) 0x18CD1C QM1 Sched (13109 * 44B= 0x8CD1C) 0x219A38 QM0 Freelist (13109 * 4B = 0xCCD4) 0x22670C QM1 Freelist (13109 * 4B = 0xCCD4) 0x2333E0 QM0 & QM1 Port Rates (10 * 4B) 0x233418 Unallocated 0x600000 Unallocated 0x7FFFFF

0x000000 QM0 QParams (16384 * 16B) 0x000000 QM0 Sched0 QParams (2048 * 16B) 0x040000 QM1 QParams (16384 * 16B) 0x008000 QM0 Sched1 QParams (2048 * 16B) 0x080000 QM2 QParams (16384 * 16B) 0x010000 QM0 Sched2 QParams (2048 * 16B) 0x0C0000 QM3 QParams (16384 * 16B) 0x018000 QM0 Sched3 QParams (2048 * 16B) 0x100000 QM0 Sched (3278 * 44B= 0x23368) 0x020000 QM0 Sched4 QParams (2048 * 16B) 0x123368 QM1 Sched (3278 * 44B= 0x23368) 0x028000 Unused QParams (2048 * 16B) 0x1466D0 QM2 Sched (3278 * 44B= 0x23368) 0x030000 Unused QParams (2048 * 16B) 0x169A38 QM3 Sched (3278 * 44B= 0x23368) 0x038000 Unused QParams (2048 * 16B) 0x18CDA0 QM0 Freelist (3278 * 4B= 0x3338) 0x040000 0x1900D8 0x199A80 QM1 Freelist (3278 * 4B= 0x3338) QSCHED Params (Rate and Intfc) (128B) 0x193410 0x199AFF QM2 Freelist (3278 * 4B= 0x3338) 0x199B00 Unallocated 0x196748 QM3 Freelist (3278 * 4B= 0x3338) 0x7FFFFF

LC Egress SRAM Channel 1 0x000000 QM0 QParams 0x100000 QM1 QParams

NPE/MR SRAM Channel 1 0x000000 QM0 QParams 0x000000 QM0 Sched0 QParams
(16384 * 16B) 0x000000 QM0 Sched0 QParams (2048 * 16B) 0x040000 QM1 QParams (16384 * 16B) 0x008000 QM0 Sched1 QParams (2048 * 16B) 0x080000 QM2 QParams (16384 * 16B) 0x010000 QM0 Sched2 QParams (2048 * 16B) 0x0C0000 QM3 QParams (16384 * 16B) 0x018000 QM0 Sched3 QParams (2048 * 16B) 0x100000 QM0 Sched (3278 * 44B= 0x23368) 0x020000 QM0 Sched4 QParams (2048 * 16B) 0x123368 QM1 Sched (3278 * 44B= 0x23368) 0x028000 Unused QParams (2048 * 16B) 0x1466D0 QM2 Sched (3278 * 44B= 0x23368) 0x030000 Unused QParams (2048 * 16B) 0x169A38 QM3 Sched (3278 * 44B= 0x23368) 0x038000 Unused QParams (2048 * 16B) 0x18CDA0 QM0 Freelist (3278 * 4B= 0x3338) 0x040000 0x1900D8 0x199A80 QM1 Freelist (3278 * 4B= 0x3338) QSCHED Params (Rate and Intfc) (128B) 0x193410 0x199AFF QM2 Freelist (3278 * 4B= 0x3338) 0x199B00 Unallocated 0x196748 QM3 Freelist (3278 * 4B= 0x3338) 0x7FFFFF

NPE/MR SRAM Channel 1 0x000000 QM0 QParams 0x100000 QM1 QParams

SPP V1 Memory Map John DeHart Applied Research Laboratory Computer Science and Engineering Department.

Similar presentations

Presentation on theme: "SPP V1 Memory Map John DeHart Applied Research Laboratory Computer Science and Engineering Department."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

SPP V1 Memory Map John DeHart Applied Research Laboratory Computer Science and Engineering Department.

Similar presentations

Presentation on theme: "SPP V1 Memory Map John DeHart Applied Research Laboratory Computer Science and Engineering Department."— Presentation transcript:

Similar presentations

About project

Feedback