March Reducing the Number of MAC Addresses Visible to Bridges in the Data Center Bob Sultan Ben Mack-Crane Linda Dunbar

March Problem: MAC per VNIC requires big FDB Hypervisor requires MAC address per VNIC; MAC per VNIC implies many MAC Addresses; Result is FDB overflow, flooding, poor performance; Our server guys say this is the real problem in the Data Center. e.g., 50 VNICs per blade e.g., 25 blades per rack e.g., 25 racks per access bridge e.g., 25 access bridges per core bridge MAC Addresses could number in hundreds of thousands

March Proposed: MAC Address Summary VNIC MAC Addresses having common high-order bits (eg., AA) are represented by a single MAC address (eg., AA00) in the network core; FDB entries in core reduced from ‘one per VNIC’ to ‘one per NIC (or blade, or rack)’. SA=AA23 summarize here restore here SA=AA17 SA=AA42 SA=AA00, [23] SA=AA00, [17] SA=AA00, [42] AA22AA24AA25AA26 NIC VNIC BB39BB40BB41BB42 CC25CC26CC27CC28 DD77DD78DD79DD80

March LoTag Carries Low-order Bits needed to Restore Tag added for Summarization; Low-order bits (red) copied to DLO / SLO; Low-order bits of DA / SA replaced by zeros; Restoration reverses the procedure; DA SA DDDDDD000078AAAAAA data LoTag LO EthertypeDLO 16 bits24 bits 8 bytes SLO data DA SA DDDDDD000000AAAAAA summarize restore

March When the NIC Doesn’t Support S/R When NIC does not support Summary/Restore (S/R) function, the function can be supported on the NIC- facing port of the external bridge; The same MAC reduction benefits are realized. S/R AA22AA24AA25AA26 BB39BB40BB41BB42 CC25CC26CC27CC28 DD77DD78DD79DD80 No S/R Function Here Deploy S/R port on external bridge

March Independent of the Type of Forwarding Function It is immaterial whether the forwarding device with which the S/R function is associated is a Bridge or a Port MUX (e.g., a VEPA or IV); The S/R function will operate in the same manner. AA22AA24AA25AA26 BB39BB40BB41BB42 CC25CC26CC27CC28 DD77DD78DD79DD80 Port Multiplexer

March Deploying S/R Further Up the Bridge Hierarchy S/R can be deployed further up the hierarchy; Bridges outside S/R boundary do not realize MAC reduction; Useful when migrating to summarization. S/R AA22AA24AA25AA26 BB39BB40BB41BB42 CC25CC26CC27CC28 DD77DD78DD79DD80 No S/R Function here or here Deploy S/R here or here No MAC reduction on this bridge

March Talking with a NIC that Doesn’t Summarize AA22 not tagged when sending to 1234 which lacks S/R; AA22 tagged when sending to CC26; S/R at AA will allows untagged frame from 1234 to pass directly to AA25; Maximize tagging while avoiding errors. AA22AA24AA25AA26 BB39BB40BB41BB42 CC25CC26CC27CC No S/R Function Here S/R provisioned with info that frames to 1234 must not be summarized

March Talking with a NIC that Doesn’t Summarize When sending from 1234 to AA22, a Tag is created at Y and the DLO/SLO fields are populated as usual. It is not necessary to zero the low-order bits of the SA (as the SA is not summarized in this case; When sending from AA22 to 1234, a Tag is created at X and the DLO/SLO fields are populated as usual. It is not necessary to zero the low-order bits of the DA (as the DA is not summarized in this case; AA22AA24AA25AA26 BB39BB40BB41BB X Y

March When the SA is not Summarized Tagging at X when AA22 sends to 1234 (see previous slide); SA is summarized as usual; DA is not summarized; MAC Reduction is realized for the SA; DA SA 1234AA22 data LoTag LO EthertypeDLO 16 bits24 bits 8 bytes SLO data DA SA 1234AA00 summarize restore

March Key Points Increasing numbers of VNICs per NIC creates significant scaling problem in large Data Center; Summarization can reduce MAC awareness from ‘one MAC per VNIC’ to ‘one MAC per NIC’; Summarization preserves the common practice of assigning one MAC address per VNIC; Summarization can be deployed on a NIC or on an external bridge port; Summarization can be performed on the DA and not on the SA or on the SA and not on the DA; Summarization is independent of whether the attached device is a Bridge or a Port MUX Device.