An NP-Based Router for the Open Network Lab Jon Turner with Patrick Crowley, John Dehart, Brandon Heller, Fred Kuhns, Jing Lu, Mike Wilson, Charlie Wiseman, Dave Zar

Objectives for ONL Router Reproduce approximately same functionality as current hardware router routes, filters (including sampling filters), stats, plugins Extensions multicast, explicit-congestion marking Use each NPU as separate 5 port router each responsible for half the external ports xScale on each NPU implements CP functions access to control variables, memory-resident statistics updating of routes, filters interaction with plugins through shared memory simple message buffer interface for request/response

add configurable per port delay (up to 150 ms total delay) ONL NP Router xScale add configurable per port delay (up to 150 ms total delay) xScale add large SRAM ring TCAM SRAM Rx (2 ME) Mux (1 ME) Parse, Lookup, Copy (3 MEs) Queue Manager (1 ME) HdrFmt (1 ME) Tx (2 ME) Stats (1 ME) large SRAM ring Each output has common set of QiDs Multicast copies use same QiD for all outputs QiD ignored for plugin copies Plugin Plugin Plugin Plugin Plugin xScale SRAM large SRAM ring

Unicast, ARP and Multicast Each port has Ethernet header with fixed source MAC address – several cases for destination MAC address Case 1 – unicast packet with destination on attached subnet requires ARP to map dAdr to MAC address ARP cache holds mappings – issue ARP request on cache miss Case 2 – other unicast packets lookup must provide next-hop IP address then use ARP to obtain MAC address, as in case 1 Case 3 – Multicast packet lookup specifies copy-vector and QiD destination MAC address formed from IP multicast address Could avoid ARP in some cases e.g. point-to-point link but little advantage, since ARP mechanism required anyway

Proposed Approach Destination MAC address passed through QM Lookup does separate route lookup and filter lookup at most one match for route, up to two for filter (primary, aux) combine route lookup with ARP cache lookup xScale adds routes for multi-access subnets, based on ARP Route lookup for unicast, stored keys are (rcv port)+(dAdr prefix) lookup key is (rcv port)+(dAdr) result includes Port/Plugin, QiD, next-hop IP or MAC address, valid next-hop bit for multicast, stored keys are (rcv port)+(dAdr)+(sAdr prefix) lookup key is (rcv port)+(dAdr)+(sAdr) result includes 10 bit copy vector, QiD Filter lookup stored key is IP 5-tuple + TCP flags – arbitrary bit masks allowed lookup key is IP 5-tuple + flags if applicable result includes Port/Plugin or copy vector, QiD, next-hop IP or MAC address, valid next-hop bit, primary-aux bit, priority Destination MAC address passed through QM

Lookup Processing On receiving unicast packet, do route & filter lookups if dAdr on local subnet (determined by port’s IP address/mask) if MAC address returned by route (or higher priority primary filter) is valid, queue the packet and continue else, pass packet to xScale, marking it as no-MAC leave it to xScale to generate ARP request, handle reply, insert route and re-inject packet into data path else if lookup returns valid MAC, queue packet and continue else, send to xScale with no-MAC bit set On receiving multicast packet, do route & filter lookups take higher priority result from route lookup or primary filter format MAC multicast address copy to queues specified by copy vector if matching auxiliary filter, filter supplies MAC address