Rough Outline for a Intra-Portal Protocol Version 02 Stephen Haddock August 23,

Configuration Configuration vs Discovery –Version 01 of this presentation observed that it would be theoretically possible for the Intra-Portal Protocol to follow the philosophy used to develop LACP to maximize discovery and have minimal configuration. –Version 01 recommended against this approach. –Feedback was overwhelmingly in favor of configuration over discovery. –Still need to decide how much information needs to be configured and what can be negotiated by protocol. –Still need a state machine to establish communication between Intra-Portal Protocol participants, verify correct configuration and connectivity, control the “Emulated System” when communication has been established, detect loss of communication between IPP participants, etc. 2

What needs to be configured? Portal Identifier for each Portal –Needs to be the same in all systems in a portal. –Needs to be unique within each system of a portal. –Could be the same as the System ID for the Emulated System Pros and cons of this to be discussed later. –Used to detect mis-connections of Intra-Portal Links –In systems with multiple Portals, used to assure Portal Ports and Intra-Portal Ports get associated with the correct Portal. Intra-Portal Port(s) for each Portal –Could be physical or virtual ports. Portal Port(s) for each Portal –These are the ports on each system that will become ports on the Emulated System for the Portal. 3

Configured/Negotiated Parameters -1 System ID for Emulated System of each Portal –Needs to be “globally” unique (i.e. unique across all inter- connected networks). Could be configured, but leaves no way to resolve conflict in case of misconfiguration –Propose the each system in a Portal advertise a candidate Emulated System ID. Numerically lowest candidate is used as the System ID. This also proposal also allows a means to negotiate other Emulated System parameters (e.g. Emulated System Port IDs). LACP Key –Needs to be unique within an Emulated System. –Indicates aggregation capability Ports in a system with the same Key can be aggregated together. –S i n c e w e h a v e c o n f i g u r e d w h i c h p o r t s b e l o n g t o t h e E m u l a t e d S y s t e m, a n d all of these ports can be aggregated together,can just define a single default value to be used as the Key for all DRNIs. 4

Configured/Negotiated Parameters - 2 Portal Port Identifiers –Need to be unique within an Emulated System. Could be configured, but leaves no way to resolve a conflict if there is a misconfiguration. Propose that each Portal System assign Port IDs that are unique within the system and have the MSB = 0. When the Intra-Portal Protocol establishes the Emulated System, the system with the numerically higher candidate System ID sets the MSB of it’s Port IDs to 1. (This concept can be extended to Portals with more than two systems.) 5

State Machine Suggestion received during presentation in San Diego was to add a “Divorced” state. –Instead of going from Estranged back to Single when all communication with spouse is lost, go to Divorced state. –Difference between Divorced and Single is that in Divorced you still remember some history of being married. Presumably after some long timeout, the memory fades and you transition to Single. –Divorced state could be beneficial for handling potential split-brain scenarios, and for implementing the Graceful Name Change proposal. 6

Version 01 slides 7

LACP in a Nutshell Each Port on a System advertises: 1.A System ID The same System ID value is used for all Ports on a System. 2.A Key The Key is an indication of aggregation capability. Ports that can be aggregated advertise the same Key value. 3.A Port ID The Port ID is included to handle some special cases. It is not important for a high level understanding of basic LACP concepts. LACP Selection Logic will form an aggregation between any ports that: 1.Advertise the same System ID and the same Key (called the Actor_System and Actor_Key), and 2.Receive advertisements containing the same System ID and the same Key (called the Partner_System and Partner_Key) 8

Two Systems without Distributed Aggregation 9 System A Port System B Port Each Port on System A advertises: 1.Actor_System = A 2.Actor_Key = Ax Where Ax may be the same value on some or all of the ports, or may be a different value on different ports. Each Port on System B advertises: 1.Actor_System = B 2.Actor_Key = Bx Where Bx may be the same value on some or all of the ports, or may be a different value on different ports.

Two Systems with Distributed Aggregation 10 System A Port System B Port Each Network Port on System A advertises: 1.Actor_System = A 2.Actor_Key = Ax Each (non Gateway) Port on System C advertises: 1.Actor_System = C (C could be the same as A or B, but does not need to be) 1.Actor_Key = Cn Where Cn is the same value on all of the ports. (possible) Network Link Intra-Portal Link (could be virtual) Emulated System C Port Each Network Port on System B advertises: 1.Actor_System = B 2.Actor_Key = Bx Network Link Gateway Link (virtual) Gateway Link (virtual)

Intra-Portal Protocol Creating and Maintaining a Distributed Aggregation requires: State machines in System A and System B (the “real” systems) to control the transitions between the state without distributed aggregation and the state with distributed aggregation. A protocol that 1.Determines the System ID and Key values for the Emulated System C. 2.Coordinates the Selection Logic for the Emulated System C. 3.Coordinates the distributed aggregation state machines in each of the “real” systems. 11

Configuration versus Discovery LACP designed to allow minimal configuration and maximal discovery: –A default configuration is all ports advertise the same key value. –LACP will then discover all groups of ports connecting the same pair of systems and automatically aggregate them. The Intra-Portal Protocol could conceivably follow the same philosophy: 1.Advertise ability to do distributed aggregation on all ports. 2.Form an Intra-Portal Link to any connected system that is also capable of distributed aggregation, and create an emulate system between them. 3.Use LACP (possibly with enhancements) to discover links that could form a distributed aggregation and “move” those ports to the emulated system. This is quite ambitious! 12

A less ambitious starting point 1.Configure the port(s) that are expected to form Intra- Portal Links. –Use protocol advertisements on those ports to verify that the other system also expects these to form Intra-Portal Links, and to agree on Emulated System parameters (System ID, Key value, Port IDs). 2.Configure which port(s) are expected to be “moved” to the Emulated System. –Once the Intra-Portal Link is active and Emulated System parameters agreed, use LACP to advertise the Emulated System parameters these ports. –Intra-Portal Protocol coordinates the Selection Logic of the Emulated System to form the distributed aggregation. 13

Rough Distributed Aggregation State Machine 14 SINGLE LACP advertises “real” system parameters on all ports. Send Intra-Portal Protocol advertisements (candidate Emulated System parameters) on potential Intra-Portal Port. BETROTHED LACP advertises “real” system parameters on all ports. Send Intra-Portal Protocol advertisements on Intra-Portal Port. MARRIED LACP advertises Emulated System parameters on Emulated System ports. Send Intra-Portal Protocol advertisements on Intra-Portal Port (and alternate paths?). Create Distributed Aggregation. ESTRANGED LACP advertises Emulated System parameters on Emulated System ports. Send Intra-Portal Protocol advertisements on Intra-Portal Port and alternate paths. Maintain Distributed Aggregation IPPort Operational & IPProtocol Advertisements received In sync (Emulated System parameters agreed) IPPort not Operational but communication through other paths possible All communication with spouse lost Begin IPPort Operational & IPProtocol Advertisements received Not in sync All communication with spouse lost

Intra-Portal Protocol Advertisements Ethertype Maybe use Slow Protocols Ethertype with new sub-type Maybe use new Ethertype (without chatter limits) Modeled after LACP advertisements –Contains Actor parameters Actor_System, Actor_Emulated_System, Actor_Distributed_Key, Actor_State –Contains copies of parameters received from Spouse Spouse_System, Spouse_Emulated_System, Spouse_Distributed_Key, State –In sync when the Spouse_* parameters in received advertisements match the Actor_* parameters in transmitted advertisements. The agreed Emulated_System identifier is the numerically lower of that proposed by the Actor or the Spouse. The agreed Distributed_Key is the value associated with the agreed Emulated_System identifier. May contain other TLVs in some or all advertisements for coordinating gateway selection, link selection, etc. 15

Obviously needs further refinement 16

Thank You. 17