Ethernet Packet Filtering – Part 2 Øyvind Holmeide 10/28/2014 by

This presentation targets two FTI network challenges: 1.How to guarantee worst-case latency for latency sensitive data? 2.How to achieve near wire speed network performance on recorder ports without packet loss? FTI network challenges

1.How to guarantee worst-case latency for latency sensitive data? – Answer: Quality of Service (QoS) FTI network challenge

Quality of Service (QoS) in Ethernet QoS is relevant for: – Real time critical data – Latency sensitive data – Loss critical data QoS can be configured on Ethernet switches based on: – Layer 1: Port based priority – Layer 2: VLAN tagging (IEEE802.1p) – Layer 3: IP ToS/CoS QoS

Standard Ethernet switches provide several QoS properties: Scalable bandwidth Full duplex connectivity (no collision) Flow control off – deterministic access (send when you want) Several queues per port: – high priority packets in high queues – Low priority packets in low queues Strict/fixed priority scheduling (QoS policy) gives best QoS properties for high priority data QoS

QoS: Traffic prioritization: QoS

Layer 1 – Priority classification based on: Port based priority: All incoming packets will be given the same priority The associated packet priority is lost for the next switch in the network chain, unless: A VLAN tag including the packet priority is kept on the switch egress port QoS 7. Application 6. Presentation 5. Session 4. Transport 3. Network 2. Data link 1. Physical

Layer 2 - Priority classification based on: IEEE802.1p QoS 7. Application 6. Presentation 5. Session 4. Transport 3. Network 2. Data link 1. Physical DestinationSource XXXX type tagDestinationSourcetype 0x81000xXXXX Tagged frame Type Interpretation - 16 bit 12-bit 802.1Q VLAN Identifier Canonical - 1 bit Tagged frame Type Interpretation 3-bit Priority Field (802.1p) Ethernet layer MAC header (layer 2) without 802.1p Ethernet MAC header (layer 2) with 802.1p FCS

Layer 3 - Priority classification based on: IP ToS/CoS QoS 7. Application 6. Presentation 5. Session 4. Transport 3. Network 2. Data link 1. Physical IPv4 header (layer 3) IHL Type of service Total lenght Identification D F M F Fragment offset Time to liveHeader checksum Source address Destination IP address Options Version Source IP address IPMAC

How to guarantee worst-case latency for latency sensitive data? High priority packets = latency sensitive data Such data will never be lost in the switch due to congestion and worst case latency for such packets can be calculated if: – The total amount of high priority traffic never exceed the bandwidths of the drop links – Worst case latency for high priority packets can be calculated if the characteristics of the high priority packets are known. Example... QoS

Example : 100 Mbps with full duplex connectivity is used on all data source drop links and 1Gps is used on switch trunk ports The switch is a store-and-forward switch with a minimum switch latency of 10μs. The switch uses strict priority scheduling The latency sensitive packet has a length of 200 bytes including preamble, MAC, IP, UDP, payload, FCS and minimum IPG. The latency sensitive packets are treated as high priority packets, all other packets have less priority Up to five other end nodes may generate similar latency sensitive packets of 200 bytes that may be in the same priority queue before the packet enters the queue, and causes extra switch delay All latency sensitive packets are generated in a cyclic manner Cont’d.. QoS

Example cont’d: The worst case switch latency of a latency sensitive packet will then be: 1.16μs, store-and-forward delay. 2.10μs, minimum switch latency. 3.12μs, worst-case latency due to flushing of a packet with maximum packet length. 4.8μs, five latency sensitive packets already in the same priority queue. 46μs, worst case - total QoS

2.How to achieve near wire speed network performance on recorder ports without packet loss? Answer: Traffic shaping and/or port trunking FTI network challenge

Half duplex network: Once a network segment utilization reaches 40% the throughput decreases substantially Network throughput From Pocketbook_ethernet_switching.pdf

Full duplex network: -Collisions on the wire is no longer an issue, but -If no smart network engineering is performed and the peak load for some time period exceeds the bandwidth of a given drop link, then there is a risk for dropping packets in the Ethernet switch as a result of such network congestion. Network throughput From Pocketbook_ethernet_switching.pdf Switch Fabric Switch Fabric

Dropped packets due to network congestion depend on: -Switch buffer capacity -Multicast filters + Head-of-Line blocking prevention -QoS -Peak load Note: average network load is an unsuitable parameter for estimating probability for dropped packets due to network congestion. Average bandwidth utilization of 20-30% may still result in dropped packets if the switch load is bursty Network throughput

Without shaping Traffic shaping Recorder Bottle-neck Average load from each cluster < 150Mbps, but data is bursty Cluster 1 Cluster 2 Cluster 6

With shaping Traffic shaping Recorder No packet loss anymore Traffic is shaped, peak load reduced from 1Gps to 150Mbps Cluster 1 Cluster 2 Cluster 6

Example: A lab test setup based on the following components was established. 30 x IED simulators (Smartbit cards), where up to 30Mbps is generated per IED simulator. Data is sent in bursts 6 x CM1600 cluster switches 1 x CM1600 central switch 1 x recorder simulator Traffic shaping

Example cont’d: Two tests were performed: 1.No rate shaping enabled on the cluster switches Result: 4-5% packet loss 2.Rate shaping enabled on each of the cluster switches. Rate shaping level set to 150Mbps Result: No packet loss Traffic shaping

With shaping and port trunking Traffic shaping and port trunking Two or more ports are combined in a logical trunk by using Link Aggregation Control Protocol (LACP) according to IEEE 802.1ax Cluster 1 Cluster 2 Cluster 6

Worst case switch latency for latency sensitive data can be guaranteed if QoS techniques are used. Worst case switch latency less than 100µs is possible for a gigabit switch. Near wire speed network performance on recorder ports without packet loss can be achieved by using traffic shaping techniques Combining two ports in a logical trunk according to IEEE 802.1ax can theoretically double the bandwidth to the recorder Conclusion