1. Flow Processing for Fast Path & Inline Acceleration
Srinivasa Addepalli – Intel
Rajesh Madabhushi- Freescale
Denis Crasta - Freescale

2. Background : Performance requirements & Solutions
VPN VM
SGW
High PPS (Packets/sec)
Low Latency
Low Jitter
High single flow performance
Packet Order maintenance in a flow
IKE
S1-CP
IPSec Control
SGW-DP
IPSec Data Path
L2TP
NAT
Router
L2TP-CP
Normal Path
Control
L2TP-DP
CGNAT-FP
Forwarding - DP
PGW FW
SLB
Control
Normal Path
Normal Path
PGW-DP
FW-FP
SLB-FP
Data Path for many network services is simple and can be separated from Control-path/Normal-Path
Data Path in physical appliances is normally implemented using specialized packet processors Specialized Packet Processors. Some examples include
Network processors (eg. PCIe based, Network based)
Inbuilt (to SoC) specialized network processing
FPGA
Few examples of network functions

3. Background : Clouds - Virtual Machines with both Control and Data Planes together
MME
HSS
PCRF
AAA
Web Server
DB Server
App Server
Anti-Malware
ADC
WAF
API Gateway/Security
Scanner
Monitoring tools
Hadoop
Example Virtual Machines
L2TP
NAT
Router
L2TP-CP
Normal Path
Control
L2TP-DP
CGNAT-FP
Forwarding - DP
VPN VM
SGW
PGW FW
SLB
IKE
S1-CP
Control
Normal Path
Normal Path
IPSec Control
SGW-DP
IPSec Data Path
PGW-DP
FW-FP
SLB-FP
Host Linux
Compute nodes
Full inspection Applications
No data packet inspection Apps
Partial inspection apps
CP-DP separation is possible for partial and no-data inspection services/applications

4. Background – Continued…
Virtualization Challenges (Over Physical appliances)
Performance Challenges
- vSwitch overhead
Unable to utilize the specialized packet processors
Virtualization Considerations
Any vNF should run on compute nodes.
Supply Chain – Multiple suppliers in creating cloud environment
Compute node hardware vendors
VMM vendors
Multiple vNFs & Multiple virtual appliance vendors.
Multiple accelerator vendors
No/Minimal dependencies among the suppliers
Virtual appliances must continue to work with newer hardware (Accelerator and compute node) vendors without operator having to go back to virtual appliance vendors for image upgrades
New virtual appliances should be able to take advantage of performance features.
Standardization and programmability is the key.
10/25/2019

5. Background : vSwitch Overheads – Mitigations
Traditional
Accelerated vSwitch compute nodes
vSwitch Overheads are minimized
Direct connectivity with no changes to vNF is achieved using virtio interface
10/25/2019

6. Requirement : vNF Data Paths in Packet processors
Local apps
Local apps
Network Function Control
Network Function Control
Mitigate performance challenges
Bring performance closer to the physical appliances
DP Control
DP Control
User space
Kernel space
Nova compute
Neutron agent
TCP/IP
ovs-vsctl
ovs-ofctl
IPtables
OVSDB
Ovs-vswitchd
Compute IPSec Control
vSwitch DP Control
Smart-NIC (FPGA, NP, Inbuilt NP) / DPDK based vSwitch acceleration
vHost Eth
Network Function Data Path for vNF1
Network Function Data Path for vNF2
But… Cloud challenges….
vSwitch DP

7. Challenge – Proliferation of Data Path functions
L2TP
NAT
Router
L2TP-CP
Normal Path
Control
VPN VM
SGW
PGW FW
SLB
IKE
S1-CP
Control
Normal Path
Normal Path
IPSec Control
Host Linux
Smart NIC/DPDK
vhost - Ethernet
Forwarding - DP
IPSec Data Path
SGW-DP
PGW-DP
FW-FP
SLB-FP
CGNAT-FP
L2TP-DP
vSwitch DP
Many Data Path Functions now and in future  Could be a challenge for smart-NIC vendors  Challenge for operators in coordinating between virtual appliance vendors and smart-NIC vendors.
Virtual appliance vendors might be satisfied with DP functions provided by smart-NIC vendors. vNF vendors might want to add their own extensions
Choices :
Provide a way for uploading of new DP modules in smart-NIC
Flow programmable DPs (Flow processor)

8. Challenges with Dynamic DP Modules
Many smart-NIC vendors and Smart-NIC technologies
Virtual appliance vendors supporting many of them could be a challenge
Reliability Concerns
Smart-NIC may not provide good isolation among DPs -> Malicious and poorly coded DP could make entire smart-NIC unstable.
Security Concerns
Performance Isolation concerns
Coordination & Code Bloating Concerns
Multiple versions of VMs of same type and multiple DP versions -> Code bloating.
Version mismatch
Dependencies among suppliers etc…
10/25/2019

9. One Logic (One code piece)
Flow Processing
One Logic (One code piece)
Run time programmable using messages from the control logic (No code upgrade).
Multiple Instances of Data Paths.
Each instance can be programmable independently with tables, flows in tables and action in flows.
Pipeline of flows across the tables.
Known best flow processing today
Openflow
Standardized by ONF.
Ability to dynamically program the flows and actions.
Ability to create multiple instances.
Ability to assign the ownership of control to instances.
Ability to control memory usage
Possible to implement processing isolation across instances
Proven/WIP to create DPs for Forwarding, IPSec, Firewall/NAT, SGW, PGW etc..
Possible to extend or create complementary specifications to meet our requirements.
10/25/2019

10. Mitigation of proliferation of DP logic modules using Openflow
VPN VM
SGW
PGW FW
SLB
IKE
S1-CP
Control
Normal Path
Normal Path
IPSec Control
Virtio-flow
Virtio-flow
Virtio-flow
Virtio-flow
Virtio-flow
Host Linux
Smart NIC/DPDK
vhost - Ethernet
vhost - flow
Openflow
DP Instance 1
DP Instance 2
DP Instance 3
DP Instance 4
DP Instance 5
As many DP instances as number of vNFs in the compute node.
Smart-NICs to have only one logic module (Openflow).
Each vNF controls its own DP instance by programming tables, flows and actions at run time.
May require upgrade to smart-NIC software only for bug fixes & to implement OF extensions.

11. Identify the components to enable Openflow based Data Paths
DPACC Proposal
Identify the components to enable Openflow based Data Paths
Openstack Controller
Openstack agents
QEMU/KVM side
Define the API and messages to communicate from Control to Data Planes
For standardization.
Binary compatibility and source compatibility.
Identify the gaps & Document them.
Security and reliability aspects
Functional limitations
Any other…
Create a PoC
Define the 1st functional PoC
Identify the existing Open Source
Identify the gaps.
10/25/2019

12. Openflow based Network Function Data Path
10/25/2019

13. Without VNF Accelerated Data Path With VNF Accelerated Data Path
Local apps
Network Function
Network Function Control
DP Control
User space
Kernel space
Nova compute
Neutron agent
TCP/IP
ovs-vsctl
ovs-ofctl
IPtables
OVSDB
Ovs-vswitchd
Compute IPSec Control
vSwitch DP Control
Smart-NIC (FPGA, NP, Inbuilt NP) / DPDK based vSwitch acceleration
vHost Eth
Network Function Data Path for vNF2
Low Latency, Jitter
High throughput
vSwitch DP
October 25, 2019

14. OF Data Path OFLS – Data Path Flexibility to configuration entity :
Parse & Validation
Lookup
Instruction
Parse & Validation
Lookup
Instruction
Parse & Validation
Reassembly
Lookup
Instruction
Pase & Validation
Traffic Management
PKT-IN I1 I2 I3 I4 I5 I1 I2 I3 I4 I5 A W M
Action
Write
Meter A
Action A1 A2 G1 A3 A1 A2 A3 A4
PKT-Out O
A11
A12
A13
A14
Output Port
Flexibility to configuration entity :
Data Path Instances can be created and deleted.
Ports can be attached/Detached from data path instances.
Flexibility to Control entity :
Ability to create multiple pipeline stages (Each stage consisting of a table, set of flows in a table).
Ability to create/delete flows in tables.
Ability to add instructions/actions to the flows for packet transformation.
Control the packet flow using go-to instruction.

15. Advantages of OF based inline/fast-path
One common interface from vNFs irrespective of fast-path type.
Multiple fast-path types can be realized using Openflow.
Many Newer enhancements to fast-path does not require software upgrade of fast-path. Can be realized using flow programming from the vNFs.
Simpler interoperability – One time, but not for each fast-path.
No/minimal dependencies between smart-NIC vendors and vNF vendors.
October 25, 2019

16. OF based Data Path realization – Compute node Components
OF Control Protocol layer using vrings
A vring for messages from control layer.
A vring to receive responses from OF DP.
A vring for notifications from OF DP.
NF DP-CP Glue Layer
A glue layer to get hold of DP objects from network function. Example : Netlink socket based module to receive Linux TCP/IP (firewall, IPSec, routing, ARP) objects that would eventually need to be sent to OF DP as OF flows.
NF DP Object translation Layer:
A layer that implements table patterns in OF DP and translated Network function objects into OF flows with right instructions/actions.
Virtio-flow vring backend :
A module that receives the flows, instructions/actions from the front-end and programs them in OF DP.
Virtio-flow setup backend :
A module that informs vring addresses to virtio-flow backend.
NOVA-compute/libvirtd/QEMU:
Integration with Openstack Controller to create OF DP instances and expose OF DP instances to vNFs.
Local apps
Network Function
NF DP to CP Glue Layer
NF DP Object translation Layer
Virtio-flow front-end driver
VNF
NOVA Compute
QEMU
Libvirtd
Virtio-flow setup backend
Smart-NIC (FPGA, NP, Inbuilt NP) / DPDK based vSwitch acceleration
vHost Eth
Virtio-flow vring backend
Openflow Switch for vNF DPs
vSwitch DP
10/25/2019

17. Backup
10/25/2019

18. Network Function Control Network Function Control
vNF1
vNF 2
Local apps
Network Function Control
Local apps
Network Function Control
Network Function Data Path
Network Function Data Path
User space
Kernel space
TCP/IP
Nova compute
Neutron agent
Vhost-net
ovs-vsctl
ovs-ofctl
IPtables
OVSDB
OVS vSwitchD
OVS DP
VxLAN + GPE
IPSec dataplane

19. Firewall FP OVS DP VxLAN + GPE Network Function Control
vNF1
vNF 2
Local apps
Network Function Control
Local apps
Network Function Control
Network Function Data Path
Network Function Data Path
User space
Kernel space
Nova compute
Neutron agent
TCP/IP
ovs-vsctl
ovs-ofctl
IPtables
OVSDB
Ovs-vswitchd
Compute IPSec Control
vSwitch DP Control
Smart-NIC (FPGA, NP, Inbuilt NP) / DPDK based vSwitch acceleration
vHost Eth
vSwitch DP
Firewall FP
OVS DP
VxLAN + GPE
Compute IPSec FP