1. Making queues a first class citizen : Abstracting queues in the UApi
Magnus Karlsson & Björn Töpel, Intel

2. Problem Statement Netdev queues XDP Tx queues AF_XDP Tx & Rx queues
Ethtool using queue_id
bind(af_xdp_fd, netdev, queue_id)
UAPI
Netdev queues
XDP Tx queues
AF_XDP Tx & Rx queues
NIC specific queues
Drivers decide what queue ids to use for what. No standard
Only possible to use pre-configured queues in the driver. No dynamic creation
XDP and AF_XDP Tx queues invisible to user space
How to get rid of HW queue ids in user space and still be able to create an AF_XDP socket and direct traffic to it?

3. Will focus on 1 – 3 only in this presentation
Basic Requirements
Rx: Create a dedicated Rx queue and steer traffic to it
Bridge the gap with a getsockopt to get queue ids for use in ethtool
Tx: Multiple users (XDP Tx and AF_XDP Tx) but no traffic steering
Traffic shaping and QoS can be present though
Requirements:
UAPI: create an AF_XDP socket without specifying a HW queue id
Kernel: common module/functionality for allocating, freeing and naming/enumeration of queues
Drivers: No change to drivers that do not use this feature
UAPI: configuring traffic-steering and QoS to and from this socket
By NIC HW steering or XDP programs
Will focus on 1 – 3 only in this presentation

4. High-Level Proposal: Option 1 – Hide Queues
Ethtool using fd (or opaque queue_id)
fd = bind(af_xdp_fd, netdev)
UAPI
Ethtool
Xsk
XDP
Queue module (QM)
Alloc, free & lookup fd -> HW qid
Register, alloc & free
XDP Tx queues
AF_XDP Tx & Rx queues
Driver
Driver registers queues with QM (non-netdev vs all?)
Components that need queues can alloc, free and lookup in QM
QM uses new ndo to perform allocs and frees in the driver

5. High-Level Proposal: Option 2 – Expose Queues
Devlink to expose and manipulate queues
fd = bind(af_xdp_fd, netdev, queue_id)
UAPI
Devlink
Xsk
XDP
Queue module (QM)
Alloc, free & lookup fd -> HW qid
Register, alloc & free
XDP Tx queues
AF_XDP Tx & Rx queues
Driver
Expose queues and required info to user space to make a decision
New queues can be created through ethtool or devlink, if needed
Feed the desired queue id into bind
Flow objects could be added to control steering

6. Reflections / Questions
Is the queue manager uneccesary and a subdev should be used?
Switchdev / tc offload to steer traffic
Extend __rx and __tx structs to implement a queue manager?
Socket, abstracted queue id or extended queue id in user space?
Should netdev use the queue manager too?
XDP Tx and AF_XDP Tx two separate paths. Unify?
AF_XDP ToDo list on xdp-project repo
Need infra in user space to list queues

7. Feedback from the Meeting
Option 2 preferred
Provides advanced configuration and control
Simplified user interface can be provided with libbpf
Start with the queue object, then the flow object
Use devlink to expose and manipulate queues and flows
Keep the current interface to bind()
Queue id can be treated as an opaque