1. Taurus: An Intelligent Data Plane
Muhammad Shahbaz
Tushar Swamy, Alex Rucker, Neeraja Yadwadkar,
Yaqi Zhang, and Kunle Olukotun
Hi, my name is Shahbaz. Today, I am going to talk about our work on Taurus, an intelligent data plane. (next slide)

2. Taurus: An Intelligent Data Plane
Programmable Data Plane
So, before I jump into the details of the talk, let me first define what we mean by an intelligent data plane. (click)
It’s a data plane (like a switch or a NIC) fused with machine intelligence (like deep learning). (next page)
fused with
Machine Intelligence
Deep Learning

3. Managing Networks is Hard!
Cloud Computing
Internet of Things (IoT)
Augmented and Virtual
Reality (AR/VR)
Managing large-scale datacenter networks today (like Amazon, Google, and Microsoft) is really hard. And, with emerging trends like (click) Cloud Computing, (click) Internet of Things, (click) Augmented and Virtual Reality, running these networks is becoming even harder; (click) thus, hindering growth and users’ quality of experience. (next page)

4. Approaches to Manage Networks are …
Control Plane
Intelligence
Slow but intelligent
Data Plane
Switch or NIC
Current approaches to managing these networks are either (click) fast-and-dumb or (click) slow-and-intelligent.
Fast yet dumb

5. Approaches to Manage Networks are …
Characteristics:
Operates on packets or flowlets
(i.e., bursts of packets)
Uses heuristics … hash, etc.
Low latency … ≤ sub µs
High throughput … Tbps
Examples:
Congestion control
Load balancing (ECMP, RSS)
Queue scheduling
and more
Data Plane
Switch or NIC
Examples of fast-and-dumb approaches are (click) congestion control, load balancing, and more; that (click) run directly in the data plane. (click) The benefit is that that data plane can operate on packets or flowlets at line rate with very low latencies and high throughput, reaching Tbps. However, these data planes today can only do basic operations, thus, resulting in solutions that rely on heuristics, e.g., hash of a packet header when load balancing traffic. It’s a strongly held belief that data plane can only do simple operations, but as I will show later in the talk, these data planes can do a lot more complicated operations without sacrificing line rate.
(next page)
Fast yet dumb
Packets In
Packets Out

6. Approaches to Manage Networks are …
Control Plane
Intelligence
Slow but intelligent
Data Plane
Switch or NIC
At the other end, we have slow-but-intelligent approaches … (next page)
Fast yet dumb

7. Approaches to Manage Networks are …
Control Plane
Intelligence
Slow but intelligent
Examples:
Anomaly detection
Automation
Recommendation
Packet
Digest
Flow
Rule
Data Plane
Switch or NIC
Examples of these approaches are anomaly detection, automation, and recommendation (i.e., suggesting what actions to perform based on certain events). (click)
In this case, (click) a packet arriving at the data plane is sent to the (click) control plane, which then decides what action to perform on the packet. (click) The control plane computes (click) a flow rule and installs it in the data plane. (click) Afterward, packets matching the given flow use the cached actions. This scheme works well when cached decisions rely on flow headers only; however, in case of machine learning, a learned model may rely on features other than the packet headers that can change asynchronously, causing packets to visit the control plane often. (click) This increases the burden on the control plane, thus, inflating latencies and lowering throughput of users’ applications running on the network.
(next page)
Characteristics:
Operates on flows
Performs complicated tasks
Sub-second latency
Low throughput
Packets In
Packets Out

8. Approaches to Manage Networks are …
Control Plane
Intelligence
Slow but intelligent
Data Plane
Switch or NIC
So, basically, there are two extremes: we can either be fast-and-dumb or slow-and-intelligent, but is that really case. (next page)
Fast yet dumb

9. Approaches to Manage Networks are …
Control Plane
Intelligence
Slow but intelligent
Data Plane
Switch or NIC
Is it possible that we can take the best of both worlds and combine them together. (next page)
Fast yet dumb

10. Approaches to Manage Networks are …
Control Plane
intelligent
Intelligence
Data Plane
Switch or NIC
Resulting in a data plane that is both fast and intelligent.
Intelligence
Fast and
intelligent

11. Taurus: An Intelligent Data Plane
Switch or NIC
Packets Out
Packets In
Intelligence
To demonstrate that, we built Taurus: a data plane that, along with its basic functionalities, can do a lot more intelligent and complicated operations like machine-learning inference within the data plane. (next page)

12. Taurus: An Intelligent Data Plane
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
Our Taurus data plane consists of five components. (next page)

13. Taurus: An Intelligent Data Plane
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
Implements a finite state machine (FSM) that operates on a user-defined parse graph
Converts the incoming packet bit stream into vectors, e.g.,
headers (IP or TCP)
features (switch, host, or other encoded state)
A packet parser (click) implements a finite state machine, and parses the incoming packet using a parse graph specified by the user. (click) It converts the incoming packet stream into headers (e.g., IP or TCP) and features (such as …).
(next page)

14. Taurus: An Intelligent Data Plane
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
Memory
ALU
A match-action table:
Memory for exact (SRAM) and ternary (TCAM) match
ALU for basic single-cycle VLIW operations (no loops or multiplication)
Performs pre- and post-processing, and switching tasks:
Pre-processing, e.g., data transformation, acquisition, and repair
Post-processing, e.g., drop, route, or encode output of the learned model on a packet
Next we have match-action tables. These tables consist of memory and an ALU stage. The memory is used to perform exact or ternary matches on the packet headers and features, followed by an ALU to carry out single-cycle VLIW operations on these headers (like modify or increment). Note that these ALUs, can’t implement loops or multiplication. This is where the Map-Reduce Unit comes in, discussed in the following slide.
In Taurus, (click) we have tables both before and after a Map-Reduce Unit, serving as pre- and post-processors respectively. The pre-processing tables perform data transformation (e.g., convert non-numeric values to real numbers) and acquisition (access out of network events from the controller). Whereas, post-processing tables drop, route, and encode results form the learned model on to the packet.

15. Taurus: An Intelligent Data Plane
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
Implements a spatial SIMD architecture, composed of compute and memory units
Compute consists of pipelined stages of functional units (FUs) and pipeline registers (PRs), running in parallel as lanes
Processes parallel map-reduce operations to accelerate linear algebra, crucial for machine learning (ML), or other network functions
The (click) map-reduce unit implements a spatial SIMD-like architecture, composed of a grid of compute and memory units. (click) The compute block further consists of stages, each having a functional unit and a pipeline register. running in parallel as lanes.
(click) Unlike match-action tables, map-reduce block can do loops and multiplications, and is designed to accelerate linear algebra, crucial for machine learning.
(next page)

16. Taurus: An Intelligent Data Plane
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
Responsible for storing and forwarding packets off of the chip:
Queuing: buffer incoming packet
Replication: clone packets across multiple egress ports (e.g., multicast)
Scheduling: forward packets based on a queuing discipline (e.g., PIFO) or instructions from the match-action tables
Lastly, we have the traffic manager responsible for (click) queuing incoming packets, (click) replicating them across ports (in case of multicast), (click) and scheduling them based on the configured queuing disciple (e.g., PIFO).
(next page)

17. Taurus: An Intelligent Data Plane
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
So, in short, these

18. Example: Anomaly Detection
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager
Packets Out
Packets In
Pre-processing
using tables
Machine
learning
Post-processing
using tables
Parsing
Scheduling
Parse incoming
packets
and read local features like an IP address
Retrieve out of
network events
like no. of failed
logins for a
connection
Process features
using a learned
function and output probability
of an anomalous packet
Select a port
based on the
probability
of an anomalous packet
Send packets
out over the
selected port

19. Evaluation 106x Within 2 – 15% [5,6]
[4] Design principles for packet parsers, ACM/IEEE ANCS’13
[5] CONGA: distributed congestion-aware load balancing for datacenters, ACM SIGCOMM’14
[6] Packet Transactions: High-Level Programming for Line-Rate Switches, ACM SIGCOMM’16

20. A Unified High-Level Abstraction ?
Future Work
A Unified High-Level Abstraction ?
Ad-hoc Low-Level DSLs
Spatial
Parse
Graph
MAT
MAT
PIFO
Packets Out
Packets In
Packet
Parser
Match-Action
Tables
Map Reduce
Unit
Match-Action
Tables
Traffic
Manager

21. Conclusion Taurus Data Plane
Switch or NIC
Intelligence
Designed to run machine-learning inference inside a data plane
Provides orders of magnitude improvement over existing approaches
Fast and
intelligent

22. Muhammad Shahbaz http://cs.stanford.edu/~mshahbaz
Questions?
With this I conclude my presentation. Thank you, and I’m happy to take any questions.
Muhammad Shahbaz