Author: Maros Marsalek (Honeycomb PTL) A management agent Honeycomb version: 1.16.9 Author: Maros Marsalek (Honeycomb PTL)

What is Honeycomb? A generic, model driven(YANG) management agent Northbound interfaces: RESTCONF NETCONF Generic data processing layer to do the heavy lifting (e.g. transaction management, validation) Extensible, pluggable translation layer to handle resulting data in any way Generic Provides model driven northbound interfaces out of box Provides generic data processing layer processing data from Northbound interfaces and serving them to plugins in translation layer Provides extensible translation layer, where plugins hand handle their data (modeled in their YANGs)

Building on existing components Utilizing existing Opendaylight components and tools: Yangtools Md-sal Netconf Restconf Wired into a special data processing pipeline Running in a JVM Yangtools for: yang language parser Data tree (tree like collection optimized for storing YANG structures) as in-memory database Md-sal for BA (java code) representation of YANG structures. In ODL there are 2 forms of data described by YANG (In HC, both forms are used): BI – binding independent – generated code is not used and only DOM like structures represent the data. BA – binding aware – instantiated generated code and MD-SAL APIs (which are implemented by Honeycomb in order to wire with NETCONF, RESTCONF) Netconf for NETCONF northbound server optimized to work with MD-SAL APIs (used completely as is) Restconf for REST northbound server optimized to work with MD-SAL APIs (used completely as is) NOT just an ODL application, it is a different distribution of ODL components + custom implementation for MD-SAL components + wiring to provide special way of data handling (not possible in a regular ODL application) Using Beryllium-SR2 version of ODL components

More information on HC + VPP: https://fd.io/technology Honeycomb and VPP Primary use case for HC VPP’s low level, high performance, shared memory APIs Exposed as high level NETCONF/YANG or RESTCONF APIs Easily consumable by e.g. Opendaylight controller Bare Metal/ VM/Container Data Plane Management Agent VPP is a high performance packet processor with low level, fast APIs. The goal of Honeycomb is to provide a Honeycomb distribution capable of managing VPP and exposing VPP’s APIs as high level NETCONF/YANG and RESTCONF APIs.. Honeycomb is on top of VPP, using its low level high performance shared memory APIs. Exposing them as high level NETCONF or RESTCONF APIs using YANG models, which are easily consumable by e.g. Opendaylight controller. Demo of an ODL based application talking to VPP through Honeycomb’s NETCONF is linked at the end of presentation (virtual bridge domain application). Data Plane Packet Processing More information on HC + VPP: https://fd.io/technology

Honeycomb architecture Honeycomb components and their APIs HC core functionality is split into 2 layers: Data processing layer Pipeline processing data from northbound interfaces down to translation layer Translation layer Invoked by above layer to handle configuration updates or when polling operational state from VPP Specific translation code lives in this layer in a form of extensions/plugins Data processing layer implements OLD’s API called DataBroker (DataBroker is an interface for transactional manipulation of data, exposing CRUD operations). It is BI (binding independent) (generated code is NOT used) and uses just DOM like representation of YANG structures. This makes it generic, since it does not depend on generated code for YANG models. Plus it also enables it to easily wire with existing implementations of NETCONF and RESTCONF northbound servers from Opendaylight. Translation layer exposes Honeycomb’s custom APIs. They are utilizing BA (binding aware) representation of YANG structures (instances of generated Java code from YANG). It is still generic, since the models are brought in with the plugins (implementations of Readers and Writers). Readers and writers APIs are split in terms of APIs, to provide the freedom for plugins in case their YANG models are asymetrical (Config and Operational data are too different). VPP is exposing its standard low level shared memory APIs and JVpp on top of VPP is exposing the same APIs, just available for Java code. JVpp was developed as part of HC + VPP integration, is part of VPP codebase and is: fully-generated JAVA API for VPP using JNI. Next slides zoom in on each layer..

This diagram shows Honeycomb’s layer with their major components Also shows how VPP specific translations are in a self contained translation layer extension

Data processing layer Custom data broker implementation supporting: Configuration data Read/Write User submitted configuration Stored in an instance of in memory data tree Pre-processed, validated data provided to translation layer for actual processing Operational data Read User requested state Polled from translation layer and transformed on the fly Context data Read/Write Special type of data used/produced by the translation layer Metadata required for the translation Data processing pipeline is the pipeline from NETCONF/RESTCONF northbounds down to translation layer DataBroker is MD-SAL API that handles all data processing operations. Honeycomb implements a custom one. This isn’t on the diagrams, but: Context DataTree is a special storage provided to translation layer so that plugins can store any non-config information required for their translations e.g. interface name from YANG -> Interface ID in VPP

Configuration data processing Transactional write sequence on the next slide Next slide shows transactional write in a sequence diagram

Honeycomb config write sequence Basically, data are applied to data tree for initial validation Only after they are valid they are passed to translation layer And after translation layer succeeds, data tree root is updated and OK response is sent to user.

Operational data polling On-demand operational read sequence on next slide Next slide shows operational read in a sequence diagram

Honeycomb operational read sequence Bassically, it goes straight to VPP for data and returns to user performing transformations on the fly

Data processing layer – additional features Persistence Configuration and context data persisted Reconciliation Ability to restore configuration from persisted data Ability to reverse-engineer configuration from current operational state in underlying layers (Honeycomb starts managing preconfigured device) Combination of the two Rollback In case of failure during configuration data processing in translation layer, previous configuration is reapplied in reverse order to get things back to normal These are features of data processing layer in addition to the basic config/operationl/context data management. Persistence: In case of VPP failure&restart, Honeycomb is capable of restoring the exact configuration to VPP to achieve before-restart-state. Stored as JSON. Reconciliation: Reverse engineering type of reconciliation is important since in case VPP was preconfigured and Honeycomb connects , users would not be able to continue the configuration, they would have to push the configuration that’s in VPP into Honeycomb to get them in sync, before proceeding. Honeycomb is also capable of combining the 2 types of reconciliation and the algorithm is following: https://wiki.fd.io/view/Honeycomb/Persistence Rollback: Simple application of “before configuration data” + throwing away the changes + user notification via Northbound protocol

Translation layer Generic and extensible registry for Honeycomb plugins Separates reader and writer trees: Readers - responsible for reading operational data subtrees Writers - responsible for writing configuration data subtrees Plugins provide just a set of readers and/or writers Readers and writers receive/provide data in BA format Translation layer shelters all the translation code (packaged in plugins) and invokes whatever necessary whenever it is necessary. Translation layer uses separate registries for config and operational data. These registries hold all the readers and writers provided by plugins and are called by the data processing layer. For writes, plugins literally just receive data in BA format (instances of Java code generated for yang by yangtools + mdsal + maven plugin) and invoke appropriate VPP calls. For Reads, plugins return data in BA format constructed from whatever VPP returned. Translation layer provides basic framework for readers/writers + lots of utilities, base implementations etc. to make the plugin development easier.

Translation layer – additional features Notifications Plugins are able to emit custom (YANG modeled) notifications Initializers Set of special transformers used during reconciliation to achieve the reverse engineering of current operational state into configuration in Honeycomb JSON based configuration attributes for plugins Additional features of Translation layer in addition to basic CRUD processing of config/operational data. RPCs not yet supported .. preferring data CRUD operations instead of RPC async invocation for VPP.

Configuration and dependency injection Not relying on ODL’s config subsystem Static Wiring: Using static wiring for core components with help of Google’s Guice framework Using static, but configurable wiring for HC plugins Configuration parameters Using external JSON based configuration files for non-wiring configuration Static means that it’s only wired and configured once during startup. No changes to class path or configuration are acceptable after that. Meaning that all the Honeycomb plugins have to be loaded (but not necessarily enabled) by default… or a specialized distribution is required. Honeycomb first started as ODL application, so the process of getting out of ODL ways included: Tracking down all the components running in ODL Analyzing which belong to HC, which are required by HC and which can be omitted Result of this analysis in a graph: https://jira.fd.io/secure/attachment/10701/dependencies-after.jpeg Writing providers for all the components involved in HC starting from SchemaContext service ending with Restconf server Writing new startup mechanism (eventually just a simple main that loads HC components) Writing new configuration (non wiring settings) providing mechanism

Distribution Not running in OSGi or any other container Simple Java SE application with static classpath (ODL, HC infra, HC plugins, Jetty, Netty, Guice etc.) Main class loading and starting components Distribution is created by maven. It just copies all the required jars, some config files + startup scripts into 1 place. Then produces zip or tar.gz.

Distribution compared to ODL Reduced footprint HC without plugins at rest needs: <100Mb of RAM Reduced startup times HC with VPP plugins starts in: <10 seconds Reduced size HC with VPP plugins takes: <30Mb Easy to create custom distro Not possible to reconfigure in runtime Not possible to rewire in runtime Small comparison between HC and ODL distros. Pros and cons. It is a bit tricky to calculate the footprint. It depends on many factors: JVM 32 or 64 bit (32 = lower footprint) JVM Oracle or OpenJDK (OpenJDK = lower footprint) Garbage collection (Single threaded GC = lower footprint) Heap/Metaspace size (Needs to start small and be limited) JIT (turning off JIT = lower footprint due to code cache elimination) Bytecode verification (turning off = slightly lower footprint) VM load/stress (if a VM is stressed or under heavy load, JVM might free some memory back to OS that was only used during initialization or data processing) Lower footprint as described here might compromise performance

Current release 1609 First release Available as Honeycomb for VPP DEB, RPM packages Zip, tar.gz archives Available as Honeycomb dev platform Jars Links at the end First release, available for use.

1609 features – Honeycomb infrastructure Config, Operational, Context data processing Persistence & Reconciliation Rollback CRUD operations on data Notifications Simple JSON plugin configuration Minimal distribution These are the features of Honeycomb as a generic agent. Just a recap from previous slides.

1609 features – Honeycomb for VPP Basic interface management (ietf-interfaces.yang, ietf-ip.yang) Vhost user interface Tap interface Bridge domain Vxlan tunneling Vxlan GPE GRE Vlan management Classifier management L2/L3 ACL management (ietf-access-control-list.yang) LISP NSH_SFC Just a subset of VPP’s features for the first release … this will grow with each release

Developing with Honeycomb Available tutorials, samples, documentation, tools Developing a Honeycomb plugin + distribution https://wiki.fd.io/view/Honeycomb/Releases/1609/Developing_Honeycomb_ Plugins#Developing_generic_plugins Developing a VPP Honeycomb plugin + integration https://wiki.fd.io/view/Honeycomb/Releases/1609/Developing_Honeycomb_ Plugins#Developing_plugins_for_VPP Honeycomb plugin overview https://wiki.fd.io/view/Honeycomb/Releases/1609/Developing_Honeycomb_ Plugins#Plugin_overview Detailed documentation + tutorials + samples + tools = easy development

Links and pointers for Honeycomb Wiki - https://wiki.fd.io/view/Honeycomb 1609 release notes, install/user/devel guides, samples&tutorials – https://wiki.fd.io/view/Honeycomb/Releases/1609 JIRA - https://jira.fd.io/projects/HONEYCOMB Gerrit - https://gerrit.fd.io/r/#/admin/projects/honeycomb Mailing-list – honeycomb-dev@lists.fd.io IRC channel - #fdio-honeycomb at freenode Virtual bridge domain application demo (ODL + HC + VPP) - https://www.youtube.com/watch?v=vs1XzOOpaCo&feature=youtu.be