Kafka Producer OpenDaylight Project Proposal Xiaoyu Chen, williscc, williscc@cisco.com Andrew McLachlan, amclach@cisco.com
Plugin Overview Provides an interface between OpenDaylight and a Kafka Cluster Implemented as a Northbound plugin Supports a Kafka Producer implementation Ability to publish to kafka topics Supports compression (currently gzip/snappy) Supports codec types (currentl raw/avro) but naturally extensible Supports sync/async modes Currently acquires data from the Event Topic Broker (ETB) >=Lithium Able to filter data by Topic ID (ETB) into a defined topic, using x-path Supports multiple Kafka endpoints.
Implementation Example Nothing out of the ordinary here Current code has been tested in this configuration NETCONF Notifications as an event source hweventsource (see Core Tutorials) is included in the project at time of writing Code tested against Lithium stable Kafka Consumer Kafka Consumer Kafka Cluster Kafka Producer OpenDaylight Event Topic Broker (ETB) NETCONF connector (supporting ETB SPI) hweventsource generator (supporting ETB SPI)
OpenDayLight RESTful API Northbound ODL-Kafka Plugin odl-message-bus {u'timestamp': 1444050535820, u'rawdata': '<?xml version="1.0" encoding="UTF-8"?><payload xmlns="urn:cisco:params:xml:ns:yang:messagebus:eventaggregator"> <SampleEventSourceNotification> <Source>EventSourceSample01</Source><Message>Hello World [Mon Oct 05 14:08:55 BST 2015]</Message></SampleEventSourceNotification></payload> ', u'host_ip': u'0.0.0.0', u'src': u'EventSourceSample01'} Request: { "event-aggregator:input": { "notification-pattern": “notification-pattern": "*sample*", "node-id-pattern":”*00" } Response: { "output": { "topic-id": "9f4e18cd-06f0-4ad6-9d16-ad9a34eb4031" RESTful API Northbound ODL-Kafka Plugin Aggregated Events Compile yang model and register NB APIs from Models Data models (YANG) odl-message-bus SAL Service Registry Event Handling OpenDayLight Subscribe to a topic {pattern matching} SB APIs from Models Data models (YANG) <?xml version="1.0" encoding="UTF-8"?><payload xmlns="urn:cisco:params:xml:ns:yang:messagebus:eventaggregator"> <SampleEventSourceNotification> <Source>EventSourceSample01</Source><Message>Hello World [Mon Oct 05 14:08:55 BST 2015]</Message></SampleEventSourceNotification></payload> Compile yang model and register Events Southbound Plugin(example is hweventsource Network Devices Some Device
Example REST Configuration PUT: http://localhost:8181/restconf/config/kafkaproducer:kafka-producer-config HEADERS: application/yang.data+json BODY: { "kafka-producer-config": { "metadata-broker-list": "192.168.1.1:9092,192.168.1.2:9092", "topic": “test-topic”, "producer-type": "sync", "compression-codec": "none", "message-serialization": "raw", "event-subscriptions": "278206ca-746e-45b9-9846-a51dfbb086f5" }
Project Status Working Code Willing Committers Looking to update for Beryllium and Boron Examining next set of feature requirements Examining other input methods
Thank you