Consulenza, soluzioni e servizi per l’ICT ©Copyright 2013 Lutech Spa Matteo Redaelli Ankara C2Sense Meeting

Task D 3.2 Kafka quick introduction Demostration Structure Producer Web Service Consumer Web Service Producer Client Consumer Client Web Console Agenda

Task 3.2 Protocol Interoperability

This task’s objective is to achieve the protocol interoperability of emergency applications/systems and sensors using heterogeneous protocols. In this respect, “Protocol interoperability” addresses the transport level protocols such as TCP/IP, HTTP, SOAP, REST or SMTP and is in charge of end-to-end delivery of messages. C2-SENSE will use Web service protocols for this layer by exposing the proprietary services of emergency applications and organizations as “Web services”. In this task, first the detailed Operational and Core Web Service categories will be identified under the supervision of SAGEM. In this layer only the abstract definition of these services will be developed. Their interface schemas describing inputs/outputs in detail will be defined in Task 3.3 and the services will be implemented in Task 3.4). After that for these services related protocol interoperability profiles will be developed and based on these profiles corresponding Protocol Adapters will be implemented. These protocol adapters will be used to access emergency applications/systems and sensors in the C2-SENSE architecture through Web Service protocols. There will be two types of Protocol Adapters: (1) Sensor Protocol Adapters (to be developed by AIT) and (2) Emergency Applications/Systems Protocol Adapters (to be developed by SAGEM). Furthermore, the communication among the components and services of the C2-SENSE Architecture will be based on an Enterprise Service Bus (ESB), which will be developed by LUTECH. Task 3.2 Protocol Interoperability

Kafka Introduction

Kafka as ESB: –Fast  Could handle hundreds of messages from different sources. –Scalable  It has an architecture that could evolve and grow without losing time. –Durable  The possibility of losing messages is reduced. –Distributed by Design  It has possibility of retrieving messages in a second moment. Introduction – Connection with C2Sense

Kafka Architecture - Glossary (1) Producer = who publish messages Consumer = who read messages Broker = node Topic = message categories Partition = ordered, immutable sequence of messages

Kafka Architecture(2) Queue, in which each message is sent only to one reader, even if there are many Publish/Subscribe, in which message is sent to any reader interested in it

Producer Web Service

Producer Web Service: –Connector for sensors –Web Service is connected to the ESB Producer Web Service

Consumer Web Service

Consumer Web Service: –Connector for retrieving messages from the ESB –Web Service is connected to the ESB Consumer Web Service

Producer Client

–Very Simple Python script –Sensor like client Producer Client tstamptopicfromlatlnghydrolevel String Timestamp String “Sensor1” String From Water Sensor String latitude String longitudine Int Random number

Consumer Client

–Very Simple Python script –Retrieving messages from Web Service Consumer Client maxNumOfMsgsfromTopicfromPartition Int 100 String “Sensor1” Int 1

Web Console

–Topic –Brokers –Settings Could be a consumer Web Console

Dove siamo Direzione e Sede Operativa Via W.A. Mozart 47, Cologno Monzese (MI) Tel Fax Sede di Roma Via A. Mantegna 4, Parco Leonardo Fiumicino (RM) Tel Fax Altre sedi a Torino, Modena, Firenze, Bari Web