1. $ whoami System administrator/developer for about 18 years
Currently technical lead/cto of todocoleccion.net for 7 years.
Organizer of meetups and a conference
Málaga:
Collectible products

2. Agenda What is Apache Kafka Differences from others systems
Use in CFML
Use cases
Whatis: a introduction to apache kafka, the basic from the point of web developers. From outside to inside.
The mains difference of concept from another systems
How we use in production
How can be used in cfml, basically from Lucee

3. What is Apache Kafka - Distributed → Cluster - Streaming → 100k msg/s
Distributed streaming platform
publish/subscriber system
Basically: send messages to a system that store them and some other process consume that messages
What it’s a message? Could be anything, tipically: just a JSON. Can have a key.
- Distributed → Cluster
- Streaming → 100k msg/s
- platform → has different APIs
Fast, scalable.

4. What is Apache Kafka (II)
Basic concepts:
Topic: Category of messages
Producer: Sends messages to topics
Consumer: Subscribe to a topic to read the messages in that topic
Consumer Group: Group of consumers that process msg from the same topics

5. TOPICS
Kafka cluster has topics where you can send/store and consume messages

6. TOPICS and PRODUCERS

7. Topics – Producers – Consumers Groups
The consummers subscribe to topics and poll data from them.

8. Topics – Producers – Consumers Groups

9. INSIDE A TOPIC / CONSUMER GROUP
NEXT: Scaling consumers

10. SCALING THE CONSUMERS

11. SCALING KAFKA

12. SCALING KAFKA

13. Inside a topic partition - Offset

14. ORDER WITHOUT KEY

15. ORDER WITH KEY

16. Differences from others systems
Kafka does not delete the consumed messages, so you can replay the messages
Different clients can read from the same topic
The clients can say from where to read
New consumers are very easy to deploy

17. Others features
KStreams
Kafka Connect
KSQL

18. Use in CFML (Lucee) Add kafka-clients.jar

19. Use in CFML - Producer
props = createobject('java', 'java.util.Properties');
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG," :9092");
producer = createobject('java', 'org.apache.kafka.clients.producer.KafkaProducer').init(props);
value = serializeJson(message)
producerrecord = createobject('java', 'org.apache.kafka.clients.producer.ProducerRecord').init(“mytopic”, value);
producer.send(produceRecord);

20. Use in CFML - Consumer
props = createobject('java', 'java.util.Properties');
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG," :9092");
consumer = createobject('java', 'org.apache.kafka.clients.consumer.KafkaConsumer').init(props);
consumer.subscribe([“mytopic”]);
while (true) {
records = consumer.poll(100);
for ( record in records.iterator()) {
event = record.value(); }
NEXT: How we use it

21. How we use it Topics as events, topics as actions
Offload heavy and statistical task from frontend
Data bus

22. How we use it Web panel with consumers groups
Scheduler task to launch the number of consumers configurate
FUTURE:
Logs
Web activity tracking
Panel: Launch & stop
Setup number of workers.

23. Example: Migration of Elasticsearch

24. Example: Migration of Elasticsearch

25. Example: Migration of Elasticsearch

26. LINKS https://github.com/davidsf/cfml-kafka
Coldbox app for conf and launch consumers: kafka-panel