Bring your favorite Kafka to Java EE with CDI Ivan St. Ivanov

Who am I? @ivan_stefanov

Agenda Kafka 10...1 CDI@Kafka Kafka portable

Showcase app User service Forum service Kafka User DB Forum DB

CDI primer @ApplicationScoped public class UserModificationObserver {      private KafkaProducer<String, String> kafkaProducer;        public void userModified(User modifiedUser) {         String record = modifiedUser.toJson().toString();         kafkaProducer.send(new ProducerRecord<>("user",                            record));     } }

CDI primer @ApplicationScoped public class UserModificationObserver {      @Inject      private KafkaProducer<String, String> kafkaProducer;        public void userModified(User modifiedUser) {         String record = modifiedUser.toJson().toString();         kafkaProducer.send(new ProducerRecord<>("user",                            record));     } }

CDI primer @ApplicationScoped public class UserModificationObserver {      @Inject      @KafkaProducerConfig(bootstrapServers = "mykafkahost:9090")      private KafkaProducer<String, String> kafkaProducer;        public void userModified(User modifiedUser) {         String record = modifiedUser.toJson().toString();         kafkaProducer.send(new ProducerRecord<>("user",                            record));     } }

CDI primer public @interface KafkaProducerConfig {          Class keyType() default String.class;         Class valueType() default String.class;     String bootstrapServers() default "localhost:9092"; }

CDI primer @ApplicationScoped public class KafkaProducerFactory {     @Produces     public <K,V> KafkaProducer<K, V> createProducer(            InjectionPoint injectionPoint) {         KafkaProducerConfig annotation = injectionPoint             .getAnnotated()             .getAnnotation(KafkaProducerConfig.class);         Properties producerProperties = new Properties();         producerProperties.setProperty(           "bootstrap.servers", annotation.bootstrapServers());         // Set other props         return new KafkaProducer<>(producerProperties);     } }

CDI primer @ApplicationScoped public class UserModificationObserver {      @Inject      @KafkaProducerConfig(bootstrapServers = "mykafkahost:9090")      private KafkaProducer<String, String> kafkaProducer;        public void userModified(@Observes User modifiedUser) {         String record = modifiedUser.toJson().toString();         kafkaProducer.send(new ProducerRecord<>("user",                            record));     } }

CDI primer public class KafkaUpdatesListener implements Runnable {     @Inject     @KafkaConsumerConfig(subscriptions = "user")     private KafkaConsumer<String, String> consumer;     @Override     public void run() {         while (continuePolling) {             ConsumerRecords<String, String> records =                consumer.poll(100);             if (!records.isEmpty()) {                 // Update microservice storage             }         }     }

But what if... Leave factories boilerplate to the framework Leave polling updates to the framework Do it this way: public class UserModificationKafkaConsumer {     @Consumes(topic = "user")     public void listenForUsers(                ConsumerRecord<String, String> record) {         String userJson = record.value();         // Store the user in DB }

CDI extensions CDI magic Hook into the lifecycle Override default behavior Portable across containers

CDI extension prerequisites Implement javax.enterprise.inject.spi.Extension Observe events of interest Register service provider

Process Annotated Types Before Bean Discovery Type discovery Process Annotated Types After Type Discovery

Type discovery Bean discovery Before Bean Discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation

Type discovery Bean discovery Running Before Bean Discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Kafka CDI extension tasks Register the factories and listener thread Create Kafka Consumer for all @Consumes methods Start polling consumers for changes Invoke @Consumes methods when change happens

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types public void addKafkaSpecificTypes(     @Observes BeforeBeanDiscovery bbd,               BeanManager beanManager) {     bbd.addAnnotatedType(         beanManager.createAnnotatedType(KafkaProducerFactory.class));     bbd.addAnnotatedType(         beanManager.createAnnotatedType(KafkaUpdatesListener.class)); } After Type Discovery Bean discovery Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point public void collectConsumerMethods(     @Observes @WithAnnotations(Consumes.class)     ProcessAnnotatedType<?> pat) {         pat.getAnnotatedType().getMethods().stream()         .filter(m -> m.isAnnotationPresent(Consumes.class))         .forEach(m -> handleConsumerMethod(pat.getAnnotatedType(), m)); } Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point private void handleConsumerMethod(AnnotatedType<?> clazz,                       AnnotatedMethod<?> am) {          List<AnnotatedParameter<?>> parameters = am.getParameters();     if (parameters.size() != 1) {         errors.add(new IllegalArgumentException(                 "@Consume methods should only have one parameter"));     } else {         consumerDescriptors.add(new KafkaConsumerDescriptor(                clazz.getJavaClass(), am,                am.getAnnotation(Consumes.class).topic()));     } } Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery public void startMonitoring(@Observes AfterDeploymentValidation adv,                            BeanManager bm) {     if (!errors.isEmpty()) {         errors.forEach(adv::addDeploymentProblem);     } else {         KafkaUpdatesListener kafkaUpdatesListener = initializeListener(bm);         startListener(kafkaUpdatesListener, adv);         this.kafkaUpdatesListener = kafkaUpdatesListener;     } } Bean discovery Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point Process Injection Target Process Bean Attributes private KafkaUpdatesListener initializeListener(BeanManager bm) {     Class<KafkaUpdatesListener> clazz = KafkaUpdatesListener.class;     KafkaUpdatesListener reference = createBean(bm, clazz);     consumerDescriptors.forEach(m -> addTopicToListen(m, reference));     return reference; } Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point Process Injection Target static Object createBean(BeanManager bm, Class<?> clazz) {     Set<Bean<?>> listenerBeans = bm.getBeans(clazz);     Bean<?> listenerBean = bm.resolve(listenerBeans);     CreationalContext<?> creationalContext =                   bm.createCreationalContext(listenerBean);     return bm.getReference(listenerBean, clazz, creationalContext); } Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery private void startListener(KafkaUpdatesListener kafkaUpdatesListener) {     try {         ExecutorService executorService =          InitialContext.doLookup("java:comp/DefaultManagedExecutorService");               executorService.submit(kafkaUpdatesListener);     } catch (NamingException e) {         // Handle NamingException     } } Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Type discovery Bean discovery Running Before shutdown Before Bean Discovery Type discovery Process Annotated Types After Type Discovery Bean discovery Process Injection Point Process Injection Target Process Bean Attributes Process Bean Process Producer Method / Field Process Observer Method public void stopListener(@Observes BeforeShutdown bsh) {     kafkaUpdatesListener.stopPolling(); } After Bean Discovery After Deployment Validation Running Before shutdown Before shutdown

Wrap up CDI is powerful Extensions are portable New features and improvements to add Create extensions for other libraries Evangelize

Resources The extension https://github.com/bgjug/kafka-cdi-extension Showcase app https://github.com/ivannov/kafka  Article by Antoine http://www.next-presso.com/2017/02/nobody-expects-the-cdi-portable- extensions/ Another Kafka CDI extension https://github.com/aerogear/kafka-cdi