Creating a Kafka notifier

The following procedure creates a Kafka notifier. At a minimum, a notifier defines:

• A unique name.

• A message protocol. This is kafka:// when sending messages to Kafka.

• The server to communicate with. For Kafka, this is the address and port number of a Kafka broker.

• The maximum message buffer size. If the queue of messages to be sent via the notifier exceed this limit, messages are dropped.

You create the notifier with CREATE NOTIFIER. This example creates a notifier named load_progress_notifier that sends messages via the Kafka broker running on kafka01.example.com on port 9092:

=> CREATE NOTIFIER load_progress_notifier
    ACTION 'kafka://kafka01.example.com:9092'
    MAXMEMORYSIZE '10M';

While not required, it's best practice to create notifiers that use an encrypted connection. The following example creates a notifier that uses an encrypted connection and verifies the Kafka server's certificate with the provided CA bundle:

=> CREATE NOTIFIER encrypted_notifier
    ACTION 'kafka://127.0.0.1:9092'
    MAXMEMORYSIZE '10M'
    TLSMODE 'verify-ca'
    CA BUNDLE ca_bundle;

Follow this procedure to create or alter notifiers for Kafka endpoints that use SASL_SSL. Note that you must repeat this procedure whenever you change the TLSMODE, certificates, or CA bundle for a given notifier.

1. Use CREATE or ALTER to disable the notifier while setting the TLSMODE, certificate, and CA bundle.

   => ALTER NOTIFIER encrypted_notifier
       DISABLE
       TLSMODE 'verify-ca'
       CA BUNDLE ca_bundle2;
   

2. ALTER the notifier and set the proper rdkafka adapter parameters for SASL_SSL.

   => ALTER NOTIFIER encrypted_notifier PARAMETERS
     'sasl.username=user;sasl.password=password;sasl.mechanism=PLAIN;security.protocol=SASL_SSL';
   

3. Enable the notifier.

   => ALTER NOTIFIER encrypted_notifier ENABLE;