Authenticating with Kafka using SASL

Kafka uses Simple Authentication and Security Layer (SASL) to authenticate producers and consumers. You can use SASL to authenticate OpenText™ Analytics Database with Kafka when using most of the Kafka-related functions such as KafkaSource.

The database uses the SASL_PLAINTEXT and SASL_SSL protocols with the following authentication mechanisms:

• PLAIN

• SCRAM-SHA-256

• SCRAM-SHA-512

You must configure your Kafka cluster to enable SASL authentication. See the Kafka documentation for your Kafka version to learn how to configure SASL authentication.

To use SASL authentication between the database and Kafka, directly set SASL-related configuration options in the rdkafka library using the kafka_conf parameter. The database uses this library to connect to Kafka. For more information about directly setting configuration options in the rdkafka library, see Directly setting Kafka library options.

Among the relevant configuration options are:

• security.protocol sets the security protocol to use to authenticate with Kafka.

• sasl.mechanism sets the security mechanism.

• sasl.username sets the SASL user to use for authentication.

• sasl.password sets the password to use for SASL authentication.

See the rdkafka configuration documentation for a list of all the SASL-related settings.

The following example demonstrates calling KafkaCheckBrokers using the SASL_PLAINTEXT security protocol:

=> SELECT KafkaCheckBrokers(USING PARAMETERS
      brokers='kafka01.example.com:9092',
      kafka_conf='{"sasl.username":"dbadmin", "sasl.mechanism":"PLAIN", "security.protocol":"SASL_PLAINTEXT"}',
      kafka_conf_secret='{"sasl.password":"password"}'
   ) OVER ();

This example demonstrates using SASL authentication when copying data from Kafka via an SSL connection. This example assumes that the database and Kafka have been configured to use TLS/SSL encryption as described in TLS/SSL encryption with Kafka:

=> COPY mytopic_table
      SOURCE KafkaSource(
        stream='mytopic|0|-2',
        brokers='kafka01.example.com:9092',
        stop_on_eof=true,
        kafka_conf='{"sasl.username":"dbadmin", "sasl.password":"pword", "sasl.mechanism":"PLAIN", "security.protocol":"SASL_SSL"}'
      )
      FILTER KafkaInsertDelimiters(delimiter = E'\n')
      DELIMITER ','
      ENCLOSED BY '"';

For more information about using SASL with the rfkafka library, see Using SASL with librdkafka on the rdkafka github site.