TLS/SSL encryption with Kafka

You can use TLS/SSL encryption between OpenText™ Analytics Database, your scheduler, and Kakfa. This encryption prevents others from accessing the data that is sent between Kafka and the database. It can also verify the identity of all parties involved in data streaming, so no impostor can pose as your database cluster or a Kafka broker.

Some common cases where you want to use SSL encryption between the database and Kafka are:

• Your database and Kafka communicate over an insecure network. For example, if your Kafka cluster is located in a cloud service and your database cluster is within your internal network. In this case, any data you read from Kafka travels over an insecure connection across the Internet.

• You are required by security policies, laws, or other requirements to encrypt all of your network traffic.

For more information about TLS/SSL encryption in the database, see TLS protocol.

Using TLS/SSL between the scheduler and OpenText™ Analytics Database

The scheduler connects to the database the same way other client applications do. There are two ways you can configure the database to use SSL/TLS authentication and encryption with clients:

• If the database is configured to use SSL/TLS server authentication, you can choose to have your scheduler confirm the identity of the database server.

• If the database is configured to use mutual SSL/TLS authentication, you can configure your scheduler to identify itself to the database as well as have it verify the identity of the database server. Depending on your database's configuration, the database server may require your scheduler to use TLS when connecting. For more information, see Client authentication with TLS.

For information about encrypted client connections with the database, refer to TLS protocol.

The scheduler runs on a Java Virtual Machine (JVM) and uses JDBC to connect to the database. It acts like any other JDBC client when connecting to the database. To use TLS/SSL encryption for the scheduler's connection to the database, use the Java keystore and truststore mechanism to hold the keys and certificates the scheduler uses to identify itself and the database.

• The keystore contains your scheduler's private encryption key and its certificate (public key).

• The truststore contains CAs that you trust. If you enable authentication, the scheduler uses these CAs to verify the identity of the database cluster to which it connects. If one of the CAs in the trust store was used to sign the server's certificate, then the Scheduler knows it can trust the identity of the database server.

You can pass options to the JVM that executes the scheduler through the Linux environment variable named VKCONFIG_JVM_OPTS. You add the parameters to this variable that alter the scheduler's JDBC settings (such as the truststore and keystore for the scheduler's JDBC connection). See Step 2: Set the VKCONFIG_JVM_OPTS Environment Variable for an example.

You can also use the --jdbc-url scheduler option to alter the JDBC configuration. See Common vkconfig script options for more information about the scheduler options and JDBC connection properties for more information about the properties they can alter.

Using TLS/SSL between OpenText™ Analytics Database and Kafka

You can stream data from Kafka into the database two ways: manually using a COPY statement and the KafkaSource UD source function, or automatically using the scheduler.

To directly copy data from Kafka via an SSL connection, you set session variables containing an SSL key and certificate. When KafkaSource finds that you have set these variables, it uses the key and certificate to create a secure connection to Kafka. See Kafka TLS/SSL Example Part 4: Loading Data Directly From Kafka for details.

When automatically streaming data from Kafka to the database, you configure the scheduler the same way you do to use an SSL connection to the database. When the scheduler executes COPY statements to load data from Kafka, it uses its own keystore and truststore to create an SSL connection to Kafka.

To use an SSL connection when producing data from the database to Kafka, you set the same session variables you use when directly streaming data from Kafka via an SSL connection. The KafkaExport function uses these variables to establish a secure connection to Kafka.

See the Apache Kafka documentation for more information about using SSL/TLS authentication with Kafka.