Apache Spark integrations

You can use the OpenText™ Analytics Database Connector for Apache Spark to transfer data between the database and Apache Spark.

You can use the OpenText™ Analytics Database Connector for Apache Spark to transfer data between the database and Apache Spark. The following table shows the versions Apache Spark and Scala the Connector supports as well as the name of the Spark Connector JAR file to use for each combination:

Apache Spark Version	Scala Version	Spark Connector JAR file
2.0*	2.11	`vertica-spark2.0_scala2.11.jar`
2.1*	2.11	`vertica-spark2.1_scala2.11.jar`
2.2	2.11	`vertica-spark2.1_scala2.11.jar`
2.3	2.11	`vertica-spark2.1_scala2.11.jar`
2.4.1	2.11	`vertica-spark2.1_scala2.11.jar`
2.4.1	2.12	`vertica-spark2.4-3.0_scala2.12.jar`
3.0	2.12	`vertica-spark2.4-3.0_scala2.12.jar`

* OpenText is phasing out this Apache Spark version. For more information, see End-of-support notices.

Notes

A Spark Connector JAR file can support multiple versions of Spark. For example, vertica-spark2.1_scala2.11.jar supports Spark 2.1, 2.2, 2.3, and 2.4.1.
It is recommended that you always use the version of the Spark Connector shipped with your version of the database server. When you upgrade your database server, you should also upgrade your version of the Spark Connector.

For more information about Apache Spark integration, see Apache Spark integration.