Vertica Documentation – New and changed in Vertica 12.0.4

New-Features: admintools

Mon, 01 Jan 0001 00:00:00 +0000

Setting re_ip timeout

You can configure how long re_ip executes a given task before it times out, by editing the setting of prepare_timeout_sec in admintools.conf. By default, this parameter is set to 7200 (seconds).

New-Features: Client connectivity

Mon, 01 Jan 0001 00:00:00 +0000

Set idle timeouts during connection initialization

You can set the IdleTimeoutInitializingConnectionsMs configuration parameter to change the length of time that the server waits for an idle client during each step in connection initialization, before a session is created.

Client hostnames

The hostnames of clients (as reported by their operating systems) are now recorded in the CLIENT_OS_HOSTNAME column of the following tables:

New-Features: Client drivers

Mon, 01 Jan 0001 00:00:00 +0000

ODBC and vsql: support for M1 and M2 processors

The ODBC and vsql client drivers now support Apple M1 and M2 processors. For details, see Client drivers support.

New-Features: Complex types

Mon, 01 Jan 0001 00:00:00 +0000

UNNEST function

The new UNNEST function expands one or more array columns. UNNEST is similar to EXPLODE, with two key differences:

UNNEST returns element values but not positions.
UNNEST automatically expands all arrays that are passed to it.

EXPLODE performance optimization

The EXPLODE function translates a single row of input into multiple rows of output, one per collection element. There is, therefore, usually no benefit to partitioning the input. Previously, a query using EXPLODE required an OVER clause and Vertica partitioned the input before calling EXPLODE. Now, you can skip this partitioning step by using the skip_partitioning function parameter and omitting the OVER clause.

CROSS JOIN EXPLODE

You can use the output of EXPLODE as if it were a relation by using CROSS JOIN EXPLODE in a query, as in the following example:

=> ALTER SESSION SET UDPARAMETER FOR ComplexTypesLib skip_partitioning = true;

=> SELECT student, score FROM tests
   CROSS JOIN EXPLODE(scores) AS t (pos, score)
   ORDER BY score DESC;
 student | score
---------+-------
 Sam     |    98
 Sam     |    97
 Bob     |    92
 Tom     |    91
 Sam     |    85
 Tom     |    82
 Bob     |    79
 Bob     |    78
 Tom     |    75
 Tom     |    68
(10 rows)

For details, see Joining on Results.

New-Features: Database management

Mon, 01 Jan 0001 00:00:00 +0000

You can now send messages to SNS topics with notifiers. For details, see Configuring reporting for the simple notification service (SNS).

New-Features: Directed queries

Mon, 01 Jan 0001 00:00:00 +0000

INSERT...SELECT hints

You can now annotate the query of an INSERT...SELECT statement with directed query hints.

New-Features: Documentation updates

Mon, 01 Jan 0001 00:00:00 +0000

Data exploration

When data is not well-specified in a schema, developing a table definition so you can load the data can be challenging. Vertica provides tools you can use to explore data and refine a table definition. These tools and processes are now described in a new top-level section of the documentation, Data exploration.

New-Features: Eon Mode

Mon, 01 Jan 0001 00:00:00 +0000

Subcluster sandboxing

Sandboxing enables you to spin-off a secondary subcluster from an existing cluster, resulting in two mutually isolated clusters that share the same data but do not interfere with each other. After sandbox creation, you can perform standard database operations and queries, such as creating new tables or loading libraries, without affecting the other cluster. For example, dropping a table in the sandboxed subcluster does not drop the table in the main cluster, and vice versa.

Sandboxes support many use cases, including the following:

Testing a new version of Vertica without needing to spin-up a new cluster and reload data.
Trying out features without compromising the consistency of the main cluster.
Sharing data with another team by giving them access to a sandboxed subcluster.

After removing the sandbox and performing necessary cleanup tasks, the subcluster can rejoin the main cluster.

New-Features: Apache Kafka integration

Mon, 01 Jan 0001 00:00:00 +0000

kafka_conf_secret parameter

The kafka_conf_secret parameter conceals sensitive data that you must pass to rdkafka to configure the Vertica and Kafka integration. Vertica does not log or store values that you pass in kafka_conf_secret.

This option is available on tools and functions that use the kafka_conf parameter.

For details, see Directly setting Kafka library options and Apache Kafka integration.

New-Features: Loading data

Mon, 01 Jan 0001 00:00:00 +0000

Optimization for external tables on object stores

When an external table uses data that is stored on an object store (S3, GCS, or Azure), and that data has many levels of partitioning, performance can be affected. The ObjectStoreGlobStrategy configuration parameter allows you to read and prune partitions using a different strategy, improving performance when queries are selective and there are many partition directory levels. With this setting, fewer calls are made to the object store API. For details, see Partitions on Object Stores.

Iceberg support

You can now define external tables backed by Parquet data stored by Apache Iceberg. An Iceberg table consists of data files and metadata describing the schema. Unlike other external tables, an Iceberg external table need not specify column definitions (DDL). The information is read from Iceberg metadata at query time. For certain data types you can adjust column definitions, for example to specify VARCHAR sizes. See CREATE EXTERNAL TABLE ICEBERG.

New-Features: Machine learning

Mon, 01 Jan 0001 00:00:00 +0000

Model versioning

Model versioning provides an infrastructure to track and manage the status of registered models in a database. The versioning infrastructure supports a collaborative environment where multiple users can submit candidate models for individual applications. dbadmin and users with the MLSUPERVISOR role can manage and change the status of all registered models, including which models are currently in production.

Vertica provides the following model versioning functionality:

REGISTER_MODEL: Registers a model and adds it to the versioning environment.
CHANGE_MODEL_STATUS: Changes the status of a registered model.
REGISTERED_MODELS: Lists registered models in the database.
MODEL_STATUS_HISTORY: Tracks the status history of registered models.

For details about the model versioning environment and an in-depth example, see Model versioning.

Poisson regression

Vertica now supports the Poisson regression algorithm to model count data. Poisson regression offers an alternative to linear regression or logistic regression and is useful when your desired prediction range is non-negative real numbers or integers.

Autoregresssive integrated moving average (ARIMA) support

Vertica now supports autoregressive integrated moving average (ARIMA) models for time series analytics. ARIMA models combine the abilities of AUTOREGRESSOR and MOVING_AVERAGE models by making future predictions based on both preceding time series values and errors of previous predictions. You can use the ARIMA function to create and train an ARIMA model and the PREDICT_ARIMA function to make predictions.

For an in-depth example that trains and makes predictions with an ARIMA model, see ARIMA model example.

New-Features: Partitioning

Mon, 01 Jan 0001 00:00:00 +0000

Window partitioning for 1:N transform functions using PARTITION ROW

Previously, window partitioning could only be used for 1:1 analytic functions. Now, the PARTITION ROW syntax can be used for window partitioning 1:N transform functions.

New-Features: Performance improvements

Mon, 01 Jan 0001 00:00:00 +0000

Refreshing partitioned projections

In previous releases, new and updated projections were not available to queries before they were completely refreshed with all table data. This delayed availability of projections to process query requests.

Now, when you refresh the projections of a partitioned table—for example, a new projection of a table with existing data—the refresh operation first loads data from the partition with the highest range of keys. After this partition is refreshed, Vertica begins to refresh the partition with next highest partition range. This process continues until all projection partitions are refreshed. In the meantime, while the refresh operation is in progress, projection partitions that have completed the refresh process become available to process query requests.

New-Features: Resource management

Mon, 01 Jan 0001 00:00:00 +0000

Default user resource pool

The new configuration parameter DefaultResourcePoolForUsers lets you configure the default user resource pool that is assigned to new users, including LDAP users. For details, see User resource allocation.

New-Features: SDK updates

Mon, 01 Jan 0001 00:00:00 +0000

One-to-many UDTFs and PARTITION ROW

Single-phase user-defined transform functions (UDTFs) written in C++ and Python now support the isExploder class property, which indicates whether the UDTF performs a one-to-many transform. If this property is set to True within the getTransformFunctionProperties factory method, the function uses an OVER(PARTITION ROW) clause by default and thereby removes the need to specify an OVER clause when calling the UDTF. Users can also include any expression in SELECT queries that call a one-to-many UDTF.

For more information about this class property and one-to-many UDTFs, see Partitioning options for UDTFs and TransformFunctionFactory class. See Python example: explode for an in-depth example detailing a one-to-many UDTF.

New-Features: Security and authentication

Mon, 01 Jan 0001 00:00:00 +0000

OAuth

JSON web token validation (keycloak only)

Vertica can now validate OAuth tokens without contacting the identity provider by verifying the signer of the OAuth token. To enable this feature, create an authentication record that uses 'oauth' and set the authentication parameters to use JWT validation. For details, see Configuring OAuth authentication.

Just-in-time user provisioning (keycloak only)

Just-in-time (JIT) user provisioning is the act of automatically configuring an authenticated user and their roles based on information provided by the identity provider. When a client authenticates to Vertica with an OAuth token the authentication record enables JIT user provisioning, Vertica automatically creates the user (if the user does not already exist in Vertica) and granting to them the roles specified by the identity provider.

For example, if a client presents an OAuth token to authenticate as user Alice with role director, and Alice does not exist in Vertica, Vertica automatically creates the user Alice, grants to her the authentication record, and grants to her the director role as a default role.

For details, see Just-in-time user provisioning

JDBC: custom truststore

You can use a custom truststore to secure the connection between the JDBC client and the identity provider. This is useful if your identity provider's certificate is not signed by a well-known certificate authority.

The path to the truststore and its password are specified with the following connection properties:

oauthtruststorepath
oauthtruststorepassword

If oauthtruststorepath is not specified, the JDBC client uses the default truststore.

For details on this and other connection properties, see JDBC connection properties. For details on configuring OAuth, see Configuring OAuth authentication.

New-Features: Stored procedures

Mon, 01 Jan 0001 00:00:00 +0000

Scheduled execution

You can now schedule the automatic execution of stored procedures. This can be used to automate various tasks, like logging database activity, revoking privileges, or creating roles. For details, see Scheduled execution.

Viewing overloaded stored procedures

You can now view the source code of a particular implementation of an overloaded stored procedure with EXPORT_OBJECTS. For details, see Exporting objects.

New-Features: User-defined extensions

Mon, 01 Jan 0001 00:00:00 +0000

Redactable user-defined extension and session parameters

User-defined extension and session parameters that meet any of the following criteria are automatically redacted from logs and system tables like QUERY_REQUESTS:

Named "secret" or "password"
End with "_secret" or "_password"

Vertica Documentation – New and changed in Vertica 12.0.4

New-Features: admintools

Setting re_ip timeout

New-Features: Client connectivity

Set idle timeouts during connection initialization

Client hostnames

New-Features: Client drivers

ODBC and vsql: support for M1 and M2 processors

New-Features: Complex types

UNNEST function

EXPLODE performance optimization

CROSS JOIN EXPLODE

New-Features: Database management

Amazon simple notification service (SNS) notifiers

New-Features: Directed queries

INSERT...SELECT hints

New-Features: Documentation updates

Data exploration

New-Features: Eon Mode

Subcluster sandboxing

New-Features: Apache Kafka integration

kafka_conf_secret parameter

New-Features: Loading data

Optimization for external tables on object stores

Iceberg support

New-Features: Machine learning

Model versioning

Poisson regression

Autoregresssive integrated moving average (ARIMA) support

New-Features: Partitioning

Window partitioning for 1:N transform functions using PARTITION ROW

New-Features: Performance improvements

Refreshing partitioned projections

New-Features: Resource management

Default user resource pool

New-Features: SDK updates

One-to-many UDTFs and PARTITION ROW

New-Features: Security and authentication

OAuth

JSON web token validation (keycloak only)

Just-in-time user provisioning (keycloak only)

JDBC: custom truststore

New-Features: Stored procedures

Scheduled execution

Viewing overloaded stored procedures

New-Features: User-defined extensions

Redactable user-defined extension and session parameters