VerticaReplicator custom resource definition

The VerticaReplicator custom resource (CR) facilitates in-database replication through the Vertica Kubernetes operator. This feature allows you to create a VerticaReplicator CR to replicate databases for copying data, testing, or performing active online upgrade. It supports replication to and from sandbox environments. Additionally, both password-based authentication and source TLS authentication are supported.

Important

The version of the target database or sandbox must be the same as or higher than the version of the source database or sandbox.

The VerticaReplicator custom resource (CR) runs replicate on a VerticaDB CR, which copies table or schema data directly from one Eon Mode database's communal storage (source VerticaDB) to another (target VerticaDB).

Prerequisites

Deploy a source and target VerticaDB CR that uses VerticaDB API version v1 with vclusterops.
Deploy a VerticaDB operator version 2.0 and higher.

Create a VerticaReplicator CR

A VerticaReplicator CR spec only requires the names of the source and target VerticaDB CR for which you want to perform replication. The following example defines the CR as a YAML-formatted file named vreplicate-example.yaml:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vreplicator-example
spec:
  source:
    verticaDB: "vertica-src"
  target:
    verticaDB: "vertica-trg"

For a complete list of parameters that you can set for a VerticaReplicator CR, see Custom resource definition parameters.

Apply the manifest

After you create the VerticaReplicator CR, apply the manifest in the same namespace as the CR specified by verticaDBName:

$ kubectl apply -f vreplicator-example.yaml
verticareplicator.vertica.com/vreplicator-example created

The operator starts the replication process and copies the table and schema data from the source VerticaDB to the target VerticaDB.

You can check the applied CRs as follows:

$ kubectl get vrep
NAME                  SOURCEVERTICADB   TARGETVERTICADB   STATE         AGE
vreplicator-example   vertica-src       vertica-trg       Replicating   2s

Replicating to a sandboxed subcluster

You can replicate from a source db to a sandboxed subcluster.

Note

Replication is asynchronous by default and we recommend using this mode for optimal performance. If synchronous replication is required, set spec.mode to sync.
In asynchronous mode, polling frequency must be set to a value greater than 0.
In synchronous mode, the maximum concurrency limit is now 3. This means up to 3 synchronous replications can run simultaneously. Any additional replication request beyond this limit will wait until a running process completes.

The following example defines the CR as a YAML-formatted file named vreplicator-trg-sandbox.yaml:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vreplicator-trg-sandbox
spec:
  mode: async
  source:
    verticaDB: "vertica-src"
  target:
    verticaDB: "vertica-trg"
    sandboxName: "sandbox1"

After you apply the manifest, the operator will copy the table and schema data from the source VerticaDB to the sandboxed subcluster “sandbox1” on the target VerticaDB.

Partial Replication

You can replicate specific namespaces, schemas, or tables only in async mode. Here are some examples of partial replication.

Note

Exclude pattern, include pattern, object name, and target namespace cannot be used in synchronous mode. For more information about parameters and limitations, see Custom resource definition parameters.

The following example replicates a specific table mytable from a source cluster to a target cluster:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vrep-async-single-obj
spec:
  mode: async
  source:
    verticaDB: "vertica-src"
    objectName: "mytable"
  target:
    verticaDB: "vertica-trg"

The following example replicates all tables in the public schema that start with the letter t:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vrep-async-pattern1
spec:
  mode: async
  source:
    verticaDB: "vertica-src"
    includePattern: "public.t*"    
  target:
    verticaDB: "vertica-trg"

The following example replicates all tables in the public schema except those that start with the string customer_:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vrep-async-pattern2
spec:
  mode: async 
  source:
    verticaDB: "vertica-src"
    includePattern: "public.*"
    excludePattern: "*.customer_*"
  target:
    verticaDB: "vertica-trg"

Note

When using wildcard patterns, ensure they are prefixed with a period to clearly separate a schema from a table name or a namespace from a schema.

The following is an example that replicates the flights table in the airline schema of the airport namespace to the airport2 namespace in the target cluster:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vrep-async-namespace
spec:
  mode: async
  source:
    verticaDB: "vertica-src"
    objectName: ".airport.airline.flights"

  target:
    verticaDB: "vertica-trg"
    namespace: "airport2"

The following example replicates all schemas and tables in the default_namespace of the source cluster to the default_namespace in the target cluster:

apiVersion: vertica.com/v1beta1
kind: VerticaReplicator
metadata:
  name: vrep-pattern3
spec:
  mode: async
  source:
    verticaDB: "vertica-src"
    includePattern: ".default_namespace.*.*"    
  target:
    verticaDB: "vertica-trg"

Replication status

You can check the replication status as follows:

$ kubectl describe vrep
Name:         vreplicator-example
Namespace:    vertica
Labels:       <none>
Annotations:  <none>
API Version:  vertica.com/v1beta1
Kind:         VerticaReplicator
Metadata:
  Creation Timestamp:  2024-07-24T12:34:51Z
  Generation:          1
  Resource Version:    19058685
  UID:                 be90db7f-3ed5-49c0-9d86-94f87d681806
Spec:
  Source:
    Vertica DB:  vertica-src
  Target:
    Vertica DB:  vertica-trg
Status:
  Conditions:
    Last Transition Time:  2024-07-24T12:34:51Z
    Message:
    Reason:                Ready
    Status:                True
    Type:                  ReplicationReady
    Last Transition Time:  2024-07-24T12:35:01Z
    Message:
    Reason:                Succeeded
    Status:                False
    Type:                  Replicating
    Last Transition Time:  2024-07-24T12:35:01Z
    Message:
    Reason:                Succeeded
    Status:                True
    Type:                  ReplicationComplete
  State:                   Replication successful
Events:
  Type    Reason                Age    From                Message
  ----    ------                ----   ----                -------
  Normal  ReplicationStarted    4m3s   verticadb-operator  Starting replication
  Normal  ReplicationSucceeded  3m57s  verticadb-operator  Successfully replicated database in 5s

Conditions

The Conditions field summarizes each stage of the replication and contains the following fields:

Last Transition Time: Timestamp that indicates when the status condition last changed.
Message: This field is not in use, you can safely ignore it.
Reason: Indicates why the replication stage is in its current Status.
Status: Boolean, indicates whether the replication stage is currently in process.
Type: The replication that the VerticaDB operator is executing in this stage.

The following table describes each Conditions.Type, and all possible value combinations for its Reason and Status field values:

Type	Description	Status	Reason
`ReplicationReady`	The operator is ready to start the database replication.	`True`	`Ready`
`ReplicationReady`	The operator is ready to start the database replication.	`False`	Source database or sandbox is running a version earlier than 24.3.0. Target database or sandbox version is lower than the source database or sandbox version. Source database is deployed using admintools.
`Replicating`	The operator is replicating the database.	`True`	`Started`
`Replicating`	The operator is replicating the database.	`False`	`Failed` `Succeeded`
`ReplicationComplete`	The database replication is complete.	`True`	`Succeeded`