This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Subclusters on Kubernetes

Eon Mode uses subclusters for workload isolation and scaling.

Eon Mode uses subclusters for workload isolation and scaling. The Vertica operator provides tools to direct external client communications to specific subclusters, and automate scaling without stopping your database.

The custom resource definition (CRD) provides parameters that allow you to fine-tune each subcluster for specific workloads. For example, you can increase the subcluster size setting for increased throughput, or adjust the resource requests and limits to manage compute power. When you create a custom resource instance, the operator deploys each subcluster as a StatefulSet. Each StatefulSet has a service object, which allows an external client to connect to a specific subcluster.

Naming conventions

Kubernetes derives names for the subcluster Statefulset, service object, and pod from the subcluster name. This naming convention tightly couples the subcluster objects to help Kubernetes manage the cluster effectively. If you want to rename a subcluster, you must delete it from the CRD and redefine it so that the operator can create new objects with a derived name.

Kubernetes forms an object's fully qualified domain name (FQDN) with its resource type name, so resource type names must follow FQDN naming conventions. The underscore character ( "_" ) does not follow FQDN rules, but you can use it in the subcluster name. Vertica converts each underscore to a hyphen ( "-" ) in the FQDN for any object name derived from the subcluster name. For example, Vertica generates a default subcluster and names it default_subcluster, and then converts the corresponding portion of the derived object's FQDN to default-subcluster.

For additional naming guidelines, see the Kubernetes documentation.

External client connections

External clients can target specific subclusters that are fine-tuned to handle their workload. Each subcluster has a service object that handles external connections. To target multiple subclusters with a single service object, assign each subcluster the same spec.subclusters.serviceName value in the custom resource (CR). For implementation details, see VerticaDB CRD.

The operator performs health monitoring that checks if the Vertica daemon is running on each pod. If it is, then the operator allows the service object to route traffic to the pod.

By default, the service object derives its name from the custom resource name and the associated subcluster and uses the customResourceName-subclusterName format. Use the subclusters[i].serviceName CR parameter to override the default naming format and use the metadata.name-serviceName format.

Vertica supports the following service object types:

  • ClusterIP: The default service type. This service provides internal load balancing, and sets a stable IP and port that is accessible from within the subcluster only.

  • NodePort: Provides external client access. You can specify a port number for each host node in the subcluster to open for client connections.

  • LoadBalancer: Uses a cloud provider load balancer to create NodePort and ClusterIP services as needed. For details about implementation, see the Kubernetes documentation and your cloud provider documentation.

For configuration details, see VerticaDB CRD.

Managing internal and external workloads

The Vertica StatefulSet is associated with an external service object. All external client requests are sent through this service object and load balanced among the pods in the cluster.

Import and export

Importing and exporting data between a cluster outside of Kubernetes requires that you expose the service with the NodePort or LoadBalancer service type and properly configure the network.

1 - Scaling subclusters

The operator enables you to scale the number of subclusters, and the number of pods per subcluster automatically.

The operator enables you to scale the number of subclusters, and the number of pods per subcluster automatically. This allows you to utilize or conserve resources depending on the immediate needs of your workload.

The following sections explain how to scale resources for new workloads. For details about scaling resources for existing workloads, see VerticaAutoscaler CRD.

Prerequisites

  • Complete Installing the Vertica DB operator.

  • Install the kubectl command line tool.

  • Complete VerticaDB CRD.

  • Confirm that you have the resources to scale.

Scaling the number of subclusters

Adjust the number of subclusters in your custom resource to fine-tune resources for short-running dashboard queries. For example, increase the number of subclusters to increase throughput. For more information, see Improving query throughput using subclusters.

  1. Use kubectl edit to open your default text editor and update the YAML file for the specified custom resource. The following command opens a custom resource named vdb for editing:

    $ kubectl edit vdb

  2. In the spec section of the custom resource, locate the subclusters subsection. Begin the isPrimary field to define a new subcluster.

    The isPrimary field accepts a boolean that specifies whether the subcluster is a primary or secondary. Because there is already a primary subcluster in our custom resource, enter false:

    spec:
...
  subclusters:
  ...
  - isPrimary: false

  3. Follow the steps in VerticaDB CRD to complete the subcluster definition. The following completed example adds a secondary subcluster for dashboard queries:

    spec:
...
  subclusters:
  - isPrimary: true
    name: primary-subcluster
  ...
  - isPrimary: false
    name: dashboard
    nodePort: 32001
    resources:
      limits:
        cpu: 32
        memory: 96Gi
      requests:
        cpu: 32
        memory: 96Gi
    serviceType: NodePort
    size: 3

  4. Save and close the custom resource file. You receive a message similar to the following when you successfully update the file:

    verticadb.vertica.com/vertica-db edited

  5. Use the kubectl wait command to monitor when the new pods are ready:

    $ kubectl wait --for=condition=Ready pod --selector app.kubernetes.io/name=verticadb --timeout 180s
pod/vdb-dashboard-0 condition met
pod/vdb-dashboard-1 condition met
pod/vdb-dashboard-2 condition met

Scaling the pods in a subcluster

For long-running, analytic queries, increase the pod count for a subcluster. See Using elastic crunch scaling to improve query performance.

  1. Use kubectl edit to open your default text editor and update the YAML file for the specified custom resource. The following command opens a custom resource named vdb for editing:

    $ kubectl edit verticadb

  2. Update the subclusters.size value to 6:

    spec:
...
  subclusters:
  ...
  - isPrimary: false
    ...
    size: 6

    Shards are rebalanced automatically.

  3. Save and close the custom resource file. You receive a message similar to the following when you successfully update the file:

    verticadb.vertica.com/verticadb edited

  4. Use the kubectl wait command to monitor when the new pods are ready:

    $ kubectl wait --for=condition=Ready pod --selector app.kubernetes.io/name=verticadb --timeout 180s
pod/vdb-subcluster1-3 condition met
pod/vdb-subcluster1-4 condition met
pod/vdb-subcluster1-5 condition met

Removing a subcluster

Remove a subcluster when it is no longer needed, or to preserve resources.

  1. Use kubectl edit to open your default text editor and update the YAML file for the specified custom resource. The following command opens a custom resource named vdb for editing:

    $ kubectl edit verticadb

  2. In the subclusters subsection nested under spec, locate the subcluster that you want to delete. Delete the element in the subcluster array represents the subcluster that you want to delete. Each element is identified by a hyphen (-).

  3. After you delete the subcluster and save, you receive a message similar to the following:

    verticadb.vertica.com/verticadb edited