Google Cloud Platform in MC

• 1: GCP Eon Mode instance recommendations
• 2: Custom GCP image
• 3: Provision an Eon Mode cluster and database on GCP in MC
• 4: Reviving an Eon Mode database on GCP in MC
• 5: Eon Mode volume configuration defaults for GCP

1 - GCP Eon Mode instance recommendations

When you use the MC to deploy an Eon Mode database to the Google Cloud Platform (GCP), you choose the instance type to deploy as the database's nodes. The default instance settings in the MC are the more conservative option (currently, n1-standard-16). They are sufficient for most workloads. However, you may choose instances with more memory (such as n1-highmem-16) if your queries perform complex joins that may otherwise spill to disk. You can also choose instances with more cores (such as n1-standard-32), if you perform highly-complex compute-intensive analysis. The following links provide additional information about GCP machine type instances and Vertica:

• Machine types: Google Cloud's documentation that describes configuration details for each instance option.

• Supported GCP machine types: Available machine types when deploying Vertica to GCP.

• Eon Mode volume configuration defaults for GCP: Details about Vertica's default volume configurations.

The more powerful instance you choose, the higher the cost per hour. You need to balance whether you want to use fewer, higher-powered but more expensive instances vs. relying on more lower-powered instances that cost less. Thanks to Eon Mode's elasticity, if you choose to use the less-powerful instances, you can always add more nodes to meet peak demands. When you reduce the number of instances to a minimum during off-peak times, you'll spend less than if you had a similar number of more-powerful instances.

Storage options

The MC's deployment wizard also asks you to select the type of local storage for your instances. You can select different options for each type of local storage that Vertica uses: the catalog, the depot, and temporary space. For all of these storage locations, you choose the type of disks to use (standard vs. SSD). You will see the best performance with SSD disks. However, SSD disks cost more.

For the depot, you also choose whether to use local or persistent disks. The local option is faster, as it resides directly on the virtual machine host. However, whenever you shut down the node, this storage is wiped clean. The persistent storage is slower than the local option, as it is not stored directly on the machine hosting the instance. However, it is not wiped out whenever you shut down the instance. See the Google Cloud documentation's Storage options page for more information.

Which of these options you choose depends on how much depot warming the nodes must perform when starting. If the content of your node's depots change little over time (or you tend to frequently start and stop instances), using persistent storage makes sense. In this case, the depot's warming period will be shorter because most of the data the node needs to participate in queries may still be in its depot when it starts. It will perform fewer fetches of data from communal storage while participating in queries.

If your working data set is rapidly changing or you tend to leave nodes stopped for extended periods of time, your best choice is usually to use local storage. In this scenario, the data in the node's depot when it restarts is usually stale. To participate in queries, the node must fetch much of the data it needs from communal storage, resulting in slower performance until it has warmed its depot. Using local ephemeral storage makes sense here, because you will get the benefit of having faster depot storage. Because your nodes have to warm their depots anyhow, there is less of a downside of having the depot on ephemeral storage.

For general guidelines on scaling your cluster for Eon Mode database, see Configuring your Vertica cluster for Eon Mode.

2 - Custom GCP image

Vertica publishes a Google Cloud Platform (GCP) image that contains third-party libraries and other software that is required to install and operate the Management Console (MC) instance and Vertica database cluster instances. In some circumstances, you might need to add customizations to your MC environment, such as monitoring or security updates.

You cannot modify the published Vertica image directly, but you can create a custom image from the published image with your customizations. You can select this custom image when you create or revive a cluster.

Prerequisites

GCP account with administrative rights.

Create a custom GCP image

To create a custom GCP image, you must launch a base Virtual Machine (VM) instance from the published Vertica image so that you can make your changes. After you create this base instance, you can save it as a custom image.

Create the base instance

To begin, you must create an instance that includes the published Vertica image. This is a temporary instance that you can delete after you create the custom image:

1. Log in to GCP.
2. From the GCP console, go to Compute Engine.
3. In the side navigation bar, go to Storage > Images.
4. Filter the images table for the published Vertica image that you want to use in the base instance. For example, filter for vertica-23-4-0 to customize the 23.4.0-x image.
5. Copy the name of the published Vertica image and save it for later use.
6. Go to Virtual machines > VM instances in the side navigation bar.
7. Select CREATE INSTANCE.
8. Enter a name for your instance.
9. Select a region.
10. Scroll down to Boot disk, and select CHANGE.
11. In the Boot disk window, select the CUSTOM IMAGES tab.
12. In the image field, paste the name of the published Vertica image to filter the image list.
13. Select SELECT. The Boot disk window closes and you return to Create an instance.
14. Select CREATE to create the base instance.

After you select CREATE, you return to Virtual machines > VM instances in Compute Engine. When GCP finishes creating the base instance, you have a running instance that you can access and customize.

Customize the base instance

      
$ ssh-keygen -t rsa -b 4096 -C gcpAdminUsername
Generating public/private rsa key pair.
Enter file in which to save the key: /path/to/gcp_custom_keys
...
Your public key has been saved in /path/to/gcp_custom_keys.pub
...

After you create the base instance, add the contents of your public key to the instance so you can SSH into the machine to add customizations or configurations:

1. In Virtual machines > VM instances, filter for the base instance.
2. Select the instance name. The instance Details tab displays.
3. Select EDIT in the top menu.
4. In Security and access > SSH Keys, select ADD ITEM.
5. In the SSH key 1 * field, paste the contents of the public SSH key that you just created.

   Important

   If you use public key with the default comment format, do not include the @hostname portion of the comment. The key that you paste into SSH key 1 * must end with username.
6. Select SAVE.
7. Return to the VM instances window, go to the Network interfaces section and retrieve the External IP address for the base instance.
8. Remotely log in to the base instance with the user string from the public key. The -i option uses the local machine's private key for authentication:

   $ ssh -i path/to/.ssh/private-key username@baseInstanceExternalIPAddress
   

9. Add custom configurations to the base instance environment.

Now, the base instance includes the published Vertica base image, and any custom configurations.

Create the custom image

After you customize the base instance, you can create a new image that includes the published Vertica base image and your customizations:

1. On Virtual Machines > VM instances, select the base instance and select STOP.
2. After the instance stops, select the instance name in the list. The image DETAILS tab displays.
3. Scroll to the Boot disk section. In the Name column, select the instance name.
4. In the top navigation bar, select CREATE IMAGE. The Create an image page displays.
5. In the Name field, enter a name for the instance. You must use the following naming convention, or the MC cannot identity the image when you create or revive a cluster:

   vertica-major-minor-servicepack-hotfix-imageName
   

   For example, the following name uses Vertica version 12, minor release 0, service pack 0, hotfix zero:

   vertica-12-0-0-0-customImageName
   

6. Verify that Source disk lists the base instance name.
7. Select CREATE.
8. Record the name of the image that you just created so you can enter it in the MC.

After GCP creates the image, you have a custom image that includes the published Vertica base image, and any environment customizations. This custom image is available in the current GCP project.

If you no longer need the customized base instance, you can delete it from your GCP VM instances.

Select the custom image in MC

After you create the custom image in GCP, return to the MC and use the custom image in a workflow:

1. Log into the MC and begin one of the following workflows:
   • Create database cluster
   • Revive cluster
2. On the wizard page Enter Vertica database name and login credentials, select the Use Custom Image box.
3. In Custom Image Name, enter the custom image name. For example, vertica-23-4-0-0-customImageName.
4. Complete the create or revive workflow.

After you complete the workflow, any new or revived Vertica database cluster instances run the custom image. If you scale a new or revived cluster, the added instances use the same custom image.

3 - Provision an Eon Mode cluster and database on GCP in MC

You can use Google Marketplace and MC to provision an Eon Mode database on GCP. The sections below give an overview of how to set up an Eon Mode database on GCP, with links to the detailed procedures

Prerequisites

• GCP Eon Mode instance recommendations
• Eon Mode on GCP prerequisites

Step 1: provision an MC instance using Google marketplace

These steps are an overview of the procedure. For more detailed instructions, see Deploy an MC instance in GCP for Eon Mode.

In Google Marketplace:

1. Select the Vertica Eon Mode solution.

2. Fill in the fields to configure a GCP MC instance.

3. Click the Deploy button to provision the MC instance.

4. Connect to and log into the MC instance.

You are now in the new MC instance running on GCP, and the MC home page appears.

Step 2: use the MC instance to provision an Eon Mode database on GCP

To use the MC to provision and deploy a new Eon Mode database on GCP:

1. From the MC home screen, click Create new database to launch the Create a Vertica Cluster on Google Cloud wizard.

2. On the first page of the wizard enter the following information:

   • Google Cloud Storage HMAC Access Key and HMAC Secret Key: Copy and paste the HMAC access key and secret you created earlier. You find these values on the Interoperability tab of the of the Storage Settings page. See Eon Mode on GCP prerequisites for details.

   • Zone: This value defaults to the zone containing your MC instance. Make this value is the same as the zone containing the Google Cloud Storage bucket that your database will use for communal storage.

   • Caution

     You will see significant performance issues if you choose different zones for cluster instances, storage, or the MC.

   • CIDR Range: The IP address range for clients to whom you want to grant access to your database. Make this range as restrictive as possible to limit access to your database.

3. Click Next, and supply the following information:

   • Vertica Database Name: the name for your new database. See Creating a database name and password for database name requirements.

   • Vertica Version: select the desired Vertica database version. You can select from the latest hotfix of recent Vertica releases. For each database version, you can also select the operating system.

   • Vertica Database User Name: the name of the database superuser. This name defaults to dbadmin, but you can enter another user name here.

     For details about using a custom GCP instance, see Custom GCP image.

   • Password and Confirm Password: Enter a password for the database superuser account.

   • Database Size: The number of nodes in your initial database. If you specify more than three nodes here, you must supply a valid Vertica license file in the Vertica License field (below).

   • Vertica License: Click Browse to locate and upload your Vertica license key file. If you do not supply a license key file here, the wizard deploys your database with a Vertica Community Edition license. This license has a three node limit, so the value in the Database Size filed cannot be larger than 3 if you do not supply a license. If you use a Community Edition license for your deployment, you can upgrade the license later to expand your cluster load more than 1TB of data. See Managing licenses form more information.

     Note

     This field does not appear if you created your MC instance using a by-the-hour (BTH) launcher. The BTH license is automatically applied to all clusters you create using a BTH MC instance. For a by-the-hour license, cloud vendors charge the customer for licensed Vertica usage along with their cloud infrastructure charges.

   • Load example data: Check this box if you want your deployed database to load some example clickstream data. This option is useful if you are testing features and just want some preloaded data in the database to query.

4. Click Next and supply the following information:

   • Instance Type: the specifications of the virtual machine instances the MC will use to deploy your database nodes. See the Google Cloud documentation's Machine types page for details of each instance type. Also see GCP Eon Mode instance recommendations.

   • Database Depot Path and Disk Type: the local mount point for the depot, and the type and number of local disks dedicated to the depot for each node. You cannot change the mount path for the depot. The disks you select in the Disk Type field are only used to store the depot. On the next page of the wizard, you will configure disks for the catalog and temporary disk space. You will see the best performance when using SSD disks, although at a higher cost. You can choose to use faster local storage for your depot. However, local storage is ephemeral—GCP wipes the disk clean whenever you stop the instance. This means each time you start a node, it will have to warm its depot from scratch, rather than taking advantage of any still-current data in its depot. See the Google Cloud documentation's Storage options page for more information about the local disk options.

   • Volume Size: the amount of disk space available on each disk attached to each node in your cluster. This field shows you the total disk space available per node in your cluster. For the best practices on choosing the amount of disk space for your nodes, see Configuring your Vertica cluster for Eon Mode.

   • Data Segmentation Shards: sets the number of shards in your database. After you set this value, you cannot change it later. See Configuring your Vertica cluster for Eon Mode for recommendations. The default value is based on the number of nodes you entered in the Database size you specified earlier. It is usually sufficient, unless you anticipate greatly expanding your cluster beyond your initial node count.

   • Communal Location: a Google Cloud Storage URL that specifies where to store your database's communal data.

     The storage bucket must use the Standard storage class. See Eon Mode on GCP prerequisites for additional requirements.

   • Instance IP settings: specify whether the nodes in your database will have static or ephemeral network addresses that are accessible from the internet, or addresses that are only accessible from within the internal virtual network.

5. Click Next. The wizard validates your communal storage location URL. If there is an problem with the URL you entered, it displays an error message and prompts you to fix the URL.

   After your communal storage URL passes validation, fill in the following information:

   • Database Catalog Path, Disk Type, and Size (GB) per Available Node: the mount point disk type, and disk size for the local copy of the database catalog on each node. You cannot edit the mount point. You choose the type of local disk to use for the catalog, and its size. You can only choose persistent disk storage for the catalog. SSD drives are faster, but more expensive than standard disks. The default setting for the disk size is adequate for most medium size databases. Increase the size if you anticipate maintaining a large database.

   • Database Temp Path, Disk Type, and Size (GB) per Available Node: the mount point disk type, and disk size for the temporary storage space on each node. You cannot edit the mount point. You choose the type of local disk to use, and its size. You can only choose persistent disk storage for the temporary disk space. SSD drives are faster, but more expensive than standard disks. The default setting is adequate for most databases. Consider increasing the temporary space if you perform many complex merges that spill to disk.

   • Label Instances: check this box to enable adding labels to your node's instances. Many organizations use labels to organize, track responsibility, and assign costs for instances. See the Google Cloud documentation's Labeling resources page for more information. If you choose to add labels, enter the label name and value, and click Add.

6. Click Next. Review the summary of all your database settings. If you need to make a correction, use the Back button to step back to previous pages of the wizard.

7. When you are satisfied with the database settings, check Accept terms and conditions and click Create.

The process of provisioning and creating the database takes several minutes. After it completes successfully, the MC displays a Get Started button. This button leads to a page of useful links for getting started with your new database.

See also

• Managing an Eon Mode database in MC
• Stopping and starting an Eon Mode cluster
• Reviving an Eon Mode database cluster

4 - Reviving an Eon Mode database on GCP in MC

Follow these steps to revive an Eon Mode database on GCP:

1. On the MC home page, click Revive Eon Mode Database. MC launches the Provision and Revive an Eon Mode Database wizard.

2. On the first page of the wizard, enter the following information:

   • Google Cloud Storage HMAC Access Key and HMAC Secret Key: Copy and paste the HMAC access key and secret you created when you created the database. See Eon Mode on GCP prerequisites for details.

   • Zone: This value defaults to the zone containing your MC instance. Make this value the same as zone containing the Google Cloud Storage bucket your database will use for communal storage. You will see significant performance issues if you choose different zones for cluster instances, storage, or the MC.

   • CIDR Range: The IP address range for clients you want to grant access to your database. Make this range as restrictive as possible to limit the exposure of your database.

3. Click Next. On the second page of the wizard, set Google Storage Path for Communal Storage of Database to the URL of the communal storage bucket for the Eon Mode database to revive. For requirements, see Eon Mode on GCP prerequisites.

4. Click Discover. MC displays a list of all Eon Mode databases available on the specified communal storage location.

5. Select the database to revive. MC prepopulates the choices for the Data, Depot, and Temp catalogs, using the same machine types and configuration choices used when the database was created.

6. Click Next. Review the summary of all your database settings. If you need to make a correction, use the Back button to step back to previous pages of the wizard.

7. When you are satisfied with the database settings, check the Accept terms and conditions box and click Revive Database.

MC displays a progress screen while creating the cluster and reviving the database onto it, a process which takes several minutes. After it completes successfully, the MC displays a Get Started button. This button leads to a page of useful links for getting started with your new database.

5 - Eon Mode volume configuration defaults for GCP

Vertica supports a variety of disk volume resources for provisioning instances on Google Cloud Platform (GCP).

All data is secured with Google-managed data encryption. Management Console does not support user-managed data encryption.

For performance information, see Google's Block storage performance documentation.

Persistent disk defaults

You can allocate up to 128 persistent disks (PDs). The following table describes the default persistent disk volume resources that Vertica provides:

Local SSD defaults (ephemeral storage)

Allocate up to 24 local SSDs for ephemeral storage with the following considerations:

• There is an extra cost for each local SSD.

• All local SSD are 375G fixed size, with the option of SCSI or NVMe interface. An NVMe disk has twice the input/output operations per second (IOPs) compared to a SCSI disk.

The following table describes the default local SSD disk volume resources that Vertica provides: