Vertica Documentation – Google Cloud Platform in MC

Mc: GCP Eon Mode instance recommendations

Mon, 01 Jan 0001 00:00:00 +0000

When you use the MC to deploy an Eon Mode database to the Google Cloud Platform (GCP), you choose the instance type to deploy as the database's nodes. The default instance settings in the MC are the more conservative option (currently, n1-standard-16). They are sufficient for most workloads. However, you may choose instances with more memory (such as n1-highmem-16) if your queries perform complex joins that may otherwise spill to disk. You can also choose instances with more cores (such as n1-standard-32), if you perform highly-complex compute-intensive analysis. The following links provide additional information about GCP machine type instances and Vertica:

Machine types: Google Cloud's documentation that describes configuration details for each instance option.
Supported GCP machine types: Available machine types when deploying Vertica to GCP.
Eon Mode volume configuration defaults for GCP: Details about Vertica's default volume configurations.

The more powerful instance you choose, the higher the cost per hour. You need to balance whether you want to use fewer, higher-powered but more expensive instances vs. relying on more lower-powered instances that cost less. Thanks to Eon Mode's elasticity, if you choose to use the less-powerful instances, you can always add more nodes to meet peak demands. When you reduce the number of instances to a minimum during off-peak times, you'll spend less than if you had a similar number of more-powerful instances.

Storage options

The MC's deployment wizard also asks you to select the type of local storage for your instances. You can select different options for each type of local storage that Vertica uses: the catalog, the depot, and temporary space. For all of these storage locations, you choose the type of disks to use (standard vs. SSD). You will see the best performance with SSD disks. However, SSD disks cost more.

For the depot, you also choose whether to use local or persistent disks. The local option is faster, as it resides directly on the virtual machine host. However, whenever you shut down the node, this storage is wiped clean. The persistent storage is slower than the local option, as it is not stored directly on the machine hosting the instance. However, it is not wiped out whenever you shut down the instance. See the Google Cloud documentation's Storage options page for more information.

Which of these options you choose depends on how much depot warming the nodes must perform when starting. If the content of your node's depots change little over time (or you tend to frequently start and stop instances), using persistent storage makes sense. In this case, the depot's warming period will be shorter because most of the data the node needs to participate in queries may still be in its depot when it starts. It will perform fewer fetches of data from communal storage while participating in queries.

If your working data set is rapidly changing or you tend to leave nodes stopped for extended periods of time, your best choice is usually to use local storage. In this scenario, the data in the node's depot when it restarts is usually stale. To participate in queries, the node must fetch much of the data it needs from communal storage, resulting in slower performance until it has warmed its depot. Using local ephemeral storage makes sense here, because you will get the benefit of having faster depot storage. Because your nodes have to warm their depots anyhow, there is less of a downside of having the depot on ephemeral storage.

For general guidelines on scaling your cluster for Eon Mode database, see Configuring your Vertica cluster for Eon Mode.

Mc: Provision an Eon Mode cluster and database on GCP in MC

Mon, 01 Jan 0001 00:00:00 +0000

You can use Google Marketplace and MC to provision an Eon Mode database on GCP. The sections below give an overview of how to set up an Eon Mode database on GCP, with links to the detailed procedures

Prerequisites

Step 1: provision an MC instance using Google marketplace

These steps are an overview of the procedure. For more detailed instructions, see Deploy an MC instance in GCP for Eon Mode.

In Google Marketplace:

Select the Vertica Eon Mode solution.
Fill in the fields to configure a GCP MC instance.
Click the Deploy button to provision the MC instance.
Connect to and log into the MC instance.

You are now in the new MC instance running on GCP, and the MC home page appears.

Step 2: use the MC instance to provision an Eon Mode database on GCP

To use the MC to provision and deploy a new Eon Mode database on GCP:

From the MC home screen, click Create new database to launch the Create a Vertica Cluster on Google Cloud wizard.
On the first page of the wizard enter the following information:
- Google Cloud Storage HMAC Access Key and HMAC Secret Key: Copy and paste the HMAC access key and secret you created earlier. You find these values on the Interoperability tab of the of the Storage Settings page. See Eon Mode on GCP prerequisites for details.
- Zone: This value defaults to the zone containing your MC instance. Make this value is the same as the zone containing the Google Cloud Storage bucket that your database will use for communal storage.
- Caution
  You will see significant performance issues if you choose different zones for cluster instances, storage, or the MC.
- CIDR Range: The IP address range for clients to whom you want to grant access to your database. Make this range as restrictive as possible to limit access to your database.
Click Next, and supply the following information:
- Vertica Database Name: the name for your new database. See Creating a database name and password for database name requirements.
- Vertica Version: select the desired Vertica database version. You can select from the latest hotfix of recent Vertica releases. For each database version, you can also select the operating system.
- Vertica Database User Name: the name of the database superuser. This name defaults to dbadmin, but you can enter another user name here.
- Password and Confirm Password: Enter a password for the database superuser account.
- Database Size: The number of nodes in your initial database. If you specify more than three nodes here, you must supply a valid Vertica license file in the Vertica License field (below).
- Vertica License: Click Browse to locate and upload your Vertica license key file. If you do not supply a license key file here, the wizard deploys your database with a Vertica Community Edition license. This license has a three node limit, so the value in the Database Size filed cannot be larger than 3 if you do not supply a license. If you use a Community Edition license for your deployment, you can upgrade the license later to expand your cluster load more than 1TB of data. See Managing licenses form more information.
  
  Note
  This field does not appear if you created your MC instance using a by-the-hour (BTH) launcher. The BTH license is automatically applied to all clusters you create using a BTH MC instance. For a by-the-hour license, cloud vendors charge the customer for licensed Vertica usage along with their cloud infrastructure charges.
- Load example data: Check this box if you want your deployed database to load some example clickstream data. This option is useful if you are testing features and just want some preloaded data in the database to query.
Click Next and supply the following information:
- Instance Type: the specifications of the virtual machine instances the MC will use to deploy your database nodes. See the Google Cloud documentation's Machine types page for details of each instance type. Also see GCP Eon Mode instance recommendations.
- Database Depot Path and Disk Type: the local mount point for the depot, and the type and number of local disks dedicated to the depot for each node. You cannot change the mount path for the depot. The disks you select in the Disk Type field are only used to store the depot. On the next page of the wizard, you will configure disks for the catalog and temporary disk space. You will see the best performance when using SSD disks, although at a higher cost. You can choose to use faster local storage for your depot. However, local storage is ephemeral—GCP wipes the disk clean whenever you stop the instance. This means each time you start a node, it will have to warm its depot from scratch, rather than taking advantage of any still-current data in its depot. See the Google Cloud documentation's Storage options page for more information about the local disk options.
- Volume Size: the amount of disk space available on each disk attached to each node in your cluster. This field shows you the total disk space available per node in your cluster. For the best practices on choosing the amount of disk space for your nodes, see Configuring your Vertica cluster for Eon Mode.
- Data Segmentation Shards: sets the number of shards in your database. After you set this value, you cannot change it later. See Configuring your Vertica cluster for Eon Mode for recommendations. The default value is based on the number of nodes you entered in the Database size you specified earlier. It is usually sufficient, unless you anticipate greatly expanding your cluster beyond your initial node count.
- Communal Location: a Google Cloud Storage URL that specifies where to store your database's communal data. See Eon Mode on GCP prerequisites for requirements.
- Instance IP settings: specify whether the nodes in your database will have static or ephemeral network addresses that are accessible from the internet, or addresses that are only accessible from within the internal virtual network.
Click Next. The wizard validates your communal storage location URL. If there is an problem with the URL you entered, it displays an error message and prompts you to fix the URL.

After your communal storage URL passes validation, fill in the following information:
- Database Catalog Path, Disk Type, and Size (GB) per Available Node: the mount point disk type, and disk size for the local copy of the database catalog on each node. You cannot edit the mount point. You choose the type of local disk to use for the catalog, and its size. You can only choose persistent disk storage for the catalog. SSD drives are faster, but more expensive than standard disks. The default setting for the disk size is adequate for most medium size databases. Increase the size if you anticipate maintaining a large database.
- Database Temp Path, Disk Type, and Size (GB) per Available Node: the mount point disk type, and disk size for the temporary storage space on each node. You cannot edit the mount point. You choose the type of local disk to use, and its size. You can only choose persistent disk storage for the temporary disk space. SSD drives are faster, but more expensive than standard disks. The default setting is adequate for most databases. Consider increasing the temporary space if you perform many complex merges that spill to disk.
- Label Instances: check this box to enable adding labels to your node's instances. Many organizations use labels to organize, track responsibility, and assign costs for instances. See the Google Cloud documentation's Labeling resources page for more information. If you choose to add labels, enter the label name and value, and click Add.
Click Next. Review the summary of all your database settings. If you need to make a correction, use the Back button to step back to previous pages of the wizard.
When you are satisfied with the database settings, check Accept terms and conditions and click Create.

The process of provisioning and creating the database takes several minutes. After it completes successfully, the MC displays a Get Started button. This button leads to a page of useful links for getting started with your new database.

Mc: Reviving an Eon Mode database on GCP in MC

Mon, 01 Jan 0001 00:00:00 +0000

An Eon Mode database keeps an up-to-date version of its data and metadata in its communal storage location. After a cluster hosting an Eon Mode database is terminated, this data and metadata continue to reside in communal storage. When you revive the database later, Vertica uses the data in this location to restore the database in the same state on a newly provisioned cluster.

Follow these steps to revive an Eon Mode database on GCP:

On the MC home page, click Revive Eon Mode Database. MC launches the Provision and Revive an Eon Mode Database wizard.
On the first page of the wizard, enter the following information:
- Google Cloud Storage HMAC Access Key and HMAC Secret Key: Copy and paste the HMAC access key and secret you created when you created the database. See Eon Mode on GCP prerequisites for details.
- Zone: This value defaults to the zone containing your MC instance. Make this value the same as zone containing the Google Cloud Storage bucket your database will use for communal storage. You will see significant performance issues if you choose different zones for cluster instances, storage, or the MC.
- CIDR Range: The IP address range for clients you want to grant access to your database. Make this range as restrictive as possible to limit the exposure of your database.
Click Next. On the second page of the wizard, set Google Storage Path for Communal Storage of Database to the URL of the communal storage bucket for the Eon Mode database to revive. For requirements, see Eon Mode on GCP prerequisites.
Click Discover. MC displays a list of all Eon Mode databases available on the specified communal storage location.
Select the database to revive. MC prepopulates the choices for the Data, Depot, and Temp catalogs, using the same machine types and configuration choices used when the database was created.
Click Next. Review the summary of all your database settings. If you need to make a correction, use the Back button to step back to previous pages of the wizard.
When you are satisfied with the database settings, check the Accept terms and conditions box and click Revive Database.

MC displays a progress screen while creating the cluster and reviving the database onto it, a process which takes several minutes. After it completes successfully, the MC displays a Get Started button. This button leads to a page of useful links for getting started with your new database.

Mc: Eon Mode volume configuration defaults for GCP

Mon, 01 Jan 0001 00:00:00 +0000

Vertica supports a variety of disk volume resources for provisioning instances on Google Cloud Platform (GCP).

All data is secured with Google-managed data encryption. Management Console does not support user-managed data encryption.

For performance information, see Google's Block storage performance documentation.

Persistent disk defaults

You can allocate up to 128 persistent disks (PDs). The following table describes the default persistent disk volume resources that Vertica provides:

Instance Types	Catalog	Depot	Temp
n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64 n2-standard/highmem-16 n2-standard/highmem-32 n2-standard/highmem-48 n2-standard/highmem-64	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	Configurable 8 PD (Standard/SSD) volume Default: 600 GB	Configurable 1 PD (Standard/SSD) volume Default: 100 GB

Instance Types

Catalog

Depot

Temp

n1-standard/highmem-16

n1-standard/highmem-32

n1-standard/highmem-64

n2-standard/highmem-16

n2-standard/highmem-32

n2-standard/highmem-48

n2-standard/highmem-64

Configurable 1 PD (Standard/SSD) volume

Default: 50 GB

Configurable 8 PD (Standard/SSD) volume

Default: 600 GB

Configurable 1 PD (Standard/SSD) volume

Default: 100 GB

Local SSD defaults (ephemeral storage)

Allocate up to 24 local SSDs for ephemeral storage with the following considerations:

There is an extra cost for each local SSD.
All local SSD are 375G fixed size, with the option of SCSI or NVMe interface. An NVMe disk has twice the input/output operations per second (IOPs) compared to a SCSI disk.

Important

When the local SSD is in use, you cannot stop or start the instance or a cluster containing local SSD instances. If you do shut down an instance with a local SSD through the guest operating system, you will not be able to restart the instance and the data on the local SSD will be lost.

For details, see Google's Adding Local SSDs page.

The following table describes the default local SSD disk volume resources that Vertica provides:

Total Local SSDs	Instance Types	Catalog (Persistent)	Depot (Ephemeral)	Temp (Ephemeral)
4	n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64 n2-standard/highmem-16 n2-standard/highmem-32	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	3 x 375 GB	1 x 375 GB
5	n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	4 x 375 GB	1 x 375 GB
6	n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	5 x 375 GB	1 x 375 GB
8	n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64 n2-standard/highmem-16 n2-standard/highmem-32 n2-standard/highmem-48 n2-standard/highmem-64	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	6 x 375 GB	2 x 375 GB
16	n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64 n2-standard/highmem-16 n2-standard/highmem-32 n2-standard/highmem-48 n2-standard/highmem-64	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	12 x 375 GB	4 x 375 GB
24	n1-standard/highmem-16 n1-standard/highmem-32 n1-standard/highmem-64 n2-standard/highmem-16 n2-standard/highmem-32 n2-standard/highmem-48 n2-standard/highmem-64	Configurable 1 PD (Standard/SSD) volume Default: 50 GB	20 x 375 GB	4 x 375 GB

Vertica Documentation – Google Cloud Platform in MC

Mc: GCP Eon Mode instance recommendations

Storage options

Mc: Provision an Eon Mode cluster and database on GCP in MC

Prerequisites

Step 1: provision an MC instance using Google marketplace

Step 2: use the MC instance to provision an Eon Mode database on GCP

Caution

Note

See also

Mc: Reviving an Eon Mode database on GCP in MC

Mc: Eon Mode volume configuration defaults for GCP

Persistent disk defaults

Local SSD defaults (ephemeral storage)

Important