Vertica on Microsoft Azure

• 1: Deploying Vertica from the Azure Marketplace
• 2: Manually deploy Vertica on Microsoft Azure
• 2.1: Recommended Azure VM types
• 2.2: Supported Azure operating systems
• 2.3: Configuring and launching a new instance
• 2.4: Connect to a virtual machine
• 2.5: Prepare the virtual machines
• 2.6: Configure storage
• 2.7: Download Vertica
• 2.8: Form a cluster and install Vertica
• 2.9: After your cluster is up and running
• 3: Eon Mode databases on Azure
• 3.1: Eon Mode on Azure prerequisites
• 3.2: Manually creating an Eon Mode database on Azure
• 4: page_title

1 - Deploying Vertica from the Azure Marketplace

Deploy Vertica in the Microsoft Azure Cloud using the Vertica Analytics Platform entry in the Azure Marketplace. Vertica provides the following deployment options:

• Eon Mode: Deploy a Management Console (MC) instance, and then provision and create an Eon Mode database from the MC. For cluster and storage requirements, see Eon Mode on Azure prerequisites.

• Enterprise Mode: Deploy a four-node Enterprise Mode database comprised of one MC instance and three database nodes. This requires an Azure subscription with a minimum of 12 cores for the Vertica Marketplace solution.

  The Enterprise Mode deployment uses the MC primarily as a monitoring tool. For example, you cannot provision and create a database with an Enterprise Mode MC. For information about creating and managing an Enterprise Mode database, see Create a database using administration tools.

Creating a deployment

Eon Mode and Enterprise Mode require much of the same information for deployment. Any information that is not required for both deployment types is clearly marked.

1. selecting the deployment type

1. Sign in to your Microsoft Azure account. From the Home screen, select Create a resource under Azure services.

2. Search for Vertica Analytics Platform and select it from the search results.

3. On the Vertica Analytics Platform page, select one of the following:

   • To deploy an MC instance that can manage an Eon Mode database, select Vertica Data Warehouse, Eon BYOL.

   • To deploy an Enterprise Mode database, select Vertica Analytics Platform.

4. On the next screen, select Create.

After you select your deployment type, the Basics tab on the Create Vertica Analytics Platform page displays.

2. adding project and instance details on the basics tab

Provide the following information in the Project details and Instance details sections:

1. Subscription: Azure bills this subscription for the cluster resources.

2. Resource group: The location to save all of the Azure resources. Create a new resource group or choose an existing one from the dropdown list.

3. Region: The location where the virtual machine running your MC instance is deployed.

4. VerticaManagement ConsoleUser: Eon Mode only. The administrator username for the MC.

5. SSH public key for OS Access: Provide the SSH public key associated with the Vertica User, for command line access to the virtual machine.

6. Password for MC Access: Enter a password to log in to Management Console. Note that Management Console requires that you change your password after the initial login.

7. Confirm password: Reenter the value you entered in Password for MC Access.

8. Select Next: Virtual Machine Settings >.

3. selecting virtual machine settings

Provide the following information on the Virtual Machine Settings tab:

1. Management Console VM size: Select Change size to customize the VM settings or select the default. For a list of VM types recommended by use case, see Recommended Azure VM types.

2. Storage account of Eon DB: Eon Mode only. The storage account associated with the database deployment.

3. Number of Vertica Cluster nodes: Enterprise Mode only. The number of nodes to deploy in the cluster, in addition to the MC instance.
   The Community Edition (CE) license is automatically applied to the cluster. This license is limited to 1 TB of RAW data 3 Vertica nodes. If you select more than 3 nodes with a CE license, the initial database is created on the first 3 nodes. For information about upgrading your license, see Managing licenses.

4. Vertica Node VM size: Enterprise Mode only. Select the VM type to deploy in your cluster. Use the default or select Change size to customize the VM settings. For a list of VM types recommended by use case, see Recommended Azure VM types.

5. Total RAW storage per node: Enterprise Mode only. Select the amount of storage per node from the dropdown list. Each VM has a set of premium data disks that are configured and presented as a single storage location.

6. Select Next: Network Settings >.

4. selecting network settings

Provide the following information on the Network Settings tab:

1. Virtual Network: The virtual network that hosts the Vertica cluster. Create a new virtual network or select an existing one from the dropdown list.
   If you select an existing virtual network, Vertica recommends that you already created a subnet to use for the deployment.

2. First subnet: The subnet for the associated Virtual Network. Create a new subnet or select an existing one from the dropdown list.

3. Public IP Address Resource Name: Each VM is configured with a publicly accessible IP address. This field allows you to specify the resource name for those IP addresses, and whether they are static or dynamic. The first public IP address resource is created exactly as entered, and associated with the VerticaManagement Console. Azure appends a number from 1 to 16 to the resource name for each additional Vertica cluster node created. This number associates each VM with a resource.

4. Domain Name Label for Management Console: Because each VM has a public IP address, each node requires a DNS name. Enter a prefix for the name. The first DNS name is created exactly as entered, and associated with the VerticaManagement Console. Azure appends a number from 1 to 16 to the DNS name for each Vertica cluster node created. That number associates each VM with a resource. Azure adds the remaining part of the fully qualified domain name based on the location where you created the cluster.

5. Select Next: Review + create >.

5. verifying on review + create

As the Review + create page loads, Azure validates your settings. After it passes validation, review your settings. When you are satisfied with your selections, select Create.

Accessing the MC after deployment

After your resources are successfully deployed, you are brought to the Overview page on Home > resources-name > Deployments. You must retrieve your Management Console IP address and username to log in.

1. From the Overview page, select Outputs in the left navigation.

2. Copy the vertica management console URL and vertica management console user name.

3. Paste the vertica management console URL in the browser address bar and press Enter.

4. Depending on your browser, you might receive a warning of a security risk. If you receive the warning, select the Advanced button and follow the browsers instructions to proceed to the Management Console.

5. On the VerticaManagement Console log in page, paste the vertica management console user name, and enter the Password for MC Access that you entered on Basics > Project details when you were deploying your MC instance.

Deleting a resource group

For details about the Azure Resource Manager and deleting a resource group, see the Azure documentation.

2 - Manually deploy Vertica on Microsoft Azure

Manually creating a database cluster for your Vertica deployment lets you customize your VMs to meet your specific needs. You often want to manually configure your VMs when deploying a Vertica cluster to host an Eon Mode database.

To start creating your Vertica cluster in Azure using manual steps, you first need to create a VM. During the VM creation process, you create and configure the other resources required for your cluster, which are then available for any additional VMs that you create.

The topics in this section explain how to manually deploy Vertica on Azure.

2.1 - Recommended Azure VM types

Vertica supports a range of Microsoft Azure virtual machine (VM) types, each optimized for different purposes. Choose the VM type that best matches your performance and price needs as a user.

For the best performance in most common scenarios, use one of the following VMs:

2.2 - Supported Azure operating systems

For best performance, use one of the following operating systems when deploying Vertica on Azure:

• Red Hat 7.3 or later

• CentOS 7.3 or later. The Azure Marketplace solution as of this writing (June 2017) is based on CentOS 7.3.1611.

For more information, see Supported platforms.

2.3 - Configuring and launching a new instance

An Azure VM is similar to a traditional host. Just as with an on-premises cluster, you must prepare and configure the hardware settings for your cluster and network before you install Vertica.

The first steps are:

1. From the Azure marketplace, select an operating system that Vertica supports.

2. Select a VM type.See Recommended Azure VM types.

3. Choose a deployment model. For best results, choose the resource manager deployment model.

Configure network security group

Vertica has specific network security group requirements, as described in page_title.

Create and name your own network security group, following these guidelines.

You must configure SSH as:

• Protocol: TCP

• Source port range: Any

• Destination port range: 22

• Source: Any

• Destination: Any

You can make additional modifications, based on your specific requirements.

Add disk containers

Create an Azure storage account, which later contains your cluster storage disk containers.

For optimal throughput, select Premium storage and align the storage to your chosen VM type.

For more information about what a storage account is, and how to create one, refer to About Azure storage accounts.

For an Enterprise Mode database deployment, provision enough space

Configure credentials

Create a password or assign an SSH key pair to use with Vertica.

For information about how to use key pairs in Azure, see How to create and use an SSH public and private key pair for Linux VMs in Azure.

Assign a public IP address

A public IP is an IP address that you can use to connect to your cluster externally. For best results, assign a single static public IP to a node in your cluster. You can then connect to other nodes in your cluster from your primary node using the internal IP addresses that Azure generated when you specified your virtual network settings.

By default, a public IP address is dynamic; it changes every time you shut down the server. You can choose a static IP address, but doing so can add cost to your deployment.

During a VM installation, you cannot set a DNS name. If you use dynamic public IPs, set the DNS name in the public IP resource for each VM after deployment.

For information about public IP addresses, refer to IP address types and allocation methods in Azure.

Create additional VMs

If needed, to create additional VMs, repeat the previous instructions in this document.

2.4 - Connect to a virtual machine

Before you can connect to any of the VMs you created, you must first make your virtual network externally accessible. To do so, you must attach the public IP address you created during network configuration to one of your VMs.

Connect to your VM

To connect to your VM, complete the following tasks:

1. Connect to your VM using SSH with the public IP address you created in the configuration steps.

2. Authenticate using the credentials and authentication method you specified during the VM creation process.

Connect to other VMs

Connect to other virtual machines in your virtual network by first using SSH to connect to your publicly connected VM. Then, use SSH again from that VM to connect through the private IP addresses of your other VMs.

If you are using private key authentication, you may need to move your key file to the root directory of your publicly connected VM. Then, use PuTTY or WinSCP to connect to other VMs in your virtual network.

2.5 - Prepare the virtual machines

After you create your VMs, you need to prepare them for cluster formation.

Add the Vertica license and private key

Prepare your nodes by adding your private key (if you are using one) to each node and to your Vertica license. These steps assume that the initial user you configured is the DBADMIN user.

1. As the dbadmin user, copy your private key file from where you saved it locally onto your primary node.

   Depending upon the procedure you use to copy the file, the permissions on the file may change. If permissions change, the install_vertica script fails with a message similar to the following:

   Failed Login Validation 10.0.2.158, cannot resolve or connect to host as root.
   

   If you receive a failure message, enter the following command to correct permissions on your private key file:

   $ chmod 600 /<name-of-key>.pem
   

2. Copy your Vertica license to your primary VM. Save it in your home directory or other known location.

Install software dependencies for Vertica on Azure

In addition to the Vertica standard Package dependencies, as the root user, you must install the following packages before you install Vertica on Azure:

• pstack

• mcelog

• sysstat

• dialog

2.6 - Configure storage

Use a dedicated Azure storage account for node storage.

When configuring your storage, make sure to use a supported file system. For details, see Recommended storage format types.

Attach disk containers to virtual machines (VMs)

Using your previously created storage account, attach disk containers to your VMs that are appropriate to your needs.

For best performance, combine multiple storage volumes into RAID-0. For most RAID-0 implementations, attach 6 storage disk containers per VM.

Combine disk containers for storage

If you are using RAID, follow these steps to create a RAID-0 drive on your VMs. The following example shows how you can create a RAID-0 volume named md10 from 6 individual volumes named:

• sdc

• sdd

• sde

• sdf

• sdg

• sdh

1. Form a RAID-0 volume using the mdadm utility:
   

   $ mdadm --create /dev/md10 --level 0 --raid-devices=6 \
     /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh
   

2. Format the file system to be one that Vertica supports:
   

   $ mkfs.ext4 /dev/md10
   

3. Find the UUID on the newly-formed RAID volume using the blkid command. In the output, look for the device you assigned to the RAID volume:
   

   $ blkid
    . . .
    /dev/md10 : UUID="e7510a6f-2922-4413-b5fa-9dcd725967fd" TYPE="ext4" PARTUUID="fb9b7449-08c3-4231-9ee5-086f7b0c9001"
    . . .
   

4. The RAID device can be renamed after a reboot. To ensure the filesystem is mounted in a predictable location on your VM, create a directory to use as the mount point to mount the filesystem. For example, you can choose to create a mount point named /data that you will use to store your database's catalog and data (or depot, if you are running Vertica in Eon Mode).

   $ mkdir /data
   

5. Using a text editor, add an entry to the /etc/fstab file for the UUID of the filesystem and your mount point so it is mounted when the system boots:

   UUID=RAID_UUID mountpoint      ext4    defaults,nofail,nobarrier    0   2
   

   For example, if you have the UUID shown in the previous example and the mount point /data, add the following line to the /etc/fstab file:

   UUID=e7510a6f-2922-4413-b5fa-9dcd725967fd  /data      ext4    defaults,nofail,nobarrier    0   2
   

6. Mount the RAID filesystem you added to the fstab file. For example, to mount a mount point named /data use the command:

   $ mount /data
   

7. Create folders for your Vertica data and catalog under your mount point.
   

   $ mkdir /data/vertica
   $ mkdir /data/vertica/data
   

   If you are planning to run Vertica in Eon Mode, create a directory for the depot instead of data:

   $ mkdir /data/vertica/depot
   

Create a swap file

In addition to storage volumes to store your data, Vertica requires a swap volume or swap file to operate.

Create a swap file or swap volume of at least 2 GB. The following steps show how to create a swap file within Vertica on Azure:

1. Install devnull and swapfile:
   

   $ install -o root -g root -m 0600 /dev/null /swapfile
   

2. Create the swap file:
   

   $ dd if=/dev/zero of=/swapfile bs=1024 count=2048k
   

3. Prepare the swap file using mkswap:
   

   $ mkswap /swapfile
   

4. Use swapon to instruct Linux to swap on the swap file:
   

   $ swapon /swapfile
   

5. Persist the swapfile in FSTAB:
   

   $ echo "/swapfile       swap    swap    auto      0       0" >> /etc/fstab
   

Repeat the volume attachment, combination, and swap file creation procedures for each VM in your cluster.

For more information

• About Azure storage accounts

• Prepare disk storage locations

2.7 - Download Vertica

To download the Vertica server appropriate for your operating system and license type, go to www.vertica.com/download/vertica.

Run the rpm to extract the files.

After you complete the download and extraction, the next section describes how to use the install_vertica script to form a cluster and install the Vertica database software.

2.8 - Form a cluster and install Vertica

Use the install_vertica script to combine two or more individual VMs to form a cluster and install the Vertica database.

Before you start

Before you run the install_vertica script:

• Check the Virtual Network page for a list of current VMs and their associated private IP addresses.

• Identify your storage location. The installer assumes that you have mounted your storage to /vertica/data. To specify another location, use the --data-dir argument.

• Identify your storage location. To create your database's data directory on mounted RAID drive, when you run the install_vertica script, provide /vertica/data as the value of the --data-dir option .

  Caution

  Caution: Do not store your data on the root drive.

Combine virtual machines (VMs)

The following example shows how to combine VMs using the install_vertica script.

1. While connected to your primary node, construct the following command to combine your nodes into a cluster.

   $ sudo /opt/vertica/sbin/install_vertica --hosts 10.2.0.164,10.2.0.165,10.2.0.166 --dba-user-password-disabled --point-to-point --data-dir /vertica/data --ssh-identity ~/<name-of-private-key>.pem --license <license.file>
   

2. Substitute the IP addresses for your VMs and include your root key file name, if applicable.

3. Include the --point-to-point parameter to configure spread to use direct point-to-point communication between all Vertica nodes, as required for clusters on Azure when installing or updating Vertica.

4. If you are using Vertica Community Edition, which limits you to three nodes, specify -L CE with no license file.

5. After you combine your nodes, to reduce security risks, keep your key file in a secure place—separate from your cluster—and delete your on-cluster key with the shred command:

   $ shred examplekey.pem
   

   Important

   You need your key file to perform future Vertica updates.

6. Reboot your cluster to complete the cluster formation and Vertica installation.

For complete information on the install_vertica script and its parameters, see Installing Vertica with the installation script.

2.9 - After your cluster is up and running

Now that your cluster is configured and running, take these steps:

1. Log into one of the database nodes using the database administrator account (named dbadmin by default).

2. Create and start a database:

   • To create an Enterprise Mode database, see Create a database using administration tools.
   • To create an Eon Mode database, see Manually creating an Eon Mode database on Azure.

3. Configure your database. See Configuring the database.

3 - Eon Mode databases on Azure

You can create an Eon Mode database on a cluster that is hosted on Azure. In this configuration, your database stores its data communally in Azure Blob storage. See Eon Mode to learn more about this database mode.

Eon Mode databases on Azure support some of the encryption features built into Azure Storage. You can use its encryption at rest feature transparently—you do not need to configure Vertica to take advantage of it. You can use Microsoft-managed or customer-managed keys for storage encryption. Vertica does not support Azure Storage's client-side encryption and encryption using customer-provided keys. See the Azure Data Encryption at rest page in the Azure documentation for more information about the encryption at rest features in Azure Storage.

This section explains how you create an Eon Mode database running on Azure cloud.

3.1 - Eon Mode on Azure prerequisites

Before you can create an Eon Mode database on Azure, you must have a database cluster and an Azure blob storage container to store your database's data.

Cluster requirements

Before you can create an Eon Mode database on Azure, you must provision a cluster to host it. See Configuring your Vertica cluster for Eon Mode for suggestions on choosing VM configurations and the number of nodes your cluster should start with.

Storage requirements

An Eon Mode database on Azure stores its data communally in Azure blob storage. Vertica only supports block blob storage for communal data storage, not append or page blob storage.

You must create a storage path for Vertica to use exclusively. This path can be a blob container or a folder within a blob container. This path must not contain any files. If you attempt to create an Eon Mode database with a container or folder that contains files, admintools returns an error.

You pass Vertica a URI for the storage path using the azb:// schema. See Azure Blob Storage object store for the format of this URI.

You must also configure the storage container so Vertica is authorized to access it. Depending on authentication method you use, you may need to supply Vertica the with credentials to access the container. Vertica can use one of following methods to authenticate with the blob storage container:

• Using Azure managed identities. This authentication method is transparent—you do not need to add any authentication configuration information to Vertica. Vertica automatically uses the managed identity bound to the VMs it runs on to authenticate with the blob storage container. See the Azure AD-managed identities for Azure resources documentation page in the Azure documentation for more information.

  If you provide credentials for either of the other two supported authentication methods, Vertica uses them instead of authenticating using a managed identity bound to your VM.

  Note

  If your Azure VMs have more than one managed identity bound to them, you must tell Vertica which identity to use when authenticating with the blob storage container. Vertica gets the identity to use from a tag set on the VMs that it is running on.

  On your VMs, create a tag with its key named VerticaManagedIdentityClientId and its value to the name of a managed identity bound to your VMs. See the Use tags to organize your Azure resources and management hierarchy page in the Azure documentation for more information.

• Using an account name and access key credentials for a service account that has full access to the blob storage container. In this case, you provide Vertica with the credentials when you create the Eon Mode database. See Creating an Authentication File for details.

• Using a shared access signature (SAS) that grants Vertica access to the storage container. See Grant limited access to Azure Storage resources using shared access signatures (SAS) in the Azure documentation. See Creating an Authentication File for details.

For details on how Vertica accesses Azure blob storage, see Azure Blob Storage object store.

3.2 - Manually creating an Eon Mode database on Azure

Once you have met the cluster and storage requirements for using an Eon Mode database on Azure, you are ready to create an Eon Mode database. Use the admintools create_db tool to create your Eon Mode database.

Creating an authentication file

If your database will use a managed identity to authenticate with the Azure storage container, you do not need to supply any additional configuration information to the create_db tool.

If your database will not use a managed identity, you must supply create_db with authentication information in a configuration file. It must contain at least the AzureStorageCredentials parameter that defines one or more account names and keys Vertica will use to access blob storage. It can also contain an AzureStorageEnpointConfig parameter that defines an alternate endpoint to use instead of the the default Azure host name. This option is useful if you are creating a test environment using an Azure storage emulator such as Azurite.

The following table defines the values that can be set in these two parameters.

AzureStorageCredentials

Collection of JSON objects, each of which specifies connection credentials for one endpoint. This parameter takes precedence over Azure managed identities.

The collection must contain at least one object and may contain more. Each object must specify at least one of accountName or blobEndpoint, and at least one of accountKey or sharedAccessSignature.

• accountName: If not specified, uses the label of blobEndpoint.
• blobEndpoint: Host name with optional port (host:port). If not specified, uses account.blob.core.windows.net.
• accountKey: Access key for the account or endpoint.
• sharedAccessSignature: Access token for finer-grained access control, if being used by the Azure endpoint.

AzureStorageEndpointConfig

Collection of JSON objects, each of which specifies configuration elements for one endpoint. Each object must specify at least one of accountName or blobEndpoint.

• accountName: If not specified, uses the label of blobEndpoint.
• blobEndpoint: Host name with optional port (host:port). If not specified, uses account.blob.core.windows.net.
• protocol: HTTPS (default) or HTTP.
• isMultiAccountEndpoint: true if the endpoint supports multiple accounts, false otherwise (default is false). To use multiple-account access, you must include the account name in the URI. If a URI path contains an account, this value is assumed to be true unless explicitly set to false.

The authentication configuration file is a text file containing the configuration parameter names and their values. The values are in a JSON format. The name of this file is not important. The following examples use the file name auth_params.conf.

The following example is a configuration file for a storage account hosted on Azure. The storage account name is mystore, and the key value is a placeholder. In your own configuration file, you must provide the storage account's access key. You can find this value by right-clicking the storage account in the Azure Storage Explorer and selecting Copy Primary Key.

AzureStorageCredentials=[{"accountName": "mystore", "accountKey": "access-key"}]

The following example shows a configuration file that defines an account for a storage container hosted on the local system using the Azurite storage system. The user account and key are the "well-known" account provided by Azurite by default. Because this configuration uses an alternate storage endpoint, it also defines the AzureStorageEndpointConfig parameter. In addition to reiterating the account name and endpoint definition, this example sets the protocol to the non-encrypted HTTP.

AzureStorageCredentials=[{"accountName": "devstoreaccount1", "blobEndpoint": "127.0.0.1:10000 ",
                          "accountKey":
"Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=="
                        }]

AzureStorageEndpointConfig=[{"accountName": "devstoreaccount1",
                             "blobEndpoint": "127.0.0.1:10000", "protocol": "http"}]

Creating the Eon Mode database

Use the admintools create_db tool to create your Eon Mode database. The required arguments you pass to this tool are:

Some other common optional arguments for create_db are:

To view all arguments for the create_db tool, run the command:

admintools -t create_db --help

The following example demonstrates creating an Eon Mode database with the following settings:

• Vertica will use a storage account named mystore.

• The communal data will be stored in a directory named verticadb located in a storage container named db_blobs.

• The authentication information Vertica needs to access the storage container is in the file named auth_params.conf in the current directory. The contents of this file are shown in the first example under Creating an Authentication File.

• The hostnames of the nodes in the cluster are node01 through node03.

$ admintools -t create_db \
             --communal-storage-location=azb://mystore/db_blobs/verticadb \
             -x auth_params.conf -s node01,node02,node03  \
             -d verticadb --depot-path /vertica/depot --shard-count 3 \
             -p 'mypassword'

4 - page_title

Vertica has the following network security group requirements.

For details on security groups and how to create one, see the Azure documentation.

Inbound settings

Outbound settings