This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Adding nodes

There are many reasons for adding one or more nodes to an installation of Vertica:.

1: Adding hosts to a cluster
2: Adding nodes to a database
3: Add nodes to a cluster in AWS

There are many reasons to add one or more nodes to a Vertica cluster:

Increase system performance or capacity. Add nodes due to a high query load or load latency, or increase disk space in Enterprise Mode without adding storage locations to existing nodes.

The database response time depends on factors such as type and size of the application query, database design, data size and data types stored, available computational power, and network bandwidth. Adding nodes to a database cluster does not necessarily improve the system response time for every query, especially if the response time is already short or not hardware-bound.
Make the database K-safe (K-safety=1) or increase K-safety to 2. See Failure recovery for details.
Swap or replace hardware. Swap out a node to perform maintenance or hardware upgrades.

Important

If you install Vertica on a single node without specifying the IP address or host name (or you used localhost), you cannot expand the cluster. You must reinstall Vertica and specify an IP address or host name that is not localhost/127.0.0.1.

Adding nodes consists of the following general tasks:

Back up the database.

Vertica strongly recommends that you back up the database before you perform this significant operation because it entails creating new projections, refreshing them, and then deleting the old projections. See Backing up and restoring the database for more information.

The process of migrating the projection design to include the additional nodes could take a while; however during this time, all user activity on the database can proceed normally, using the old projections.
Configure the hosts you want to add to the cluster.

See Before you Install Vertica. You will also need to edit the hosts configuration file on all of the existing nodes in the cluster to ensure they can resolve the new host.
Add one or more hosts to the cluster.
Add the hosts you added to the cluster (in step 3) to the database.

When you add a host to the database, it becomes a node. You can add nodes to your database using either the Administration tools or the Management Console (See Monitoring with MC). Adding nodes using admintools preserves the specific order of the nodes you add.

After you add nodes to the database, Vertica automatically distributes updated configuration files to the rest of the nodes in the cluster and starts the process of rebalancing data in the cluster. See Rebalancing data across nodes for details.

If you have previously created storage locations using CREATE LOCATION...ALL NODES, you must create those locations on the new nodes.

1 - Adding hosts to a cluster

After you have backed up the database and configured the hosts you want to add to the cluster, you can now add hosts to the cluster using the script.

After you have backed up the database and configured the hosts you want to add to the cluster, you can now add hosts to the cluster using the update_vertica script.

You cannot use the MC to add hosts to a cluster in an on-premises environment. However, after the hosts are added to the cluster, the MC does allow you to add the hosts to a database as nodes.

Prerequisites and restrictions

If you installed Vertica on a single node without specifying the IP address or hostname (you used localhost), it is not possible to expand the cluster. You must reinstall Vertica and specify an IP address or hostname.

Procedure to add hosts

From one of the existing cluster hosts, run the update_vertica script with a minimum of the --add-hosts host(s) parameter (where host(s) is the hostname or IP address of the system(s) that you are adding to the cluster) and the --rpm or --deb parameter:

# /opt/vertica/sbin/update_vertica --add-hosts host(s) --rpm package

Note

See Install Vertica with the installation script for the full list of parameters. You must also provide the same options you used when originally installing the cluster.

The update_vertica ** script uses all the same options as install_vertica and:

Installs the Vertica RPM on the new host.
Performs post-installation checks, including RPM version and N-way network connectivity checks.
Modifies spread to encompass the larger cluster.
Configures the Administration Tools to work with the larger cluster.

Important Tips:

Consider using --large-cluster with more than 50 nodes.
A host can be specified by the hostname or IP address of the system you are adding to the cluster. However, internally Vertica stores all host addresses as IP addresses.
Do not use include spaces in the hostname/IP address list provided with --add-hosts if you specified more than one host.
If a package is specified with --rpm/--deb, and that package is newer than the one currently installed on the existing cluster, then, Vertica first installs the new package on the existing cluster hosts before the newly-added hosts.
Use the same command line parameters for the database administrator username, password, and directory path you used when you installed the cluster originally. Alternatively, you can create a properties file to save the parameters during install and then re-using it on subsequent install and update operations. See Installing Vertica Silently.
If you are installing using sudo, the database administrator user (dbadmin) must already exist on the hosts you are adding and must be configured with passwords and home directory paths identical to the existing hosts. Vertica sets up passwordless ssh from existing hosts to the new hosts, if needed.
If you initially used the --point-to-point option to configure spread to use direct, point-to-point communication between nodes on the subnet, then use the --point-to-point option whenever you run install_vertica or update_vertica. Otherwise, your cluster's configuration is reverted to the default (broadcast), which may impact future databases.
The maximum number of spread daemons supported in point-to-point communication and broadcast traffic is 80. It is possible to have more than 80 nodes by using large cluster mode, which does not install a spread daemon on each node.

Examples

--add-hosts host01 --rpm
--add-hosts 192.168.233.101
--add-hosts host02,host03

2 - Adding nodes to a database

After you add one or more hosts to the cluster, you can add them as nodes to the database with one of the following:.

After you add one or more hosts to the cluster, you can add them as nodes to the database with one of the following:

admintools command line, to ensure nodes are added in a specific order
Administration Tools
Management Console

If you have previously created storage locations using CREATE LOCATION...ALL NODES, you must create those locations on the new nodes.

Command line

With the admintools db_add_node tool, you can control the order in which nodes are added to the database cluster. It specifies the hosts of new nodes with its -s or --hosts option, which takes a comma-delimited argument list. Vertica adds new nodes in the list-specified order. For example, the following command adds three nodes:

$ admintools -t db_add_node \
      -d VMart \
      -p 'password' \
      -s 192.0.2.1,192.0.2.2,192.0.2.3

Tip

When adding nodes to an Eon Mode database, you can also specify the subcluster that the new nodes should belong to. See Adding and removing nodes from subclusters for more information.

Administration tools

You add nodes to a database with the Administration Tools as follows:

Open the Administration Tools.
On the Main Menu, select View Database Cluster State to verify that the database is running. If it is not, start it.
From the Main Menu, select Advanced Menu and click OK.
In the Advanced Menu, select Cluster Management and click OK.
In the Cluster Management menu, select Add Host(s) and click OK.
Select the database to which you want to add one or more hosts, and then select OK.

A list of unused hosts is displayed.
Select the hosts you want to add to the database and click OK.
When prompted, click Yes to confirm that you want to add the hosts.
When prompted, enter the password for the database, and then select OK.
When prompted that the hosts were successfully added, select OK.
Vertica now automatically starts the rebalancing process to populate the new node with data. When prompted, enter the path to a temporary directory that the Database Designer can use to rebalance the data in the database and select OK.
Either press Enter to accept the default K-safety value, or enter a new higher value for the database and select OK.
Select whether to rebalance the database immediately, or later. In both cases, Vertica creates a script, which you can use to rebalance at any time.

Review the summary of the rebalancing process and select Proceed.

If you choose to automatically rebalance, the rebalance process runs. If you chose to create a script, the script is generated and saved. In either case, you are shown a success screen.
Select OK to complete the Add Node process.

Management Console

To add nodes to an Eon Mode database using MC, see Add nodes to a cluster in AWS using Management Console.

To add hosts to an Enterprise Mode database using MC, see Adding hosts to a cluster

3 - Add nodes to a cluster in AWS

This section gives an overview on how to add nodes if you are managing your cluster using admintools.

This section gives an overview on how to add nodes if you are managing your cluster using admintools. Each main step points to another topic with the complete instructions.

Step 1: before you start

Before you add nodes to a cluster, verify that you have an AWS cluster up and running and that you have:

Created a database.
Defined a database schema.
Loaded data.
Run the Database Designer.
Connected to your database.

Step 2: launch new instances to add to an existing cluster

Perform the procedure in Configure and launch an instance to create new instances (hosts) that you then will add to your existing cluster. Be sure to choose the same details you chose when you created the original instances (VPC, placement group, subnet, and security group).

Step 3: include new instances as cluster nodes

You need the IP addresses when you run the install_vertica script to include new instances as cluster nodes.

If you are configuring Amazon Elastic Block Store (EBS) volumes, be sure to configure the volumes on the node before you add the node to your cluster.

To add the new instances as nodes to your existing cluster:

Configure and launch your new instances.
Connect to the instance that is assigned to the Elastic IP. See Connect to an instance if you need more information.

Run the Vertica installation script to add the new instances as nodes to your cluster. Specify the internal IP addresses for your instances and your *.pem file name.

$ sudo /opt/vertica/sbin/install_vertica --add-hosts instance-ip --dba-user-password-disabled \
  --point-to-point --data-dir /vertica/data --ssh-identity ~/name-of-pem.pem

Step 4: add the nodes

After you have added the new instances to your existing cluster, add them as nodes to your cluster, as described in Adding nodes to a database.

Step 5: rebalance the database

After you add nodes to a database, always rebalance the database.