Reconfiguring node messaging

Sometimes, nodes of an existing, operational Vertica database cluster require new IP addresses.

Sometimes, nodes of an existing, operational Vertica database cluster require new IP addresses. Cluster nodes might also need to change their messaging protocols—for example, from broadcast to point-to-point. The admintools re_ip utility performs both tasks.

Changing IP addresses

You can use re_ip to perform two tasks:

In both cases, re_ip requires a mapping file that identifies the current node IP addresses, which are stored in admintools.conf. You can get these addresses in two ways:

  • Use the admintools utility list_allnodes:

    $ admintools -t list_allnodes
Node             | Host          | State | Version        | DB
-----------------+---------------+-------+----------------+-----------
v_vmart_node0001 | 192.0.2.254   | UP    | vertica-12.0.1 | VMart
v_vmart_node0002 | 192.0.2.255   | UP    | vertica-12.0.1 | VMart
v_vmart_node0003 | 192.0.2.256   | UP    | vertica-12.0.1 | VMart

  • Print the content of admintools.conf:

    $ cat /opt/vertica/config/admintools.conf
...
[Cluster]
hosts = 192.0.2.254, 192.0.2.255, 192.0.2.256

[Nodes]
node0001 = 192.0.2.254/home/dbadmin,/home/dbadmin
node0002 = 192.0.2.255/home/dbadmin,/home/dbadmin
node0003 = 192.0.2.256/home/dbadmin,/home/dbadmin
...

Update node IP addresses

You can update IP addresses with re_ip as described below. re_ip automatically backs up admintools.conf so you can recover the original settings if necessary.

  1. Create a mapping file with lines in the following format:

    oldIPaddress newIPaddress[, controlAddress, controlBroadcast]
...

    For example:

    192.0.2.254 198.51.100.255, 198.51.100.255, 203.0.113.255
192.0.2.255 198.51.100.256, 198.51.100.256, 203.0.113.255
192.0.2.256 198.51.100.257, 198.51.100.257, 203.0.113.255

    controlAddress and controlBroadcast are optional. If omitted:

    • controlAddress defaults to newIPaddress.

    • controlBroadcast defaults to the host of newIPaddress’s broadcast IP address.

  2. Stop the database.

  3. Run re_ip to map old IP addresses to new IP addresses:

    $ admintools -t re_ip -f mapfile

    re_ip issues warnings for the following mapping file errors:

    • IP addresses are incorrectly formatted.

    • Duplicate IP addresses, whether old or new.

    If re_ip finds no syntax errors, it performs the following tasks:

    • Remaps the IP addresses as listed in the mapping file.

    • If the -i option is omitted, asks to confirm updates to the database.

    • Updates required local configuration files with the new IP addresses.

    • Distributes the updated configuration files to the hosts using new IP addresses.

    For example:

    Parsing mapfile...
New settings for Host 192.0.2.254 are:
address: 198.51.100.255
New settings for Host 192.0.2.255 are:
address: 198.51.100.256
New settings for Host 192.0.2.254 are:
address: 198.51.100.257

The following databases would be affected by this tool: Vmart

Checking DB status ...
Enter "yes" to write new settings or "no" to exit > yes
Backing up local admintools.conf ...
Writing new settings to local admintools.conf ...

Writing new settings to the catalogs of database Vmart ...
The change was applied to all nodes.
Success. Change committed on a quorum of nodes.

Initiating admintools.conf distribution ...
Success. Local admintools.conf sent to all hosts in the cluster.

  4. Restart the database.

re_ip and export IP address

By default, a node's IP address and its export IP address are identical. For example:

=> SELECT node_name, node_address, export_address FROM nodes;
    node_name      | node_address    | export_address
------------------------------------------------------
v_VMartDB_node0001 | 192.168.100.101 | 192.168.100.101
v_VMartDB_node0002 | 192.168.100.102 | 192.168.100.101
v_VMartDB_node0003 | 192.168.100.103 | 192.168.100.101
v_VMartDB_node0004 | 192.168.100.104 | 192.168.100.101
(4 rows)

The export address is the IP address of the node on the network. This address provides access to other DBMS systems, and enables you to import and export data across the network.

If node IP and export IP addresses are the same, then running re_ip changes both to the new address. Conversely, if you manually change the export address, subsequent re_ip operations leave your export address changes untouched.

Change node control and broadcast addresses

You can map IP addresses for the database only, by using the re_ip option -O (or --db-only). Database-only operations are useful for error recovery. The node names and IP addresses that are specified in the mapping file must be the same as the node information in admintools.conf. In this case, admintools.conf is not updated. Vertica updates only spread.conf and the catalog with the changes.

You can also use re_ip to change the node control and broadcast addresses. In this case the mapping file must contain the control messaging IP address and associated broadcast address. This task allows nodes on the same host to have different data and control addresses.

  1. Create a mapping file with lines in the following format:

    nodeName nodeIPaddress, controlAddress, controlBroadcast
    ...

    For example:

    vertica_node001 192.0.2.254, 203.0.113.255, 203.0.113.258
vertica_node002 192.0.2.255, 203.0.113.256, 203.0.113.258
vertica_node003 192.0.2.256, 203.0.113.257, 203.0.113.258

  2. Stop the database.

  3. Run the following command to map the new IP addresses:

    $ admintools -t re_ip -f mapfile -O -d dbname

  4. Restart the database.

Changing node messaging protocols

You can use re_ip to reconfigure spread messaging between Vertica nodes. re_ip configures node messaging to broadcast or point-to-point (unicast) messaging with these options:

  • -U, --broadcast (default)

  • -T, --point-to-point

Both options support up to 80 spread daemons. You can exceed the 80-node limit by using large cluster mode, which does not install a spread daemon on each node.

For example, to set the database cluster messaging protocol to point-to-point:

$ admintools -t re_ip -d dbname -T

To set the messaging protocol to broadcast:

$ admintools -t re_ip -d dbname -U

Setting re_ip timeout

You can configure how long re_ip executes a given task before it times out, by editing the setting of prepare_timeout_sec in admintools.conf. By default, this parameter is set to 7200 (seconds).

In this section