Complete all of the tasks in this section before you install Vertica. When you have completed this section, proceed to Install Vertica using the command line.
This is the multi-page printable view of this section. Click here to print.
Before you install Vertica
- 1: Platform and hardware requirements and recommendations
- 2: Communal storage for on-premises Eon Mode databases
- 3: Configure the network
- 3.1: Reserved ports
- 3.2: Firewall considerations
- 4: Operating system configuration overview
- 5: Automatically configured operating system settings
- 5.1: Sysctl
- 5.2: Nice limits configuration
- 5.3: min_free_kbytes setting
- 5.4: User max open files limit
- 5.5: System max open files limit
- 5.6: Pam limits
- 5.7: pid_max setting
- 5.8: User address space limits
- 5.9: User file size limit
- 5.10: User process limit
- 5.11: Maximum memory maps configuration
- 6: Manually configured operating system settings
- 6.1: SUSE control groups configuration
- 6.2: Cron required for scheduled jobs
- 6.3: Disk readahead
- 6.4: I/O scheduling
- 6.5: Enabling or disabling transparent hugepages
- 6.6: Check for swappiness
- 6.7: Enabling network time protocol (NTP)
- 6.8: Enabling chrony or ntpd for red hat 7/CentOS 7 systems
- 6.9: SELinux configuration
- 6.10: CPU frequency scaling
- 6.11: Enabling or disabling defrag
- 6.12: Support tools
- 7: System user configuration
1 - Platform and hardware requirements and recommendations
Hardware recommendations
The Vertica Analytics Platform is based on a massively parallel processing (MPP), shared-nothing architecture, in which the query processing workload is divided among all nodes of the Vertica database. OpenText highly recommends using a homogeneous hardware configuration for your Vertica cluster; that is, each node of the cluster should be similar in CPU, clock speed, number of cores, memory, and operating system version.
Note that OpenText has not tested Vertica on clusters made up of nodes with disparate hardware specifications. While it is expected that a Vertica database would functionally work in a mixed hardware configuration, performance will be limited to that of the slowest node in the cluster.
Vertica performs best on processors with higher clock frequency. When possible, choose a faster processor with fewer cores as opposed to a slower processor with more cores.
Tests performed both internally and by customers have shown performance differences between processor architectures even when accounting for differences in core count and clock frequency. When possible, compare platforms by installing Vertica and running experiments using your data and workloads. Consider testing on cloud platforms that offer VMs running on different processor architectures, even if you intend to deploy your Vertica database on premises.
Detailed hardware recommendations are available in Recommendations for Sizing Vertica Nodes and Clusters.
Platform requirements and recommnedations
You must verify that your servers meet the platform requirements described in Supported Platforms. The Supported Platforms topics detail supported versions for the following:
-
OS for Server and Management Console (MC)
-
Supported Browsers for MC
-
Supported File Systems
Important
Deploy Vertica as the only active process on each host—other than Linux processes or software explicitly approved by Vertica. Vertica cannot be co-located with other software. Remove or disable all non-essential applications from cluster hosts.Install the latest vendor-specific system software
Install the latest vendor drivers for your hardware.
Data storage recommendations
-
All internal drives connect to a single RAID controller.
-
The RAID array should form one hardware RAID device as a contiguous /data volume.
Install Perl
Before you perform the cluster installation, install Perl 5 on all the target hosts. Perl is available for download from www.perl.org.
Validation utilities
Vertica provides several validation utilities that validate the performance on prospective hosts. The utilities are installed when you install the Vertica RPM, but you can use them before you run the install_vertica
script. See Validation scripts for more details on running the utilities and verifying that your hosts meet the recommended requirements.
Verify sudo
Vertica uses the sudo command during installation and some administrative tasks. Ensure that sudo is available on all hosts with the following command:
# which sudo
/usr/bin/sudo
If sudo is not installed, on all hosts, follow the instructions in How to Enable sudo on Red Hat Enterprise Linux.
When you use sudo to install Vertica, the user that performs the installation must have privileges on all nodes in the cluster.
Configuring sudo with privileges for the individual commands can be a tedious and error-prone process; thus, the Vertica documentation does not include every possible sudo command that you can include in the sudoers file. Instead, Vertica recommends that you temporarily elevate the sudo user to have all privileges for the duration of the install.
Note
See the sudoers and visudo man pages for the details on how to write/modify a sudoers file.To allow root sudo access on all commands as any user on any machine, use visudo as root to edit the /etc/sudoers
file and add this line:
## Allow root to run any commands anywhere
root ALL=(ALL) ALL
After the installation completes, remove (or reset) sudo privileges to the pre-installation settings.
BASH shell requirements
All shell scripts included in Vertica must run under the BASH shell. If you are on a Debian system, then the default shell can be DASH. DASH is not supported. Change the shell for root and for the dbadmin user to BASH with the chsh
command.
For example:
# getent passwd | grep root
root:x:0:0:root:/root:/bin/dash
# chsh
Changing shell for root.
New shell [/bin/dash]: /bin/bash
Shell changed.
Then, as root, change the symbolic link for /bin/sh
from /bin/dash
to /bin/bash
:
# rm /bin/sh
# ln -s /bin/bash /bin/sh
Log out and back in for the change to take effect.
2 - Communal storage for on-premises Eon Mode databases
If you create an Eon Mode database, you must plan for your use of communal storage to store your database's data. Communal storage is based on a shared storage, such as AWS S3 or Pure Storage FlashBlade servers.
Whatever communal storage platform you use, you must ensure that it is durable (protected against data loss). The data in your Eon Mode database is only as safe as the communal storage that contains it. Most cloud provider's object stores come with a guaranteed redundancy to prevent data loss. When you install an Eon Mode database on-premises, you may have to take additional steps to prevent data loss.
Planning communal storage capacity for on-premises databases
Most cloud providers do not limit the amount of data you can store in their object stores. The only real limit is your budget; storing more data costs more money.
When you create an Eon Mode database on-premises, your storage is limited to the size of your communal storage. Unlike the cloud, you must plan ahead for the amount of storage you will need. For example, if you have a Pure Admin FlashBlade installation with three 8TB blades, then in theory, your database can grow up to 24TB. In practice, you need to account other uses of your object store, as well as factors such as data compression, and space consumed by unreaped ROS containers (storage containers no longer used by Vertica but not yet deleted by the object store).
The following calculator helps you determine the size for your communal storage needs, based on your estimated data size and additional uses of your communal storage. The values with white backgrounds in the Value column are editable. Change them to reflect your environment.
Note
The calculator currently does not work in mobile browsers. Please use a desktop browser to view the calculator.3 - Configure the network
This group of steps involve configuring the network. These steps differ depending on your installation scenario. A single node installation requires little network configuration, because the single instance of the Vertica server does not need to communication with other nodes in a cluster. For cluster install scenarios, you must make several decisions regarding your configuration.
Vertica supports server configuration with multiple network interfaces. For example, you might want to use one as a private network interface for internal communication among cluster hosts (the ones supplied via the --hosts
option to install_vertica
) and a separate one for client connections.
Important
Vertica performs best when all nodes are on the same subnet and have the same broadcast address for one or more interfaces. A cluster that has nodes on more than one subnet can experience lower performance due to the network latency associated with a multi-subnet system at high network utilization levels.Important notes
-
Network configuration is exactly the same for single nodes as for multi-node clusters, with one special exception. If you install Vertica on a single host machine that is to remain a permanent single-node configuration (such as for development or Proof of Concept), you can install Vertica using
localhost
or the loopback IP (typically 127.0.0.1) as the value for--hosts
. Do not use the hostnamelocalhost
in a node definition if you are likely to add nodes to the configuration later. -
If you are using a host with multiple network interfaces, configure Vertica to use the address which is assigned to the NIC that is connected to the other cluster hosts.
-
Use a dedicated gigabit switch. If you do not performance could be severely affected.
-
Do not use DHCP dynamically-assigned IP addresses for the private network. Use only static addresses or permanently-leased DHCP addresses.
Choose IPv4 or IPv6 addresses for host identification and communications
Vertica supports using either IPv4 or IPv6 IP addresses for identifying the hosts in a database cluster. Vertica uses a single address to identify a host in the database cluster. All the IP addresses used to identify hosts in the cluster must use the same IP family.
The hosts in your database cluster can have both IPv4 and IPv6 network addresses assigned to them. Only one of these addresses is used to identify the node within the cluster. You can use the other addresses to handle client connections or connections to other systems.
You tell Vertica which address family to use when you install it. By default, Vertica uses IPv4 addresses for hosts. If you want the nodes in your database to use IPv6 addresses, add the --ipv6
option to the arguments you pass to the install_vertica
script.
Note
You cannot change the address family a database cluster uses after you create it. For example, suppose you created a Vertica database using IPv4 addresses to identify the hosts in your cluster. Then you cannot later change the hosts to use an IPv6 address for internal communications.In most cases, the address family you select does not impact how your database functions. However, there are a few exceptions:
-
Use IPv4 addresses to identify the nodes in your cluster if you want to use the Management Console to manage your database. Currently, the MC does not support databases that use IPv6 addresses.
-
If you select IPv6 addressing for your cluster, it automatically uses point-to-point networking mode.
-
Currently, AWS is the only cloud platform on which Vertica supports IPv6 addressing. To use IPv6 on AWS, you must identify cluster hosts using IP addresses instead of host names. The AWS DNS does not support resolving host names to IPv6.
-
If you only assign IPv6 addresses to the hosts in your database cluster, you may have problems interfacing to other systems that do not support IPv6.
Part of the information you pass to the install script is the list of hosts it will use to form the Vertica cluster. If you use host names in this list instead of IP addresses, ensure that the host names resolve to the IP address family you want to use for your cluster. For example, if you want your cluster to use IPv6 addresses, ensure your DNS or /etc/hosts
file resolves the host names to IPv6 addresses.
You can configure DNS to return both IPv4 and IPv6 addresses for a host name. In this case, the installer uses the IPv4 address unless you supply the --ipv6
argument. If you use /etc/hosts
for host name resolution (which is the best practice), host names cannot resolve to both IPv4 and IPv6 addresses.
Optionally run spread on a separate control network
If your query workloads are network intensive, you can use the --control-network
parameter with the
install_vertica
script (see Install Vertica with the installation script) to allow spread communications to be configured on a subnet that is different from other Vertica data communications.
The --control-network
parameter accepts either the default
value or a broadcast network IP address (for example, 192.168.10.255
).
Configure SSH
-
Verify that root can use Secure Shell (SSH) to log in (ssh) to all hosts that are included in the cluster. SSH (SSH client) is a program for logging into a remote machine and for running commands on a remote machine.
-
If you do not already have SSH installed on all hosts, log in as root on each host and install it before installing Vertica. You can download a free version of the SSH connectivity tools from OpenSSH.
-
Make sure that
/dev/pts
is mounted. Installing Vertica on a host that is missing the mount point/dev/pts
could result in the following error when you create a database:
TIMEOUT ERROR: Could not login with SSH. Here is what SSH said:Last login: Sat Dec 15 18:05:35 2007 from v_vmart_node0001
Allow passwordless SSH access for the dbadmin user
The dbadmin user must be authorized for passwordless ssh. In typical installs, you won't need to change anything; however, if you set up your system to disallow passwordless login, you'll need to enable it for the dbadmin user. See Enable secure shell (SSH) logins.
3.1 - Reserved ports
N0020
.The install_vertica script checks that required ports are open and available to Vertica. The installer reports any issues with identifier N0020
.
You can also verify that ports required by Vertica are not in use by running the following command as the root user and comparing it with the ports required, as shown below:
$ netstat -atupn
If you are using a Red Hat 7/CentOS 7 system, use the following command instead:
$ ss -atupn
Firewall requirements
Vertica requires several ports to be open on the local network. Vertica does not recommend placing a firewall between nodes (all nodes should be behind a firewall), but if you must use a firewall between nodes, ensure the following ports are available:
Port | Protocol | Service | Notes |
---|---|---|---|
22 | TCP | sshd | Required by Administration tools and the Management Console Cluster Installation wizard. |
5433 | TCP | Vertica | Vertica client (vsql, ODBC, JDBC, etc) port. |
5434 | TCP | Vertica | Intra- and inter-cluster communication. Vertica opens the Vertica client port +1 (5434 by default) for intra-cluster communication, such as during a plan. If the port +1 from the default client port is not available, then Vertica opens a random port for intra-cluster communication. |
5433 | UDP | Vertica | Vertica spread monitoring and MC cluster import. |
5444 | TCP |
Vertica Management Console |
MC-to-node and node-to-node (agent) communications port. See Changing MC or agent ports. |
5450 | TCP |
Vertica Management Console |
Port used to connect to MC from a web browser and allows communication from nodes to the MC application/web server. See Connecting to Management Console. |
4803 | TCP | Spread | Client connections. |
8443 | TCP | HTTPS | HTTPS service. To change the port, use HTTPServerPortOffset. |
4803 | UDP | Spread | Daemon to daemon connections. |
4804 | UDP | Spread | Daemon to daemon connections. |
6543 | UDP | Spread | Monitor to daemon connection. |
3.2 - Firewall considerations
Vertica requires multiple ports be open between nodes. You may use a firewall (IP Tables) on Redhat/CentOS and Ubuntu/Debian based systems. Note that firewall use is not supported on SuSE systems and that SuSE systems must disable the firewall. The installer reports issues found with your IP tables configuration with the identifiers N0010 for (systems that use IP Tables) and N011 (for SuSE systems).
The installer checks the IP tables configuration and issues a warning if there are any configured rules or chains. The installer does not detect if the configuration may conflict with Vertica. It is your responsibility to verify that your firewall allows traffic for Vertica as described in Reserved ports.
Note
The installer does not check NAT entries in iptables.You can modify your firewall to allow for Vertica network traffic, or you can disable the firewall if your network is secure. Note that firewalls are not supported for Vertica systems running on SuSE.
Important
You may encounter the N0010 issue even when the firewall is disabled. If this occurs, you can workaround this issue and install Vertica by ignoring installer WARN messages. To do this, install (or update) with a failure threshold of FAIL. For example,/opt/vertica/sbin/install_vertica --failure-threshold FAIL <other install options...>
.
Red hat 6 and CentOS 6 systems
For details on how to configure iptables and allow specific ports to be open, see the platform-specific documentation for your platform:
- RedHat: https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sect-Security_Guide-IPTables.html
- CentOS: http://wiki.centos.org/HowTos/Network/IPTables
To disable iptables, run the following command as root or sudo:
# service iptables save
# service iptables stop
# chkconfig iptables off
To disable iptables if you are using the ipv6 versions of iptables, run the following command as root or sudo:
# service ip6tables save
# service ip6tables stop
# chkconfig ip6tables off
Red hat 7 and CentOS 7 systems:
To disable the system firewall, run the following command as root or sudo:
# systemctl mask firewalld
# systemctl disable firewalld
# systemctl stop firewalld
Ubuntu and debian systems
For details on how to configure iptables and allow specific ports to be open, see the platform-specific documentation for your platform:
-
Debian: https://wiki.debian.org/iptables
-
Ubuntu: https://help.ubuntu.com/12.04/serverguide/firewall.html.
Note
Ubuntu uses the ufw program to manage iptables.To disable iptables on Debian, run the following command as root or sudo:
$ /etc/init.d/iptables stop
$ update-rc.d -f iptables remove
To disable iptables on Ubuntu, run the following command:
$ sudo ufw disable
SuSE systems
The firewall must be disabled on SUSE systems. To disable the firewall on SuSE systems, run the following command:
# /sbin/SuSEfirewall2 off
4 - Operating system configuration overview
This topic provides a high-level overview of the OS settings required for Vertica. Each item provides a link to additional details about the setting and detailed steps on making the configuration change. The installer tests for all of these settings and provides hints, warnings, and failures if the current configuration does not meet Vertica requirements.
Before you install the operating system
The below sections detail system settings that must be configured when you install the operating system. These settings cannot be easily changed after the operating system is installed.
Configuration | Description |
---|---|
Supported Platforms |
Verify that your servers meet the platform requirements described in Supported Platforms. Unsupported operating systems are detected by the installer. The installer generates one of the following issue identifiers if it detects an unsupported operating system:
|
LVM | Vertica Analytic Database supports Linux Volume Manager (LVM) on all supported operating systems. For information on LVM requirements and restrictions, see the section, Vertica Support for LVM. |
File system |
Choose the storage format type based on deployment requirements. Vertica recommends the following storage format types where applicable:
NoteFor the Vertica I/O profile, the ext4 file system is considerably faster than ext3. The storage format type at your backup and temporary directory locations must support fcntl lockf (POSIX) file locking. |
Swap Space |
A 2GB swap partition is required, regardless of the amount of RAM installed on your system. Larger swap space is acceptable, but unnecessary. Partition the remaining disk space in a single partition under "/". If you do not have the required 2GB swap partition, the installer reports this issue with identifier S0180. You typically define the swap partition when you install Linux. See your platform’s documentation for details on configuring the swap partition. NoteDo not place a swap file on a disk containing the Vertica data files. If a host has only two disks (boot and data), put the swap file on the boot disk. |
Disk Block Size | The disk block size for the Vertica data and catalog directories should be 4096 bytes, the default on ext4 and XFS file systems. You set the disk block size when you format your file system. If you change the block size, you will need to reformat the disk. |
Memory |
Vertica requires that your hosts have a minimum of 1GB of RAM per logical processor. If your hosts do not meet this requirement, the installer reports this issue with the identifier S0190. For performance reasons, you typically require more RAM than the minimum. In addition to the individual host RAM requirement, the installer also reports a hint if the hosts in your cluster do not have identical amounts of RAM. Ensuring your host have the same amount of RAM helps prevent performance issues if one or more nodes has less RAM than the other nodes in your database. NoteIn an Eon Mode database, after you create the initial cluster, you can configure subclusters that have different hardware specifications (including RAM) than the initial primary subcluster the installer creates. For more information on sizing your hardware, see the Vertica Knowledge Base Hardware documents. |
Automatically configured operating system settings
These general OS settings are automatically made by the installer if they do not meet Vertica requirements. You can prevent the installer from automatically making these configuration changes by using the --no-system-configuration
parameter for the install_vertica
script.
For more information on each configuration setting, see Automatically configured operating system settings.
Configuration | Description |
---|---|
Nice Limits | The database administration user must be able to nice processes back to the default level of 0. |
min_free_kbytes |
The vm.min_free_kbytes setting in /etc/sysctl.conf must be configured sufficiently high. The specific value depends on your hardware configuration. |
User Open Files Limit | The open file limit for the dbadmin user should be at least 1 file open per MB of RAM, 65536, or the amount of RAM in MB; whichever is greater. |
System Open File Limits | The maximum number of files open on the system must not be less than at least the amount of memory in MB, but not less than 65536. |
Pam Limits |
/etc/pam.d/su must contain the line: This allows for the conveying of limits to commands run with the |
Address Space Limits |
The address space limits (as setting) defined in /etc/security/limits.conf must be unlimited for the database administrator. |
File Size Limits |
The file sizelimits (fsize setting) defined in /etc/security/limits.conf must be unlimited for the database administrator. |
User Process Limits |
The nproc setting defined in /etc/security/limits.conf must be 1024 or the amount of memory in MB, whichever is greater. |
Maximum Memory Maps | The vm.max_map_count in /etc/sysctl.conf must be 65536 or the amount of memory in KB / 16, whichever is greater. |
Manually configured operating system settings
For more information on each configuration setting, see Manually configured operating system settings.
Configuration | Description |
---|---|
Disk Readahead | This disk readahead must be at least 2048, with a high of 8192. Set this high limit only with the help of Vertica support. The specific value depends on your hardware configuration. |
NTP Services | The NTP daemon must be enabled and running, with the exception of Red Hat 7 and CentOS 7 systems. |
chrony | For Red Hat 7 and CentOS 7 systems, chrony must be enabled and running. |
SELinux | SElinux must be disabled or run in permissive mode. |
CPU Frequency Scaling |
Vertica recommends that you disable CPU Frequency Scaling. ImportantYour systems may use significantly more energy when CPU frequency scaling is disabled. |
Transparent Hugepages |
For Red Hat 7, CentOS 7 and Amazon Linux 2.0, Transparent Hugepages must be set to always. For all other operating systems, Transparent Hugepages must be disabled or set to madvise. |
I/O Scheduler | The I/O Scheduler for disks used by Vertica must be set to deadline or noop. |
Support Tools | Several optional packages can be installed to assist Vertica support when troubleshooting your system. |
System user requirements
The following tasks pertain to the configuration of the system user required by Vertica.
For more information on each configuration setting, see System user configuration.
Configuration | Required Setting(s) |
---|---|
System User Requirements |
The installer automatically creates a user with the correct settings. If you specify a user with --dba-use r, then the user must conform to the requirements for the Vertica system user. |
LANG Environment Settings | The LANG environment variable must be set and valid for the database administration user. |
TZ Environment Settings | The TZ environment variable must be set and valid for the database administration user. |
5 - Automatically configured operating system settings
These general Operating System settings are automatically made by the installer. You can prevent the installer from automatically making these configuration changes by using the --no-system-configuration
parameter for the install_vertica
script.
5.1 - Sysctl
During installation, Vertica attempts to automatically change various OS level settings. The installer may not change values on your system if they exceed the threshold required by the installer. You can prevent the installer from automatically making these configuration changes by using the --no-system-configuration
parameter for the install_vertica
script.
To permanently edit certain settings and prevent them from reverting on reboot, use sysctl.
The sysctl settings relevant to the installation of Vertica include:
Permanently changing settings with sysctl:
-
As the root user, open the /etc/sysctl.conf file:
# vi /etc/sysctl.conf
-
Enter a parameter and value:
parameter = value
For example, to set the parameter and value for
fs.file-max
to meet Vertica requirements, enter:fs.file-max = 65536
-
Save your changes, and close the /etc/sysctl.conf file.
-
As the root user, reload the config file:
# sysctl -p
Identifying settings added by the installer
You can see whether the installer has added a setting by opening the /etc/sysctl.conf file:
# vi /etc/sysctl.conf
If the installer has added a setting, the following line appears:
# The following 1 line added by Vertica tools. 2015-02-23 13:20:29
parameter = value
5.2 - Nice limits configuration
The Vertica system user (dbadmin by default) must be able to raise and lower the priority of Vertica processes. To do this, the nice option in the /etc/security/limits.conf
file must include an entry for the dbadmin user. The installer reports this issue with the identifier: S0010.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
Note
Vertica never raises priority above the default level of 0. However, Vertica does lower the priority of certain Vertica threads and needs to able to raise the priority of these threads back up to the default level. This setting allows Vertica to raise the priorities back to the default level.All systems
To set the Nice Limit configuration for the dbadmin user, edit /etc/security/limits.conf
and add the following line. Replace dbadmin with the name of your system user.
dbadmin - nice 0
5.3 - min_free_kbytes setting
This topic details how to update the min_free_kbytes setting so that it is within the range supported by Vertica. The installer reports this issue with the identifier: S0050 if the setting is too low, or S0051 if the setting is too high.
The vm.min_free_kbytes setting configures the page reclaim thresholds. When this number is increased the system starts reclaiming memory earlier, when its lowered it starts reclaiming memory later. The default min_free_kbytes is calculated at boot time based on the number of pages of physical RAM available on the system.
The setting must be whichever value is the greatest from the following options:
-
The default value configured by the system
-
4096
-
The result of running the commands:
$ memtot=`grep MemTotal /proc/meminfo | awk '{printf "%.0f",$2}'` $ echo "scale=0;sqrt ($memtot*16)" | bc
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
All systems
To manually set min_free_kbytes:
-
Determine the current/default setting with the following command:
$ sysctl vm.min_free_kbytes
-
If the result of the previous command is
No such file or directory
or the default value is less than 4096, then run these commands to determine the correct value:$ memtot=`grep MemTotal /proc/meminfo | awk '{printf "%.0f",$2}'` $ echo "scale=0;sqrt ($memtot*16)" | bc
-
Edit or add the current value of
vm.min_free_kbytes
in/etc/sysctl.conf
with the value from the output of the previous command.# The min_free_kbytes setting vm.min_free_kbytes=16132
-
Run
sysctl -p
to apply the changes insysctl.conf
immediately.Note
These steps must be repeated for each node in the cluster.
5.4 - User max open files limit
This topic details how to change the user max open-files limit setting to meet Vertica requirements. The installer reports this issue with the identifier S0060.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
Vertica requires that the dbadmin user not be limited when opening files. The open file limit per user is calculated as follows:
user max open files = greater of { ≥ 65536 | ≤ RAM-MBs }
As a dbadmin user, you can determine the open file limit by running ulimit -n
. For example:
$ ulimit -n
65536
To manually set the limit, edit /etc/security/limits.conf
and edit/add the nofile
setting for the user who is configured as the database administrator—by default, dbadmin
. For example:
dbadmin - nofile 65536
The setting must be no less than 65536 MB, but not greater than the system value of fs.nr_open
. For example, the default value of fs.nr_open
value on Red Hat Enterprise Linux 9 is 1048576 MB.
Note
The system also has a limit on open files.5.5 - System max open files limit
This topic details how to modify the limit for the number of open files on your system so that it meets Vertica requirements. The installer reports this issue with the identifier: S0120.
Vertica opens many files. Some platforms have global limits on the number of open files. The open file limit must be set sufficiently high so as not to interfere with database operations.
The recommended value is at least the amount of memory in MB, but not less than 65536.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
All systems
To manually set the open file limit:
-
Run
/sbin/sysctl fs.file-max
to determine the current limit. -
If the limit is not 65536 or the amount of system memory in MB (whichever is higher), then edit or add
fs.file-max=
max number of files
to/etc/sysctl.conf
.# Controls the maximum number of open files fs.file-max=65536
-
Run
sysctl -p
to apply the changes insysctl.conf
immediately.
Note
These steps will need to be replicated for each node in the cluster.5.6 - Pam limits
This topic details how to enable the "su" pam_limits.so module required by Vertica. The installer reports issues with the setting with the identifier: S0070.
On some systems the pam module called pam_limits.so
is not set in the file /etc/pam.d/su
. When it is not set, it prevents the conveying of limits (such as open file descriptors) to any command started with su -
.
In particular, the Vertica init script would fail to start Vertica because it calls the Administration Tools to start a database with the su -
command. This problem was first noticed on Debian systems, but the configuration could be missing on other Linux distributions. See the pam_limits man page for more details.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
All systems
To manually configure this setting, append the following line to the /etc/pam.d/su
file:
session required pam_limits.so
See the pam_limits man page for more details: man pam_limits
.
5.7 - pid_max setting
This topic explains how to change pid_max
to a supported value. The value of pid_max
should be
pid_max = num-user-proc + 2**15 = num-user-proc + 32768
where num-user-proc
is the size of memory in megabytes.
The minimum value for pid_max
is 524288.
If your pid_max
value is too low, the installer reports this problem and indicates the minimum value.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
All systems
To change the pid_max
value:
# sysctl -w kernel.pid_max=524288
5.8 - User address space limits
This topic details how to modify the Linux address space limit for the dbadmin user so that it meets Vertica requirements. The address space setting controls the maximum number of threads and processes for each user. If this setting does not meet the requirements then the installer reports this issue with the identifier: S0090.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
The address space available to the dbadmin user must not be reduced via user limits and must be set to unlimited.
All systems
To manually set the address space limit:
-
Run
ulimit -v
as the dbadmin user to determine the current limit. -
If the limit is not unlimited, then add the following line to
/etc/security/limits.conf
. Replace dbadmin with your database admin user
dbadmin - as unlimited
5.9 - User file size limit
This topic details how to modify the file size limit for files on your system so that it meets Vertica requirements. The installer reports this issue with the identifier: S0100.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
The file size limit for the dbadmin user must not be reduced via user limits and must be set to unlimited.
All systems
To manually set the file size limit:
-
Run
ulimit -f
as the dbadmin user to determine the current limit. -
If the limit is not unlimited, then edit/add the following line to
/etc/security/limits.conf
. Replace dbadmin with your database admin user.
dbadmin - fsize unlimited
5.10 - User process limit
This topic details how to change the user process limit so that it meets Vertica requirements.The installer reports this issue with the identifier: S0110.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
The user process limit must be high enough to allow for the many threads opened by Vertica. The recommended limit is the amount of RAM in MB and must be at least 1024.
All systems
To manually set the user process limit:
-
Run
ulimit -u
as the dbadmin user to determine the current limit. -
If the limit is not the amount of memory in MB on the server, then edit/add the following line to
/etc/security/limits.conf
. Replace 4096 with the amount of system memory, in MB, on the server.
dbadmin - nproc 4096
5.11 - Maximum memory maps configuration
This topic details how to modify the limit for the number memory maps a process can have on your system so that it meets Vertica requirements. The installer reports this issue with the identifier: S0130.
The installer automatically configures the correct setting if the default value does not meet system requirements. If an issue occurs when setting this value, or you use the --no-system-configuration
argument to the installer and the current setting is incorrect, then the installer reports this as an issue.
Vertica uses a lot of memory while processing and can approach the default limit for memory maps per process.
The recommended value is at least the amount of memory on the system in KB / 16, but not less than 65536.
All systems
To manually set the memory map limit:
-
Run
/sbin/sysctl vm.max_map_count
to determine the current limit. -
If the limit is not 65536 or the amount of system memory in KB / 16 (whichever is higher), then edit/add the following line to
/etc/sysctl.conf
. Replace 65536 with the value for your system.# The following 1 line added by Vertica tools. 2014-03-07 13:20:31 vm.max_map_count=65536
-
Run
sysctl -p
to apply the changes insysctl.conf
immediately.
Note
These steps will need to be replicated for each node in the cluster.6 - Manually configured operating system settings
The topics in this section detail general Operating System settings that must be set manually.
Persisting operating system settings
To prevent manually set Operating System settings from reverting on reboot, you should configure some of these settings in the /etc/rc.local
script. This script contains commands and scripts that run each time the system is booted.
Important
On reboot, SUSE systems use the/etc/init.d/after.local
file rather than /etc/rc.local
.
Vertica uses settings in /etc/rc.local
to set the following functionality:
Editing /etc/rc.local
-
As the root user, open
/etc/rc.local
:# vi /etc/rc.local
-
Enter a script or command. For example, to configure transparent hugepages to meet Vertica requirements, enter the following:
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
Important
On some Ubuntu/Debian systems, the last line in/etc/rc.local
must beexit 0
. All additions to/etc/rc.local
must precede this line. -
Save your changes, and close
/etc/rc.local
. -
If you use Red Hat 7.0 or CentOS 7.0 or higher, run the following command as root or sudo:
$ chmod +x /etc/rc.d/rc.local
On reboot, the command runs during startup. You can also run the command manually as the root user, if you want it to take effect immediately.
Disabling tuning system service
If you use Red Hat 7.0 or CentOS 7.0 or higher, make sure the tuning system service does not start on when Vertica reboots. Turning off tuning prevents monitoring of your OS and any tuning of your OS based on this monitoring. Tuning also enables THP silently, which can cause issues in other areas such as read ahead.
Run the following command as sudo or root:
$ chkconfig tuned off
6.1 - SUSE control groups configuration
On SuSE 12, the installer checks the control group (cgroup) setting for the cgroups that Vertica may run under:
-
verticad
-
vertica_agent
-
sshd
The installer verifies that the pid.max
resource is large enough for all the threads that Vertica creates. We check the contents of:
-
/sys/fs/cgroup/pids/system.slice/verticad.service/pids.max
-
/sys/fs/cgroup/pids/system.slice/vertica_agent.service/pids.max
-
/sys/fs/cgroup/pids/system.slice/sshd.service/pids.max
If these files exist and they fail to include the value max
, the installation stops and the installer returns a failure message (code S0340).
If these files do not exist, they are created automatically when the systemd
runs the verticad
and vertica_agent
startup scripts. However, the site's cgroup configuration process managed their default values. Vertica does not change the defaults.
Pre-installation configuration
Before installing Vertica, configure your system as follows:
# Create the following directories:
sudo mkdir /sys/fs/cgroup/pids/system.slice/verticad.service/
sudo mkdir /sys/fs/cgroup/pids/system.slice/vertica_agent.service/
# sshd service dir should already exist, so don't need to create it
# Set pids.max values:
sudo sh -c 'echo "max" > /sys/fs/cgroup/pids/system.slice/verticad.service/pids.max'
sudo sh -c 'echo "max" > /sys/fs/cgroup/pids/system.slice/vertica_agent.service/pids.max'
sudo sh -c 'echo "max" > /sys/fs/cgroup/pids/system.slice/sshd.service/pids.max'
Persisting configuration for restart
After installation, you can configure control groups for subsequent reboots of the Vertica database. You do so by editing configuration file /etc/init.d/after.local
and adding the commands shown earlier.
Note
Becauseafter.local
is executed as root, it can omit sudo
commands.
6.2 - Cron required for scheduled jobs
Admintools uses the Linux cron
package to schedule jobs that regularly rotate the database logs. Without this package installed, the database logs will never be rotated. The lack of rotation can lead to a significant consumption of storage for logs. On busy clusters, Vertica can produce hundreds of gigabytes of logs per day.
cron
is installed by default on most Linux distributions, but it may not be present on some SUSE 12 systems.
To install cron
, run this command:
$ sudo zypper install cron
6.3 - Disk readahead
Vertica requires that Disk Readahead be set to at least 2048. The installer reports this issue with the identifier: S0020.
Note
-
These commands must be executed with root privileges and assumes the blockdev program is in
/sbin
. -
The blockdev program operates on whole devices, and not individual partitions. You cannot set the readahead value to different settings on the same device. If you run blockdev against a partition, for example: /dev/sda1, then the setting is still applied to the entire /dev/sda device. For instance, running
/sbin/blockdev --setra 2048 /dev/sda1
also causes /dev/sda2 through /dev/sdaN to use a readahead value of 2048.
RedHat/CentOS and SuSE based systems
For each drive in the Vertica system, Vertica recommends that you set the readahead value to at least 2048 for most deployments. The command immediately changes the readahead value for the specified disk. The second line adds the command to /etc/rc.local
so that the setting is applied each time the system is booted. Note that some deployments may require a higher value and the setting can be set as high as 8192, under guidance of support.
Note
For systems that do not support/etc/rc.local
, use the equivalent startup script that is run after the destination runlevel has been reached. For example SUSE uses /etc/init.d/after.local
.The following example sets the readahead value of the drive sda to 2048:
$ /sbin/blockdev --setra 2048 /dev/sda
$ echo '/sbin/blockdev --setra 2048 /dev/sda' >> /etc/rc.local
If you are using Red Hat 7.0 or CentOS 7.0 or higher, run the following command as root or sudo:
$ chmod +x /etc/rc.d/rc.local
Ubuntu and debian systems
For each drive in the Vertica system, set the readahead value to 2048. Run the command once in your shell, then add the command to /etc/rc.local
so that the setting is applied each time the system is booted. Note that on Ubuntu systems, the last line in rc.local must be "exit 0
". So you must manually add the following line to etc/rc.local
before the last line with exit 0
.
Note
For systems that do not support/etc/rc.local
, use the equivalent startup script that is run after the destination runlevel has been reached. For example SuSE uses /etc/init.d/after.local
./sbin/blockdev --setra 2048 /dev/sda
6.4 - I/O scheduling
Vertica requires that I/O Scheduling be set to
deadline
or
noop
. The installer checks what scheduler the system is using, reporting an unsupported scheduler issue with identifier: S0150. If the installer cannot detect the type of scheduler in use (typically if your system is using a RAID array), it reports that issue with identifier: S0151.
If your system is not using a RAID array, then complete the following steps to change your system to a supported I/O Scheduler. If you are using a RAID array, then consult your RAID vendor documentation for the best performing scheduler for your hardware.
Configure the I/O scheduler
The Linux kernel can use several different I/O schedulers to prioritize disk input and output. Most Linux distributions use the Completely Fair Queuing (CFQ) scheme by default, which gives input and output requests equal priority. This scheduler is efficient on systems running multiple tasks that need equal access to I/O resources. However, it can create a bottleneck when used on Vertica drives containing the catalog and data directories, because it gives write requests equal priority to read requests, and its per-process I/O queues can penalize processes making more requests than other processes.
Instead of the CFQ scheduler, configure your hosts to use either the Deadline or NOOP I/O scheduler for the drives containing the catalog and data directories:
-
The Deadline scheduler gives priority to read requests over write requests. It also imposes a deadline on all requests. After reaching the deadline, such requests gain priority over all other requests. This scheduling method helps prevent processes from becoming starved for I/O access. The Deadline scheduler is best used on physical media drives (disks using spinning platters), since it attempts to group requests for adjacent sectors on a disk, lowering the time the drive spends seeking.
-
The NOOP scheduler uses a simple FIFO approach, placing all input and output requests into a single queue. This scheduler is best used on solid state drives (SSDs). Because SSDs do not have a physical read head, no performance penalty exists when accessing non-adjacent sectors.
Failure to use one of these schedulers for the Vertica drives containing the catalog and data directories can result in slower database performance. Other drives on the system (such as the drive containing swap space, log files, or the Linux system files) can still use the default CFQ scheduler (although you should always use the NOOP scheduler for SSDs).
You can set your disk device scheduler by writing the name of the scheduler to a file in the /sys
directory or using a kernel boot parameter.
Changing the scheduler through the /sys directory
You can view and change the scheduler Linux uses for I/O requests to a single drive using a virtual file under the /sys
directory. The name of the file that controls the scheduler a block device uses is:
/sys/block/deviceName/queue/scheduler
Where deviceName
is the name of the disk device, such as sda
or cciss\!c0d1
(the first disk on an OpenText RAID array). Viewing the contents of this file shows you all of the possible settings for the scheduler. The currently-selected scheduler is surrounded by square brackets:
# cat /sys/block/sda/queue/scheduler
noop deadline [cfq]
To change the scheduler, write the name of the scheduler you want the device to use to its scheduler file. You must have root privileges to write to this file. For example, to set the sda drive to use the deadline scheduler, run the following command as root:
# echo deadline > /sys/block/sda/queue/scheduler
# cat /sys/block/sda/queue/scheduler
noop [deadline] cfq
Changing the scheduler immediately affects the I/O requests for the device. The Linux kernel starts using the new scheduler for all of the drive's input and output requests.
Note
While tests show that changing the scheduler settings while Vertica is running does not cause problems, Vertica recommends shutting down. Before changing the I/O schedule, or making any other changes to the system configuration, consider shutting down any running database.Changes to the I/O scheduler made through the /sys
directory only last until the system is rebooted, so you need to add the commands that change the I/O scheduler to a startup script (such as those stored in /etc/init.d
, or though a command in /etc/rc.local
). You also need to use a separate command for each drive on the system whose scheduler you want to change.
For example, to make the configuration take effect immediately and add it to rc.local so it is used on subsequent reboots.
Note
For systems that do not support/etc/rc.local
, use the equivalent startup script that is run after the destination runlevel has been reached. For example SuSE uses /etc/init.d/after.local
.echo deadline > /sys/block/sda/queue/scheduler
echo 'echo deadline > /sys/block/sda/queue/scheduler' >> /etc/rc.local
Note
On some Ubuntu/Debian systems, the last line in rc.local must be "exit 0
". So you must manually add the following line to etc/rc.local
before the last line with exit 0
.
You may prefer to use this method of setting the I/O scheduler over using a boot parameter if your system has a mix of solid-state and physical media drives, or has many drives that do not store Vertica catalog and data directories.
If you are using Red Hat 7.0 or CentOS 7.0 or higher, run the following command as root or sudo:
$ chmod +x /etc/rc.d/rc.local
Changing the scheduler with a boot parameter
Use the elevator
kernel boot parameter to change the default scheduler used by all disks on your system. This is the best method to use if most or all of the drives on your hosts are of the same type (physical media or SSD) and will contain catalog or data files. You can also use the boot parameter to change the default to the scheduler the majority of the drives on the system need, then use the /sys
files to change individual drives to another I/O scheduler. The format of the elevator boot parameter is:
elevator=schedulerName
Where schedulerName
is deadline
, noop
, or cfq
. You set the boot parameter using your bootloader (grub or grub2 on most recent Linux distributions). See your distribution's documentation for details on how to add a kernel boot parameter.
6.5 - Enabling or disabling transparent hugepages
You can modify transparent hugepages to meet Vertica configuration requirements:
-
For Red Hat 7/CentOS 7 and Amazon Linux 2.0, you must enable transparent hugepages. The installer reports this issue with the identifier: S0312.
-
For Red Hat 8/CentOS 8 and SUSE 15.1, Vertica provides recommended settings to optimize your system performance by workload.
-
For all other systems, you must disable transparent hugepages or set them to
madvise
. The installer reports this issue with the identifier: S0310.
Recommended settings by workload for red hat 8/CentOS 8 and SUSE 15.1
Vertica recommends transparent hugepages settings to optimize performance by workload. The following table contains recommendations for systems that primarily run concurrent queries (such as short-running dashboard queries), or sequential SELECT or load (COPY) queries:
Operating System | Concurrent | Sequential | Important Notes |
---|---|---|---|
Red Hat 8.0/CentOS 8.0 | Disable | Enable | |
SUSE 15.1 | Disable | Enable |
Additionally, Vertica recommends the following Concurrent Workloads:
Sequential Workloads:
|
See Enabling or disabling defrag for additional settings that optimize your system performance by workload.
Enabling transparent hugepages on red hat 7/8, CentOS 7/8, SUSE 15.1, and Amazon Linux 2.0
Determine if transparent hugepages is enabled. To do so, run the following command.
cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
The setting returned in brackets is your current setting.
For systems that do not support /etc/rc.local
, use the equivalent startup script that is run after the destination runlevel has been reached. For example SuSE uses /etc/init.d/after.local
.
You can enable transparent hugepages by editing /etc/rc.local
and adding the following script:
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo always > /sys/kernel/mm/transparent_hugepage/enabled
fi
You must reboot your system for the setting to take effect, or, as root, run the following echo line to proceed with the install without rebooting:
# echo always > /sys/kernel/mm/transparent_hugepage/enabled
If you are using Red Hat 7.0 or CentOS 7.0 or higher, run the following command as root or sudo:
$ chmod +x /etc/rc.d/rc.local
Disabling transparent hugepages on other systems
Note
SUSE did not offer transparent hugepage support in its initial 11.0 release. However, subsequent SUSE service packs do include support for transparent hugepages.To determine if transparent hugepages is enabled, run the following command.
cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
The setting returned in brackets is your current setting. Depending on your platform OS, the madvise
setting may not be displayed.
You can disable transparent hugepages one of two ways:
-
Edit your boot loader (for example
/etc/grub.conf
). Typically, you add the following to the end of the kernel line. However, consult the documentation for your system before editing your bootloader configuration.transparent_hugepage=never
-
Edit
/etc/rc.local
(on systems that support rc.local) and add the following script.if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi
For systems that do not support /etc/rc.local
, use the equivalent startup script that is run after the destination runlevel has been reached. For example SuSE uses /etc/init.d/after.local
.
Regardless of which approach you choose, you must reboot your system for the setting to take effect, or run the following two echo lines to proceed with the install without rebooting:
echo never > /sys/kernel/mm/transparent_hugepage/enabled
6.6 - Check for swappiness
The swappiness kernel parameter defines the amount, and how often, the kernel copies RAM contents to a swap space. Vertica recommends a value of 0. The installer reports any swappiness issues with identifier S0112.
You can check the swappiness value by running the following command:
$ cat /proc/sys/vm/swappiness
To set the swappiness value add or update the following line in /etc/sysctl.conf
:
vm.swappiness = 0
This also ensures that the value persists after a reboot.
If necessary, you change the swappiness value at runtime by logging in as root and running the following:
$ echo 0 > /proc/sys/vm/swappiness
6.7 - Enabling network time protocol (NTP)
Important
Data damage and performance issues might occur if you change host NTP settings while the database is running. Before you change the NPT settings, stop the database. If you cannot stop the database, stop the Vertica process of each host and change the NTP settings one host at a time.
For details, see Stopping Vertica on host.
The network time protocol (NTP) daemon must be running on all of the hosts in the cluster so that their clocks are synchronized. The spread daemon relies on all of the nodes to have their clocks synchronized for timing purposes. If your nodes do not have NTP running, the installation can fail with a spread configuration error or other errors.
Note
Different Linux distributions refer to the NTP daemon in different ways. For example, SUSE and Debian/Ubuntu refer to it asntp
, while CentOS and Red Hat refer to it as ntpd
. If the following commands produce errors, try using the other NTP daemon reference name.
Verify that NTP is running
To verify that your hosts are configured to run the NTP daemon on startup, enter the following command:
$ chkconfig --list ntpd
Debian and Ubuntu do not support chkconfig
, but they do offer an optional package. You can install this package with the command sudo apt-get install sysv-rc-conf
. To verify that your hosts are configured to run the NTP daemon on startup with the sysv-rc-conf
utility, enter the following command:
$ sysv-rc-conf --list ntpd
The chkconfig
command can produce an error similar to ntpd: unknown service
. If you get this error, verify that your Linux distribution refers to the NTP daemon as ntpd
rather than ntp
. If it does not, you need to install the NTP daemon package before you can configure it. Consult your Linux documentation for instructions on how to locate and install packages.
If the NTP daemon is installed, your output should resemble the following:
ntp 0:off 1:off 2:on 3:on 4:off 5:on 6:off
The output indicates the runlevels where the daemon runs. Verify that the current runlevel of the system (usually 3 or 5) has the NTP daemon set to on
. If you do not know the current runlevel, you can find it using the runlevel
command:
$ runlevel
N 3
Configure NTP for red hat 6/CentOS 6 and SLES
If your system is based on Red Hat 6/CentOS 6 or SUSE Linux Enterprise Server, use the service
and chkconfig
utilities to start NTP and have it start at startup.
$ /sbin/service ntpd restart
$ /sbin/chkconfig ntpd on
-
Red Hat 6/CentOS 6—NTP uses the default time servers at ntp.org. You can change the default NTP servers by editing
/etc/ntpd.conf
. -
SLES—By default, no time servers are configured. You must edit
/etc/ntpd.conf
after the install completes and add time servers.
Configure NTP for ubuntu and debian
By default, the NTP daemon is not installed on some Ubuntu and Debian systems. First, install NTP, and then start the NTP process. You can change the default NTP servers by editing /etc/ntpd.conf
as shown:
$ sudo apt-get install ntp
$ sudo /etc/init.d/ntp reload
Verify that NTP is operating correctly
To verify that the Network Time Protocol Daemon (NTPD) is operating correctly, issue the following command on all nodes in the cluster.
For Red Hat 6/CentOS 6 and SLES:
$ /usr/sbin/ntpq -c rv | grep stratum
For Ubuntu and Debian:
$ ntpq -c rv | grep stratum
A stratum level of 16 indicates that NTP is not synchronizing correctly.
If a stratum level of 16 is detected, wait 15 minutes and issue the command again. It may take this long for the NTP server to stabilize.
If NTP continues to detect a stratum level of 16, verify that the NTP port (UDP Port 123) is open on all firewalls between the cluster and the remote machine to which you are attempting to synchronize.
Red hat documentation related to NTP
The preceding links were current as of the last publication of the Vertica documentation and could change between releases.
6.8 - Enabling chrony or ntpd for red hat 7/CentOS 7 systems
Before you can install Vertica, you must enable one of the following on your system for clock synchronization:
-
chrony
-
NTPD
You must enable and activate the Network Time Protocol (NTP) before installation. Otherwise, the installer reports this issue with the identifier S0030.
For information on installing and using chrony, see the information below. For information on NTPD see Enabling network time protocol (NTP). For more information about chrony, see Using chrony in the Red Hat documentation.
Install chrony
The chrony suite consists of:
-
chronyd - the daemon for clock synchronization.
-
chronyc - the command-line utility for configuring chronyd .
chrony is installed by default on some versions of Red Hat/CentOS 7. However, if chrony is not installed on your system, you must download it. To download chrony, run the following command as sudo or root:
# yum install chrony
Verify that chrony is running
To view the status of the chronyd daemon, run the following command:
$ systemctl status chronyd
If chrony is running, an output similar to the following appears:
chronyd.service - NTP client/server
Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled)
Active: active (running) since Mon 2015-07-06 16:29:54 EDT; 15s ago
Main PID: 2530 (chronyd)
CGroup: /system.slice/chronyd.service
ââ2530 /usr/sbin/chronyd -u chrony
If chrony is not running, execute the following command as sudo or root. This command also causes chrony to run at boot time:
# systemctl enable chronyd
Verify that chrony is operating correctly
To verify that the chrony daemon is operating correctly, issue the following command on all nodes in the cluster:
$ chronyc tracking
An output similar to the following appears:
Reference ID : 198.247.63.98 (time01.website.org)
Stratum : 3
Ref time (UTC) : Thu Jul 9 14:58:01 2015
System time : 0.000035685 seconds slow of NTP time
Last offset : -0.000151098 seconds
RMS offset : 0.000279871 seconds
Frequency : 2.085 ppm slow
Residual freq : -0.013 ppm
Skew : 0.185 ppm
Root delay : 0.042370 seconds
Root dispersion : 0.022658 seconds
Update interval : 1031.0 seconds
Leap status : Normal
A stratum level of 16 indicates that chrony is not synchronizing correctly. If chrony continues to detect a stratum level of 16, verify that the UDP port 323 is open. This port must be open on all firewalls between the cluster and the remote machine to which you are attempting to synchronize.
6.9 - SELinux configuration
Vertica does not support SELinux except when SELinux is running in permissive mode. If it detects that SELinux is installed and the mode cannot be determined the installer reports this issue with the identifier: S0080. If the mode can be determined, and the mode is not permissive, then the issue is reported with the identifier: S0081.
Red hat and SUSE systems
You can either disable SELinux or change it to use permissive mode.
To disable SELinux:
-
Edit
/etc/selinux/config
and change setting for SELinux to disabled (SELINUX=disabled
). This disables SELinux at boot time. -
As root/sudo, type
setenforce 0
to disable SELinux immediately.
To change SELinux to use permissive mode:
-
Edit
/etc/selinux/config
and change setting for SELINUX to permissive (SELINUX=Permissive
). -
As root/sudo, type
setenforce Permissive
to switch to permissive mode immediately.
Ubuntu and debian systems
You can either disable SELinux or change it to use permissive mode.
To disable SELinux:
-
Edit
/selinux/config
and change setting for SELinux to disabled (SELINUX=disabled
). This disables SELinux at boot time. -
As root/sudo, type
setenforce 0
to disable SELinux immediately.
To change SELinux to use permissive mode:
-
Edit
/selinux/config
and change setting for SELinux to permissive (SELINUX=Permissive
). -
As root/sudo, type
setenforce Permissive
to switch to permissive mode immediately.
6.10 - CPU frequency scaling
This topic details the various CPU frequency scaling methods supported by Vertica. In general, if you do not require CPU frequency scaling, then disable it so as not to impact system performance.
Important
Your systems may use significantly more energy when frequency scaling is disabled.The installer allows CPU frequency scaling to be enabled when the cpufreq scaling governor is set to performance
. If the cpu scaling governor is set to ondemand, and ignore_nice_load
is 1 (true), then the installer fails with the error S0140. If the cpu scaling governor is set to ondemand and ignore_nice_load
is 0 (false), then the installer warns with the identifier S0141.
CPU frequency scaling is a hardware and software feature that helps computers conserve energy by slowing the processor when the system load is low, and speeding it up again when the system load increases. This feature can impact system performance, since raising the CPU frequency in response to higher system load does not occur instantly. Always disable this feature on the Vertica database hosts to prevent it from interfering with performance.
You disable CPU scaling in your host's system BIOS. There may be multiple settings in your host's BIOS that you need to adjust in order to completely disable CPU frequency scaling. Consult your host hardware's documentation for details on entering the system BIOS and disabling CPU frequency scaling.
If you cannot disable CPU scaling through the system BIOS, you can limit the impact of CPU scaling by disabling the scaling through the Linux kernel or setting the CPU frequency governor to always run the CPU at full speed.
Caution
This method is not reliable, as some hardware platforms may ignore the kernel settings. For more information, see Vertica Hardware Guide.The method you use to disable frequency depends on the CPU scaling method being used in the Linux kernel. See your Linux distribution's documentation for instructions on disabling scaling in the kernel or changing the CPU governor.
6.11 - Enabling or disabling defrag
You can modify the defrag utility to meet Vertica configuration requirements, or to optimize your system performance by workload.
On all Red Hat/CentOS systems, you must disable the defrag utility to meet Vertica configuration requirements.
Note
The steps to disable defrag on Red Hat 6/CentOS 6 systems differ from those used to disable defrag on Red Hat 7/CentOS 7 and Red Hat 8/CentOS 8.For SUSE 15.1, Vertica recommends that you enable defrag for optimized performance.
Recommended settings by workload for red hat 8/CentOS 8 and SUSE 15.1
Vertica recommends defrag settings to optimize performance by workload. The following table contains recommendations for systems that primarily run concurrent queries (such as short-running dashboard queries), or sequential SELECT or load (COPY) queries:
Operating System | Concurrent | Sequential |
---|---|---|
Red Hat 8.0/CentOS 8.0 | Disable | Disable |
SUSE 15.1 | Enable | Enable |
See Enabling or disabling transparent hugepages for additional settings that optimize your system performance by workload.
Disabling defrag on red hat 6/CentOS 6 systems
-
Determine if defrag is enabled by running the following command:
cat /sys/kernel/mm/redhat_transparent_hugepage/defrag [always] madvise never
The setting returned in brackets is your current setting. If you are not using
madvise
ornever
as your defrag setting, then you must disable defrag. -
Edit
/etc/rc.local,
and add the following script:if test -f /sys/kernel/mm/redhat_transparent_hugepage/enabled; then echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag fi
You must reboot your system for the setting to take effect, or run the following echo line to proceed with the install without rebooting:
# echo never > /sys/kernel/mm/redhat_transparent_hugepage/defrag
Disabling defrag on red hat 7/CentOS 7, red hat 8/CentOS 8, and SUSE 15.1
-
Determine if defrag is enabled by running the following command:
cat /sys/kernel/mm/transparent_hugepage/defrag [always] madvise never
The setting returned in brackets is your current setting. If you are not using
madvise
ornever
as your defrag setting, then you must disable defrag. -
Edit
/etc/rc.local,
and add the following script:if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/defrag fi
You must reboot your system for the setting to take effect, or run the following echo line to proceed with the install without rebooting:
# echo never > /sys/kernel/mm/transparent_hugepage/defrag
-
If you are using Red Hat 7.0/CentOS 7.0 or Red Hat 8.0/CentOS 8.0, run the following command as root or sudo:
$ chmod +x /etc/rc.d/rc.local
Enabling defrag on red hat 7/8, CentOS 7/8, and SUSE 15.1
-
Determine if defrag is enabled by running the following command:
cat /sys/kernel/mm/transparent_hugepage/defrag [never] madvise never
The setting returned in brackets is your current setting. If you are not using
madvise
oralways
as your defrag setting, then you must enable defrag. -
Edit
/etc/rc.local,
and add the following script:if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo always > /sys/kernel/mm/transparent_hugepage/defrag fi
You must reboot your system for the setting to take effect, or run the following echo line to proceed with the install without rebooting:
# echo always > /sys/kernel/mm/transparent_hugepage/defrag
-
If you are using Red Hat 7.0/CentOS 7.0 or Red Hat 8.0/CentOS 8.0, run the following command as root or sudo:
$ chmod +x /etc/rc.d/rc.local
6.12 - Support tools
Vertica suggests that the following tools are installed so support can assist in troubleshooting your system if any issues arise:
-
pstack (or gstack) package. Identified by issue S0040 when not installed.
- On Red Hat 7 and CentOS 7 systems, the pstack package is installed as part of the gdb package.
-
mcelog package. Identified by issue S0041 when not installed.
-
sysstat package. Identified by issue S0045 when not installed.
Red hat 6 and CentOS 6 systems
To install the required tools on Red Hat 6 and CentOS 6 systems, run the following commands as sudo or root:
# yum install pstack
# yum install mcelog
# yum install sysstat
Red hat 7 and CentOS 7 systems
To install the required tools on Red Hat 7/CentOS 7 systems, run the following commands as sudo or root:
# yum install gdb
# yum install mcelog
# yum install sysstat
Ubuntu and debian systems
To install the required tools on Ubuntu and Debian systems, run the following commands as sudo or root:
$ apt-get install pstack
$ apt-get install mcelog
$ apt-get install sysstat
Important
For Ubuntu versions 18.04 and higher, runapt-get install rasdaemon
instead of apt-get install mcelog
.
SuSE systems
To install the required tools on SuSE systems, run the following commands as sudo or root.
# zypper install sysstat
# zypper install mcelog
There is no individual SuSE package for pstack/gstack. However, the gdb package contains gstack, so you could optionally install gdb instead, or build pstack/gstack from source. To install the gdb package:
# zypper install gdb
7 - System user configuration
The following tasks pertain to the configuration of the system user required by Vertica.
7.1 - System user requirements
Vertica has specific requirements for the system user that runs and manages Vertica. If you specify a user during install, but the user does not exist, then the installer reports this issue with the identifier: S0200.
System user requirement details
Vertica requires a system user to own database files and run database processes and administration scripts. By default, the install script automatically configures and creates this user for you with the username dbadmin. See Linux users created by Vertica for details on the default user created by the install script. If you decide to manually create your own system user, then you must create the user before you run the install script. If you manually create the user:
Note
Instances ofdbadmin
and verticadba
are placeholders for the names you choose if you do not use the default values.
-
the user must have the same username and password on all nodes
-
the user must use the BASH shell as the user's default shell. If not, then the installer reports this issue with identifier [S0240].
-
the user must be in the verticadba group (for example:
usermod -a -G verticadba
userNameHere
). If not, the installer reports this issue with identifier [S0220].Note
You must create a verticadba group on all nodes. If you do not, then the installer reports the issue with identifier [S0210]. -
the user's login group must be either verticadba or a group with the same name as the user (for example, the home group for dbadmin is dbadmin). You can check the groups for a user with the id command. For example:
id dbadmin
. The "gid" group is the user's primary group. If this is not configured correctly then the installer reports this issue with the identifier [S0230]. Vertica recommends that you use verticadba as the user's primary login group. For example:usermod -g verticadba
userNameHere
. If the user's primary group is not verticadba as suggested, then the installer reports this with HINT [S0231]. -
the user must have a home directory. If not, then the installer reports this issue with identifier [S0260].
-
the user's home directory must be owned by the user. If not, then the installer reports the issue with identifier [S0270].
-
the system must be aware of the user's home directory (you can set it with the usermod command:
usermod -m -d /path/to/new/home/dir
userNameHere
). If this is not configured correctly then the installer reports the issue with [S0250]. -
the user's home directory must be owned by the dbadmin's primary group (use the
chown
andchgrp
commands if necessary). If this is not configured correctly, then the installer reports the issue with identifier [S0280]. -
the user's home directory should have secure permissions. Specifically, it should not be writable by anyone or by the group. Ideally the permissions should be, when viewing with ls, "
---
" (nothing), or "r-x
" (read and execute). If this is not configured as suggested then the installer reports this with HINT [S0290].
7.2 - TZ environment variable
This topic details how to set or change the TZ environment variable and update your tzdata package. If this variable is not set, then the installer reports this issue with the identifier: S0305.
Before installing Vertica, update the tzdata package for your system and set the default time zone for your database administrator account by specifying the TZ
environmental variable. If your database administrator is being created by the install_vertica
script, then set the TZ
variable after you have installed Vertica.
Update tzdata package
The tzdata package is a public-domain time zone database that is pre-installed on most Linux systems. The tzdata package is updated periodically for time-zone changes across the world. You should update to the latest tzdata package before installing or updating Vertica.
Update your tzdata package with the following command:
-
RedHat based systems:
yum update tzdata
-
Debian and Ubuntu systems:
apt-get install tzdata
Setting the default time zone
When a client receives the result set of a SQL query, all rows contain data adjusted, if necessary, to the same time zone. That time zone is the default time zone of the initiator node unless the client explicitly overrides it using the SQL SET TIME ZONE command described in the SQL Reference Manual. The default time zone of any node is controlled by the TZ
environment variable. If TZ
is undefined, the operating system time zone.
Important
TheTZ
variable must be set to the same value on all nodes in the cluster.If your operating system timezone is not set to the desired timezone of the database then make sure that the Linux environment variable TZ
is set to the desired value on all cluster hosts.
The installer returns a warning if the TZ variable is not set. If your operating system timezone is appropriate for your database, then the operating system timezone is used and the warning can be safely ignored.
Setting the time zone on a host
Important
If you explicitly set theTZ
environment variable at a command line before you start the Administration tools, the current setting will not take effect. The Administration Tools uses SSH to start copies on the other nodes, so each time SSH is used, the TZ
variable for the startup command is reset. TZ
must be set in the .profile
or .bashrc
files on all nodes in the cluster to take affect properly.
You can set the time zone several different ways, depending on the Linux distribution or the system administrator’s preferences.
-
To set the system time zone on Red Hat and SUSE Linux systems, edit:
/etc/sysconfig/clock
-
To set the
TZ
variable, edit,/etc/profile
, or/home/dbadmin/.bashrc
or/home/dbadmin/.bash_profile
and add the following line (for example, for the US Eastern Time Zone):export TZ="America/New_York"
For details on which timezone names are recognized by Vertica, see the appendix: Using time zones with Vertica.
7.3 - LANG environment variable settings
This topic details how to set or change the LANG environment variable. The LANG environment variable controls the locale of the host. If this variable is not set, then the installer reports this issue with the identifier: S0300. If this variable is not set to a valid value, then the installer reports this issue with the identifier: S0301.
Set the host locale
Each host has a system setting for the Linux environment variable LANG
. LANG
determines the locale category for native language, local customs, and coded character set in the absence of the LC_ALL
and other LC_ environment variables. LANG
can be used by applications to determine which language to use for error messages and instructions, collating sequences, date formats, and so forth.
To change the LANG
setting for the database administrator, edit, /etc/profile
, or /dbadmin/.bashrc
or /home/dbadmin/.bash_profile
on all cluster hosts and set the environment variable; for example:
export LANG=en_US.UTF-8
The LANG
setting controls the following in Vertica:
-
OS-level errors and warnings, for example, "file not found" during COPY operations.
-
Some formatting functions, such as TO_CHAR and TO_NUMBER. See also Template patterns for numeric formatting.
The LANG
setting does not control the following:
-
Vertica specific error and warning messages. These are always in English at this time.
-
Collation of results returned by SQL issued to Vertica. This must be done using a database parameter instead. See Implement locales for international data sets section for details.
Note
If theLC_ALL
environment variable is set, it supersedes the setting of LANG
.
7.4 - Package dependencies
For successful Vertica installation, you must first install three packages on all nodes in your cluster before installing the database platform.
The required packages are:
-
openssh—Required for Administration tools connectivity between nodes.
-
which—Required for Vertica operating system integration and for validating installations.
-
dialog—Required for interactivity with Administration Tools.
Installing the required packages
The procedure you follow to install the required packages depends on the operating system on which your node or cluster is running. See your operating system's documentation for detailed information on installing packages.
-
For CentOS/Red Hat Systems—Typically, you manage packages on Red Hat and CentOS systems using the yum utility.
Run the following yum commands to install each of the package dependencies. The yum utility guides you through the installation:
# yum install openssh # yum install which # yum install dialog
-
For Debian/Ubuntu Systems—Typically, you use the apt-get utility to manage packages on Debian and Ubuntu systems.
Run the following apt-get commands to install each of the package dependencies. The apt-get utility guides you through the installation:
# apt-get install openssh # apt-get install which # apt-get install dialog