This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Managing backups

vbr provides several tasks related to managing backups: listing them, checking their integrity, selectively deleting them, and more.

vbr provides several tasks related to managing backups: listing them, checking their integrity, selectively deleting them, and more. In addition, vbr has parameters to allow you to restrict its use of system resources.

1 - Viewing backups

You can view backups in three ways:.

You can view backups in three ways:

  • vbr listbackup task: List backups on the local or remote backup host.
  • DATABASE_BACKUPS system table: Query for historical information about backups.
  • vbr log file: Check the status of a backup. The log file resides on the node where you ran vbr, in the directory specified by the vbr configuration parameter tempDir, by default set to /tmp/vbr.

vbr listbackup

The vbr task listbackup returns a list of all backups on backup hosts, whether local or remote. If unqualified by task options, listbackup returns the list to standard output in columnar format.

The following example lists two full backups of a three-node cluster, where each node is mapped to the same backup host, bkhost. Backups are listed in reverse chronological order:

$ vbr -t listbackup -c fullbackup.ini
backup                            backup_type   epoch   objects   include_patterns   exclude_patterns   nodes(hosts)                                                                                        version            file_system_type
backup_snapshot_20220912_131918   full          3915                                                    v_vmart_node0001(10.20.100.247), v_vmart_node0002(10.20.100.248), v_vmart_node0003(10.20.100.249)   v12.0.2-20220911   [Linux]
backup_snapshot_20220909_122300   full          3910                                                    v_vmart_node0001(10.20.100.247), v_vmart_node0002(10.20.100.248), v_vmart_node0003(10.20.100.249)   v12.0.2-20220911   [Linux]

The following table contains information about output columns that are returned from a vbr listbackup task:

Column Description
backup

Identifies a backup by concatenating the configured snapshot name with the backup timestamp:

snapshot-name_YYYYMMDD_HHMMSS

For example, the following identifier identifies a backup generated by the configuration file that sets snapshotName to monthlyBackup on April 14 2022, at 13:44:52.

monthlyBackup_20220414_134452

Use the timestamp portion of this identifier—20220414_134452—to specify the archived backup you wish to restore.

backup_type Type of backup, full or object.
epoch Epoch when the backup was created.
objects Objects that were backed up, blank if a full backup.
include_patterns Wildcard patterns included in object backup tasks using the includeObjects parameter in your configuration file, blank for full backups.
exclude_patterns Wildcard patterns included in your object backup tasks using the excludeObjects parameter in your configuration file, blank for full backups.
nodes (hosts) (Enterprise Mode only) Names of database nodes and hosts that received the backup.
version Version of Vertica used to create the backup.
file_system_type Storage location file system of the Vertica hosts that comprise this backup—for example, Linux or GCS.
communal_storage (Eon Mode only) Communal storage location for the backup.

Listbackup options

You can qualify the listbackup task with one or more options:

vbr --task listbackup [--list-all] [--json] [--list-output-file filepath] --config-file filepath
Option Description
--list-all Generate a list of all snapshots stored on the hosts and paths listed in the specified configuration file.
--json Use JSON delimited format.
--list-output-file Redirect output to the specified file.

The following example qualifies the listbackup task with the --list-all option. The output shows three nightly backups from nodes vmart_1, vmart_2, and v_mart3, which the configuration file nightly.ini maps to their respective hosts doca01, doca02, and doca03. The listbackup output shows that these locations contain not only object backups that were generated with nightly.ini, but also full backups created with a second configuration file, weekly.ini, which maps to the same nodes and host:

$ vbr --task listbackup --list-all --config-file /home/dbadmin/nightly.ini

backup                   backup_type  epoch  objects      include_patterns  exclude_patterns  nodes(hosts)                                      version  file_system_type
weekly_20220508_183249   full         1720                                       vmart_1(doca01), vmart_2(doca02), vmart_3(doca03) v11.0.1  [Linux]
weekly_20220501_182816   full         1403                                       vmart_1(doca01), vmart_2(doca02), vmart_3(doca03) v11.0.1  [Linux]
weekly_20220424_192754   full         1109                                       vmart_1(doca01), vmart_2(doca02), vmart_3(doca03) v11.0.1  [Linux]
nightly_20220507_183034  object       1705   sales_schema                        vmart_1(doca01), vmart_2(doca02), vmart_3(doca03) v11.0.1  [Linux]
nightly_20220506_181808  object       1692   sales_schema                        vmart_1(doca01), vmart_2(doca02), vmart_3(doca03) v11.0.1  [Linux]
nightly_20220505_193906  object       1632   sales_schema                        vmart_1(doca01), vmart_2(doca02), vmart_3(doca03) v11.0.1  [Linux]

Query backup history

You can query the system table DATABASE_BACKUPS to get historical information about backups. The objects column lists which objects were included in object-level backups.

=> SELECT * FROM v_monitor.database_backups;
-[ RECORD 1 ]----+------------------------------
backup_timestamp | 2013-05-10 14:41:12.673381-04
node_name        | v_vmart_node0003
snapshot_name    | schemabak
backup_epoch     | 174
node_count       | 3
file_system_type | [Linux]
objects          | public, store, online_sales
-[ RECORD 2 ]----+------------------------------
backup_timestamp | 2013-05-13 11:17:30.913176-04
node_name        | v_vmart_node0003
snapshot_name    | kantibak
backup_epoch     | 175
node_count       | 3
file_system_type | [Linux]
objects          |
-[ RECORD 13 ]---+------------------------------
backup_timestamp | 2013-05-16 07:02:23.721657-04
node_name        | v_vmart_node0003
snapshot_name    | objectbak
backup_epoch     | 180
node_count       | 3
file_system_type | [Linux]
objects          | test, test2
-[ RECORD 14 ]---+------------------------------
backup_timestamp | 2013-05-16 07:19:44.952884-04
node_name        | v_vmart_node0003
snapshot_name    | table1bak
backup_epoch     | 180
node_count       | 3
file_system_type | [Linux]
objects          | test
-[ RECORD 15 ]---+------------------------------
backup_timestamp | 2013-05-16 07:20:18.585076-04
node_name        | v_vmart_node0003
snapshot_name    | table2bak
backup_epoch     | 180
node_count       | 3
file_system_type | [Linux]
objects          | test2

2 - Checking backup integrity

Vertica can confirm the integrity of your backup files and the manifest that identifies them.

Vertica can confirm the integrity of your backup files and the manifest that identifies them. By default, backup integrity checks output their results to the command line.

Quick check

The quick-check task gathers all backup metadata from the backup location specified in the configuration file and compares that metadata to the backup manifest. A quick check does not verify the objects themselves. Instead, this task outputs an exceptions list of any discrepancies between objects in the backup location and objects listed in the backup manifest.

Use the following format to perform quick check task:

$ vbr -t quick-check -c configfile.ini

For example:

$ vbr -t quick-check -c backupconfig.ini

Full check

The full-check task verifies all objects listed in the backup manifest against filesystem metadata. A full check includes the same steps as a quick check. You can include the optional --report-file parameter to output results to a delimited JSON file. This task outputs an exceptions list that identifies the following inconsistencies:

  • Incomplete restore points

  • Damaged restore points

  • Missing backup files

  • Unreferenced files

Use the following template to perform a full check task:

$ vbr -t full-check -c configfile.ini --report-file=path/filename

For example:

$ vbr -t full-check -c backupconfig.ini --report-file=logging/fullintegritycheck.json

3 - Repairing backups

Vertica can reconstruct backup manifests and remove unneeded backup objects.

Vertica can reconstruct backup manifests and remove unneeded backup objects.

Quick repair

The quick-repair task rebuilds the backup manifest, based on the manifests contained in the backup location.

Use the following template to perform a quick repair task:

$ vbr -t quick-repair -c configfile.ini

Garbage collection

The collect-garbage task rebuilds your backup manifest and deletes any backup objects that do not appear in the manifest. You can include the optional --report-file parameter to output results to a delimited JSON file.

Use the following template to perform a garbage collection task:

$ vbr -t collect-garbage -c configfile.ini --report-file=path/filename

4 - Removing backups

You can remove existing backups and restore points using vbr.

You can remove existing backups and restore points using vbr. When you use the remove task, vbr updates the manifests affected by the removal and maintains their integrity. If the backup archive contains multiple restore points, removing one does not affect the others. When you remove the last restore point, vbr removes the backup entirely.

Use the following template to perform a remove task:

$ vbr -t remove -c configfile.ini --archive timestamp

You can remove multiple restore points using the archive parameter. To obtain the timestamp for a particular restore point, use the listbackup task.

  • To remove multiple restore points, use a comma separator:

    --archive="`*`restore-point1`*`,`*`restore-point2`*`"
    
  • To remove an inclusive range of restore points, use a colon:

    --archive="`*`restore-point1`*`:`*`restore-point2`*`"
    
  • To remove all restore points, specify an archive value of all:

    --archive all
    

The following example shows how you can remove a restore point from an existing backup:

$ vbr -t remove -c backup.ini --archive  20160414_134452
Removing restore points: 20160414_134452
Remove complete!

5 - Estimating log file disk requirements

One of the vbr configuration parameters is tempDir.

One of the vbr configuration parameters is tempDir . This parameter specifies the database host location where vbr writes its log files and some other temp files (of negligible size). The default location is the /tmp/vbr directory on each database host. You can change the default location by specifying a different path in the configuration file.

The temporary storage directory also contains local log files describing the progress, throughput, and any errors encountered for each node. Each time you run vbr, the script creates a separate log file, each named with a timestamp. When using default settings, the log file typically uses about 4KB of space per node per backup.

The vbr log files are not removed automatically, so you must delete older log files manually, as necessary.

6 - Allocating resources

By default, vbr allows a single rsync connection (for Linux file systems), 10 concurrent threads (for cloud storage connections), and unlimited bandwidth for any backup or restore operation.

By default, vbr allows a single rsync connection (for Linux file systems), 10 concurrent threads (for cloud storage connections), and unlimited bandwidth for any backup or restore operation. You can change these values in your configuration file. See vbr configuration file reference for details about these parameters.

Connections

You might want to increase the number of concurrent connections. If you have many Vertica files, more connections can provide a significant performance boost as each connection increases the number of concurrent file transfers.

For more information, refer to the following parameters in [transmission]:

  • total_bwlimit_backup

  • total_bwlimit_restore

  • concurrency_backup

  • concurrency_restore

and the following parameters in [CloudStorage]:

  • cloud_storage_concurrency_backup

  • cloud_storage_concurrency_restore

Bandwidth limits

You can limit network bandwidth use through the total_bwlimit_backup and total_bwlimit_restore data transmission parameters. For more information, refer to [transmission].