OpenText Analytics Database 26.2.x – Before you upgrade

Setup: Verifying platform requirements

Mon, 01 Jan 0001 00:00:00 +0000

The database installer checks the target platform as it runs, and stops whenever it determines the platform fails to meet an installation requirement. Before you update the server package on your systems, manually verify that your platform meets all hardware and software requirements (see Platform and hardware requirements and recommendations).

By default, the installer stops on all warnings. You can configure the level where the installer stops installation, through the installation parameter --failure-threshold. If you set the failure threshold to FAIL, the installer ignores warnings and stops only on failures.

Caution

Changing the failure threshold lets you immediately upgrade and bring up the database. However, the database cannot fully optimize performance until you correct all warnings.

Setup: Checking catalog storage space

Mon, 01 Jan 0001 00:00:00 +0000

Use the commands documented here to determine how much catalog space is available before upgrading. This helps you determine how much space the updated catalog may take up.

Compare how much space the catalog currently uses against space that is available in the same directory:

Use the du command to determine how much space the catalog directory currently uses:
```
$ du -s -BG v_vmart_node0001_catalog
2G      v_vmart_node0001_catalog
```

Determine how much space is available in the same directory:

$ df -BG v_vmart_node0001_catalog
Filesystem     1G-blocks  Used Available Use% Mounted on
/dev/sda2            48G   19G       26G  43% /

Setup: Verify license compliance for ORC and Parquet data

Mon, 01 Jan 0001 00:00:00 +0000

If you are upgrading from a version before 9.1.0 and:

Your database has external tables based on ORC or Parquet files (whether stored locally on the database cluster or on a Hadoop cluster)
Your OpenText™ Analytics Database license has a raw data allowance

follow the steps in this topic before upgrading.

Background

Vertica licenses can include a raw data allowance. Since 2016, Vertica licenses have allowed you to use ORC and Parquet data in external tables. This data has always counted against any raw data allowance in your license. Previously, the audit of data in ORC and Parquet format was handled manually. Because this audit was not automated, the total amount of data in your native tables and external tables could exceed your licensed allowance for some time before being spotted.

Starting in version 9.1.0, Vertica automatically audits ORC and Parquet data in external tables. This auditing begins soon after you install or upgrade to version 9.1.0. If your Vertica license includes a raw data allowance and you have data in external tables based on Parquet or ORC files, review your license compliance before upgrading to Vertica 9.1.x. Verifying your database is compliant with your license terms avoids having your database become non-compliant soon after you upgrade.

Verifying your ORC and Parquet usage complies with your license terms

To verify your data usage is compliant with your license, run the following query as the database administrator:

SELECT (database_size_bytes + file_size_bytes) <= license_size_bytes
       "license_compliant?"
       FROM (SELECT database_size_bytes,
                    license_size_bytes FROM license_audits
                    WHERE audited_data='Total'
                    ORDER BY audit_end_timestamp DESC LIMIT 1) dbs,
            (SELECT sum(total_file_size_bytes) file_size_bytes
                    FROM external_table_details
                    WHERE source_format IN ('ORC', 'PARQUET')) ets;

This query returns one of three values:

If you do not have any external data in ORC or Parquet format, the query returns 0 rows:
```
 license_compliant?
--------------------
(0 rows)
```
In this case, you can proceed with your upgrade.
If you have data in external tables based on ORC or Parquet format, and that data does not cause your database to exceed your raw data allowance, the query returns t:
```
 license_compliant?
--------------------
 t
(1 row)
```
In this case, you can proceed with your upgrade.
If the data in your external tables based on ORC and Parquet causes your database to exceed your raw data allowance, the query returns f:
```
 license_compliant?
--------------------
 f
(1 row)
```
In this case, resolve the compliance issue before you upgrade. See below for more information.

Resolving non-compliance

If query in the previous section indicates that your database is not in compliance with your license, you should resolve this issue before upgrading. There are two ways you can bring your database into compliance:

Contact OpenText™ to upgrade your license to a larger data size allowance. See Obtaining a license key file.
Delete data (either from ORC and Parquet-based external tables or OpenText™ Analytics Database native tables) to bring your data size into compliance with your license. You should always backup any data you are about to delete from the database. Dropping external tables is a less disruptive way to reduce the size of your database, as the data is not lost—it is still in the files that your external table is based on.

Note

You can still choose to upgrade your database if it is not compliant. However, soon after you upgrade, you will begin getting warnings that your database is out of compliance. See Managing license warnings and limits for more information.

Setup: Backing up and restoring grants

Mon, 01 Jan 0001 00:00:00 +0000

After an upgrade, if the prototypes of UDx libraries change, the database will drop the grants on those libraries since they aren't technically the same function anymore. To resolve these types of issues, it's best practice to back up the grants on these libraries so you can restore them after the upgrade.

Save the following SQL to a file named user_ddl.sql. It creates a view named user_ddl which contains the grants on all objects in the database.

CREATE OR REPLACE VIEW user_ddl AS
(
SELECT 0 as grant_order,
       name principal_name,
       'CREATE ROLE "' || name || '"' ||    ';' AS sql,
       'NONE' AS object_type,
       'NONE' AS object_name
  FROM v_internal.vs_roles vr
 WHERE NOT vr.predefined_role -- Exclude system roles
   AND ldapdn = ''            -- Limit to NON-LDAP created roles
)
UNION ALL
(
  SELECT 1, -- CREATE USERs
         user_name,
         'CREATE USER "' || user_name || '"' ||
         DECODE(is_locked, TRUE, ' ACCOUNT LOCK', '') ||
         DECODE(grace_period, 'undefined', '', ' GRACEPERIOD  ''' || grace_period || '''') ||
         DECODE(idle_session_timeout, 'unlimited', '', ' IDLESESSIONTIMEOUT ''' || idle_session_timeout || '''') ||
         DECODE(max_connections, 'unlimited', '', ' MAXCONNECTIONS ' || max_connections || ' ON ' || connection_limit_mode) ||
         DECODE(memory_cap_kb, 'unlimited', '', ' MEMORYCAP ''' || memory_cap_kb || 'K''') ||
         DECODE(profile_name, 'default', '', ' PROFILE ' || profile_name) ||
         DECODE(resource_pool, 'general', '', ' RESOURCE POOL ' || resource_pool) ||
         DECODE(run_time_cap, 'unlimited', '', ' RUNTIMECAP ''' || run_time_cap || '''') ||
         DECODE(search_path, '', '', ' SEARCH_PATH ' || search_path) ||
         DECODE(temp_space_cap_kb, 'unlimited', '', ' TEMPSPACECAP ''' || temp_space_cap_kb || 'K''') || ';' AS sql,
         'NONE' AS object_type,
         'NONE' AS object_name
    FROM v_catalog.users
   WHERE NOT is_super_user -- Exclude database superuser
     AND ldap_dn = ''      -- Limit to NON-LDAP created users
)
UNION ALL
(
  SELECT 2, -- GRANTs
         grantee,
         'GRANT ' || REPLACE(TRIM(BOTH ' ' FROM words), '*', '') ||
         CASE
           WHEN object_type = 'RESOURCEPOOL' THEN ' ON RESOURCE POOL '
           WHEN object_type = 'STORAGELOCATION' THEN ' ON LOCATION '
           WHEN object_type = 'CLIENTAUTHENTICATION' THEN 'AUTHENTICATION '
           WHEN object_type IN ('DATABASE', 'LIBRARY', 'MODEL', 'SEQUENCE', 'SCHEMA') THEN ' ON ' || object_type || ' '
           WHEN object_type = 'PROCEDURE' THEN (SELECT ' ON ' || CASE REPLACE(procedure_type, 'User Defined ', '')
                                                                   WHEN 'Transform' THEN 'TRANSFORM FUNCTION '
                                                                   WHEN 'Aggregate' THEN 'AGGREGATE FUNCTION '
                                                                   WHEN 'Analytic' THEN 'ANALYTIC FUNCTION '
                                                                   ELSE UPPER(REPLACE(procedure_type, 'User Defined ', '')) || ' '
                                                                 END
                                                  FROM vs_procedures
                                                 WHERE proc_oid = object_id)
           WHEN object_type = 'ROLE' THEN ''
           ELSE ' ON '
         END ||
         NVL2(object_schema, object_schema || '.', '') || CASE WHEN object_type = 'STORAGELOCATION' THEN (SELECT '''' || location_path || ''' ON ' || node_name FROM storage_locations WHERE location_id = object_id) ELSE object_name END ||
         CASE
           WHEN object_type = 'PROCEDURE' THEN (SELECT CASE WHEN procedure_argument_types = '' OR procedure_argument_types = 'Any' THEN '()' ELSE '(' || procedure_argument_types || ')' END
                                                  FROM vs_procedures
                                                 WHERE proc_oid = object_id)
           ELSE ''
         END ||
         ' TO ' || grantee ||
         CASE WHEN INSTR(words, '*') > 0 THEN ' WITH GRANT OPTION' ELSE '' END
         || ';',
         object_type,
         object_name
  FROM (SELECT grantee, object_type, object_schema, object_name, object_id,
                 v_txtindex.StringTokenizerDelim(DECODE(privileges_description, '', ',' , privileges_description), ',')
                   OVER (PARTITION BY grantee, object_type, object_schema, object_name, object_id)
          FROM v_catalog.grants) foo
 ORDER BY CASE REPLACE(TRIM(BOTH ' ' FROM words), '*', '') WHEN 'USAGE' THEN 1 ELSE 2 END
)
UNION ALL
(
  SELECT 3, -- Default ROLEs
         user_name,
         'ALTER USER "' || user_name || '"' ||
           DECODE(default_roles, '', '', ' DEFAULT ROLE ' || REPLACE(default_roles, '*', ''))  || ';' ,
         'NONE' AS object_type,
         'NONE' AS object_name
    FROM v_catalog.users
   WHERE default_roles <> ''
)
UNION ALL -- GRANTs WITH ADMIN OPTION
(
  SELECT 4, user_name, 'GRANT ' || REPLACE(TRIM(BOTH ' ' FROM words), '*', '') || ' TO ' || user_name || ' WITH ADMIN OPTION;',
         'NONE' AS object_type ,
         'NONE' AS object_name
    FROM (SELECT user_name, v_txtindex.StringTokenizerDelim(DECODE(all_roles, '', ',', all_roles), ',') OVER (PARTITION BY user_name)
            FROM v_catalog.users
           WHERE all_roles <> '') foo
   WHERE INSTR(words, '*') > 0
)
UNION ALL
(
  SELECT 5, 'public', 'ALTER SCHEMA ' || name || ' DEFAULT ' || CASE WHEN defaultinheritprivileges THEN 'INCLUDE PRIVILEGES;' ELSE 'EXCLUDE PRIVILEGES;' END, 'SCHEMA', name
    FROM v_internal.vs_schemata
   WHERE NOT issys -- Exclude system schemas
)
UNION ALL
(
  SELECT 6, 'public', 'ALTER DATABASE ' || database_name ||  ' SET disableinheritedprivileges = ' || current_value || ';',
         'DATABASE', database_name
    FROM v_internal.vs_configuration_parameters
   CROSS JOIN v_catalog.databases
   WHERE parameter_name = 'DisableInheritedPrivileges'
)
UNION ALL -- TABLE PRIV INHERITENCE
(
  SELECT 7, 'public' , 'ALTER TABLE ' || table_schema || '.' || table_name ||
         CASE WHEN inheritprivileges THEN ' INCLUDE PRIVILEGES;' ELSE ' EXCLUDE PRIVILEGES;' END,
         'TABLE' AS object_type,
         table_schema || '.' || table_name AS object_name
    FROM v_internal.vs_tables
    JOIN v_catalog.tables ON (table_id = oid)
)
UNION ALL -- VIEW PRIV INHERITENCE
(
  SELECT 8, 'public', 'ALTER VIEW ' || table_schema || '.' || table_name || CASE WHEN inherit_privileges THEN ' INCLUDE PRIVILEGES;' ELSE ' EXCLUDE PRIVILEGES; ' END,
         'TABLE' AS object_type, table_schema || '.' || table_name AS object_name
    FROM v_catalog.views
)
UNION ALL
(
  SELECT 9, owner_name, 'ALTER TABLE ' || table_schema || '.' || table_name || ' OWNER TO ' || owner_name || ';',
         'TABLE', table_schema || '.' || table_name
    FROM v_catalog.tables
)
UNION ALL
(
  SELECT 10, owner_name, 'ALTER VIEW ' || table_schema || '.' || table_name || ' OWNER TO ' || owner_name || ';', 'TABLE',
         table_schema || '.' || table_name
    FROM v_catalog.views
);

From the Linux command line, run the script in the user_ddl.sql file:
```
$ vsql -f user_ddl.sql
CREATE VIEW
```
Connect to the database using vsql.
Export the content of the user_ddl's sql column ordered on the grant_order column to a file:
```
=> \o pre-upgrade.txt
=> SELECT sql FROM user_ddl ORDER BY grant_order ASC;
=> \o
```
Upgrade the database.

Select and save to a different file the view's SQL column with the same command.

=> \o post-upgrade.txt
=> SELECT sql FROM user_ddl ORDER BY grant_order ASC;
=> \o

Create a diff between pre-upgrade.txt and post-upgrade.txt. This collects the missing grants into grants-list.txt.
```
$ diff pre-upgrade.txt post-upgrade.txt > grants-list.txt
```
To restore any missing grants, run the remaining grants in grants-list.txt, if any:
```
=> \i 'grants-list.txt'
```

Note

Attempting to restore grants to users with the ANY keyword triggers the following error:

ERROR 4856: Syntax error at or near "Any" at character

To avoid this error, use () instead of (ANY) as shown in the following example:

=> GRANT EXECUTE ON FUNCTION public.MapLookup() TO public;
GRANT PRIVILEGE

Setup: Nonsequential FIPS database upgrades

Mon, 01 Jan 0001 00:00:00 +0000

As of Vertica 10.1.1, FIPS support has been reinstated. Prior to this, the last version to support FIPS was Vertica 9.2.x. If you are upgrading from 9.2.x and want to maintain your FIPS certification, you must first perform a direct upgrade from 9.2.x to 10.1.1 before performing further upgrades.

The following procedure performs a direct upgrade from Vertica 9.2.x running on RHEL 6.x to Vertica 10.1.1 on RHEL 8.1.

Important

If you have any questions or want additional guidance for performing this upgrade, contact Vertica Support.

Create a full backup of your Vertica 9.2.x database. This example uses the configuration file fullRestore.ini.

$ vbr --config-file=/tmp/fullRestore.ini -t init
$ vbr --config-file=/tmp/fullRestore.ini -t backup

[Transmission]
concurrency_backup = 1
port_rsync = 50000
encrypt = False
serviceAccessPass = rsyncpw
hardLinkLocal = False
checksum = False
total_bwlimit_restore = 0
serviceAccessUser = rsyncuser
total_bwlimit_backup = 0
concurrency_restore = 1

[Misc]
snapshotName = full_restore
restorePointLimit = 1
retryDelay = 1
objects =
retryCount = 0
tempDir = /tmp/vbr

[Mapping]
v_fips_db_node0001 = 198.51.100.0:/home/release/backup/
v_fips_db_node0002 = 198.51.100.1:/home/release/backup/
v_fips_db_node0003 = 198.51.100.2:/home/release/backup/

[Database]
dbPort = 5433
dbPromptForPassword = False
dbUser =
dbPassword =
dbName = fips_db

Shut down the database gracefully. Do not start the database until instructed.
Acquire a RHEL 8.1 cluster with one of the following methods:
1. Upgrade in place
2. Reimage your machines
3. Use a completely different RHEL 8.1 cluster
Enable FIPS on your RHEL 8.1 machines and reboot.
```
$ fips-mode-setup --enable
```

Install Vertica 10.1.1 on the RHEL 8.1 cluster.

$ install_vertica --hosts node0001, node0002, node0003 \
    --rpm /tmp/vertica-10.1.1.x86_64.RHEL8.rpm

If you acquired your RHEL 8.1 cluster by reimaging or using a different cluster, you must restore your database.
```
$ vbr -c /tmp/fullRestore.ini -t restore
```
If you encounter the following warning, you can safely ignore it.
```
Warning: Vertica versions do not match: v9.2.1-xx -> v10.1.1-xxxxxxxx. This operation may not be supported.
```
Start the Vertica 10.1.1 database to trigger the upgrade. This should be the first time you've started your database since shutting it down in step 2.
```
$ admintools -t start_db -d fips_db
```