OpenText Analytics Database 26.2.x – Managing table columns

Admin: Renaming columns

Mon, 01 Jan 0001 00:00:00 +0000

You rename a column with ALTER TABLE as follows:

ALTER TABLE [schema.]table-name  RENAME [ COLUMN ] column-name TO new-column-name

The following example renames a column in the Retail.Product_Dimension table from Product_description to Item_description:

=> ALTER TABLE Retail.Product_Dimension
    RENAME COLUMN Product_description TO Item_description;

If you rename a column that is referenced by a view, the column does not appear in the result set of the view even if the view uses the wild card (*) to represent all columns in the table. Recreate the view to incorporate the column's new name.

Admin: Changing scalar column data type

Mon, 01 Jan 0001 00:00:00 +0000

In general, you can change a column's data type with ALTER TABLE if doing so does not require storage reorganization. After you modify a column's data type, data that you load conforms to the new definition.

The sections that follow describe requirements and restrictions associated with changing a column with a scalar (primitive) data type. For information on modifying complex type columns, see Adding a new field to a complex type column.

Supported data type conversions

OpenText™ Analytics Database supports conversion for the following data types:

Data Types	Supported Conversions
Binary	Expansion and contraction.
Character	All conversions between CHAR, VARCHAR, and LONG VARCHAR.
Exact numeric	All conversions between the following numeric data types: integer data types—INTEGER, INT, BIGINT, TINYINT, INT8, SMALLINT—and NUMERIC values of scale <=18 and precision 0. You cannot modify the scale of NUMERIC data types; however, you can change precision in the ranges (0-18), (19-37), and so on.
Collection	The following conversions are supported: Collection of one element type to collection of another element type, if the source element type can be coerced to the target element type. Between arrays and sets. Collection type to the same type (array to array or set to set), to change bounds or binary size. For details, see Changing Collection Columns.

Unsupported data type conversions

The database does not allow data type conversion on types that require storage reorganization:

Boolean
DATE/TIME
Approximate numeric type
BINARY to VARBINARY and vice versa

You also cannot change a column's data type if the column is one of the following:

Primary key
Foreign key
Included in the SEGMENTED BY clause of any projection for that table.

You can work around some of these restrictions. For details, see Working with column data conversions.

Admin: Adding a new field to a complex type column

Mon, 01 Jan 0001 00:00:00 +0000

You can add new fields to columns of complex types (any combination or nesting of arrays and structs) in native tables. To add a field to an existing table's column, use a single ALTER TABLE statement.

Requirements and restrictions

The following are requirements and restrictions associated with adding a new field to a complex type column:

New fields can only be added to rows/structs.
The new type definition must contain all of the existing fields in the complex type column. Dropping existing fields from the complex type is not allowed. All of the existing fields in the new type must exactly match their definitions in the old type.This requirement also means that existing fields cannot be renamed.
New fields can only be added to columns of native (non-external) tables.
New fields can be added at any level within a nested complex type. For example, if you have a column defined as ROW(id INT, name ROW(given_name VARCHAR(20), family_name VARCHAR(20)), you can add a middle_name field to the nested ROW.
New fields can be of any type, either complex or primitive.
Blank field names are not allowed when adding new fields. Note that blank field names in complex type columns are allowed when creating the table. The database automatically assigns a name to each unnamed field.
If you change the ordering of existing fields using ALTER TABLE, the change affects existing data in addition to new data. This means it is possible to reorder existing fields.
When you call ALTER COLUMN ... SET DATA TYPE to add a field to a complex type column, the database will place an O lock on the table preventing DELETE, UPDATE, INSERT, and COPY statements from accessing the table and blocking SELECT statements issued at SERIALIZABLE isolation level, until the operation completes.
Performance is slower when adding a field to an array element than when adding a field to an element not nested in an array.

Examples

Adding a field

Consider a company storing customer data:

=> CREATE TABLE customers(id INT, name VARCHAR, address ROW(street VARCHAR, city VARCHAR, zip INT));
CREATE TABLE

The company has just decided to expand internationally, so now needs to add a country field:

=> ALTER TABLE customers ALTER COLUMN address
SET DATA TYPE ROW(street VARCHAR, city VARCHAR, zip INT, country VARCHAR);
ALTER TABLE

You can view the table definition to confirm the change:


=> \d customers
List of Fields by Tables
 Schema |   Table   | Column  |                                 Type                                 | Size | Default | Not Null | Primary Key | Foreign Key
--------+-----------+---------+----------------------------------------------------------------------+------+---------+----------+-------------+-------------
 public | customers | id      | int                                                                  |    8 |         | f        | f           |
 public | customers | name    | varchar(80)                                                          |   80 |         | f        | f           |
 public | customers | address | ROW(street varchar(80),city varchar(80),zip int,country varchar(80)) |   -1 |         | f        | f           |
 (3 rows)

You can also see that the country field remains null for existing customers:

=> SELECT * FROM customers;
 id | name |                                    address
----+------+--------------------------------------------------------------------------------
  1 | mina | {"street":"1 allegheny square east","city":"hamden","zip":6518,"country":null}
 (1 row)

Common error messages

While you can add one or more fields with a single ALTER TABLE statement, existing fields cannot be removed. The following example throws an error because the city field is missing:

=> ALTER TABLE customers ALTER COLUMN address SET DATA TYPE ROW(street VARCHAR, state VARCHAR, zip INT, country VARCHAR);
ROLLBACK 2377:  Cannot convert column "address" from "ROW(varchar(80),varchar(80),int,varchar(80))" to type "ROW(varchar(80),varchar(80),int,varchar(80))"

Similarly, you cannot alter the type of an existing field. The following example will throw an error because the zip field's type cannot be altered:

=> ALTER TABLE customers ALTER COLUMN address SET DATA TYPE ROW(street VARCHAR, city VARCHAR, zip VARCHAR, country VARCHAR);
ROLLBACK 2377:  Cannot convert column "address" from "ROW(varchar(80),varchar(80),int,varchar(80))" to type "ROW(varchar(80),varchar(80),varchar(80),varchar(80))"

Additional properties

A complex type column's field order follows the order specified in the ALTER command, allowing you to reorder a column's existing fields. The following example reorders the fields of the address column:

=> ALTER TABLE customers ALTER COLUMN address
SET DATA TYPE ROW(street VARCHAR, country VARCHAR, city VARCHAR, zip INT);
ALTER TABLE

The table definition shows the address column's fields have been reordered:


=> \d customers
List of Fields by Tables
 Schema |   Table   | Column  |                                 Type                                 | Size | Default | Not Null | Primary Key | Foreign Key
--------+-----------+---------+----------------------------------------------------------------------+------+---------+----------+-------------+-------------
 public | customers | id      | int                                                                  |    8 |         | f        | f           |
 public | customers | name    | varchar(80)                                                          |   80 |         | f        | f           |
 public | customers | address | ROW(street varchar(80),country varchar(80),city varchar(80),zip int) |   -1 |         | f        | f           |
 (3 rows)

Note that you cannot add new fields with empty names. When creating a complex table, however, you can omit field names, and the database automatically assigns a name to each unnamed field:

=> CREATE TABLE products(name VARCHAR, description ROW(VARCHAR));
CREATE TABLE

Because the field created in the description column has not been named, the database assigns it a default name. This default name can be checked in the table definition:

=> \d products
List of Fields by Tables
 Schema |  Table   |   Column    |        Type         | Size | Default | Not Null | Primary Key | Foreign Key
--------+----------+-------------+---------------------+------+---------+----------+-------------+-------------
 public | products | name        | varchar(80)         |   80 |         | f        | f           |
 public | products | description | ROW(f0 varchar(80)) |   -1 |         | f        | f           |
(2 rows)

Above, we see that the VARCHAR field in the description column was automatically assigned the name f0. When adding new fields, you must specify the existing database-assigned field name:

=> ALTER TABLE products ALTER COLUMN description
SET DATA TYPE ROW(f0 VARCHAR(80), expanded_description VARCHAR(200));
ALTER TABLE

Admin: Defining column values

Mon, 01 Jan 0001 00:00:00 +0000

You can define a column so the database automatically sets its value from an expression through one of the following clauses:

DEFAULT
SET USING
DEFAULT USING

DEFAULT

The DEFAULT option sets column values to a specified value. It has the following syntax:

DEFAULT default-expression

Default values are set when you:

Load new rows into a table, for example, with INSERT or COPY. The database populates DEFAULT columns in new rows with their default values. Values in existing rows, including columns with DEFAULT expressions, remain unchanged.
Execute UPDATE on a table and set the value of a DEFAULT column to DEFAULT:
```
=> UPDATE table-name SET column-name=DEFAULT;
```
Add a column with a DEFAULT expression to an existing table. The database populates the new column with its default values when it is added to the table.

Note
Altering an existing table column to specify a DEFAULT expression has no effect on existing values in that column. The database applies the DEFAULT expression only on new rows when they are added to the table, through load operations such as INSERT and COPY. To refresh all values in a column with the column's DEFAULT expression, update the column as shown above.

Restrictions

DEFAULT expressions cannot specify volatile functions with ALTER TABLE...ADD COLUMN. To specify volatile functions, use CREATE TABLE or ALTER TABLE...ALTER COLUMN statements.

SET USING

The SET USING option sets the column value to an expression when the function REFRESH_COLUMNS is invoked on that column. This option has the following syntax:

SET USING using-expression

This approach is useful for large denormalized (flattened) tables, where multiple columns get their values by querying other tables.

Restrictions

SET USING has the following restrictions:

Volatile functions are not allowed.
The expression cannot specify a sequence.
The database limits the use of several meta-functions that copy table data: COPY_TABLE, COPY_PARTITIONS_TO_TABLE, MOVE_PARTITIONS_TO_TABLE, and SWAP_PARTITIONS_BETWEEN_TABLES:
- If the source and target tables both have SET USING columns, the operation is permitted only if each source SET USING column has a corresponding target SET USING column.
- If only the source table has SET USING columns, SWAP_PARTITIONS_BETWEEN_TABLES is disallowed.
- If only the target table has SET USING columns, the operation is disallowed.

DEFAULT USING

The DEFAULT USING option sets DEFAULT and SET USING constraints on a column, equivalent to using DEFAULT and SET USING separately with the same expression on the same column. It has the following syntax:

DEFAULT USING expression

For example, the following column definitions are effectively identical:

=> ALTER TABLE public.orderFact ADD COLUMN cust_name varchar(20)
     DEFAULT USING (SELECT name FROM public.custDim WHERE (custDim.cid = orderFact.cid));

=> ALTER TABLE public.orderFact ADD COLUMN cust_name varchar(20)
     DEFAULT (SELECT name FROM public.custDim WHERE (custDim.cid = orderFact.cid))
     SET USING (SELECT name FROM public.custDim WHERE (custDim.cid = orderFact.cid));

DEFAULT USING supports the same expressions as SET USING and is subject to the same restrictions.

Supported expressions

DEFAULT and SET USING generally support the same expressions. These include:

Queries
Other columns in the same table
Literals (constants)
All operators supported by OpenText™ Analytics Database
The following categories of functions:

Expression restrictions

The following restrictions apply to DEFAULT and SET USING expressions:

The return value data type must match or be cast to the column data type.
The expression must return a value that conforms to the column bounds. For example, a column that is defined as a VARCHAR(1) cannot be set to a default string of abc.
In a temporary table, DEFAULT and SET USING do not support subqueries. If you try to create a temporary table where DEFAULT or SET USING use subquery expressions, the database returns an error.
A column's SET USING expression cannot specify another column in the same table that also sets its value with SET USING. Similarly, a column's DEFAULT expression cannot specify another column in the same table that also sets its value with DEFAULT, or whose value is automatically set to a sequence. However, a column's SET USING expression can specify another column that sets its value with DEFAULT.

Note
You can set a column's DEFAULT expression from another column in the same table that sets its value with SET USING. However, the DEFAULT column is typically set to NULL, as it is only set on load operations that initially set the SET USING column to NULL.

DEFAULT and SET USING expressions only support one SELECT statement; attempts to include multiple SELECT statements in the expression return an error. For example, given table t1:

=> SELECT * FROM t1;
 a |    b
---+---------
 1 | hello
 2 | world
(2 rows)

Attempting to create table t2 with the following DEFAULT expression returns with an error:

=> CREATE TABLE t2 (aa int, bb varchar(30) DEFAULT (SELECT 'I said ')||(SELECT b FROM t1 where t1.a = t2.aa));
ERROR 9745:  Expressions with multiple SELECT statements cannot be used in 'set using' query definitions

Disambiguating predicate columns

If a SET USING or DEFAULT query expression joins two columns with the same name, the column names must include their table names. Otherwise, the database assumes that both columns reference the dimension table, and the predicate always evaluates to true.

For example, tables orderFact and custDim both include column cid. Flattened table orderFact defines column cust_name with a SET USING query expression. Because the query predicate references columns cid from both tables, the column names are fully qualified:

=> CREATE TABLE public.orderFact
(
    ...
    cid int REFERENCES public.custDim(cid),
    cust_name varchar(20) SET USING (
        SELECT name FROM public.custDim WHERE (custDIM.cid = orderFact.cid)),
    ...
)

Examples

Derive a column's default value from another column

Create table t with two columns, date and state, and insert a row of data:

=> CREATE TABLE t (date DATE, state VARCHAR(2));
CREATE TABLE
=> INSERT INTO t VALUES (CURRENT_DATE, 'MA');
 OUTPUT
--------
      1
(1 row)

=> COMMIT;
COMMMIT
SELECT * FROM t;
    date    | state
------------+-------
 2017-12-28 | MA
(1 row)

Use ALTER TABLE to add a third column that extracts the integer month value from column date:
```
=> ALTER TABLE t ADD COLUMN month INTEGER DEFAULT date_part('month', date);
ALTER TABLE
```

When you query table t, the database returns the number of the month in column date:

=> SELECT * FROM t;
    date    | state | month
------------+-------+-------
 2017-12-28 | MA    |    12
(1 row)

Update default column values

Update table t by subtracting 30 days from date:

=> UPDATE t SET date = date-30;
 OUTPUT
--------
      1
(1 row)

=> COMMIT;
COMMIT
=> SELECT * FROM t;
    date    | state | month
------------+-------+-------
 2017-11-28 | MA    |    12
(1 row)

The value in month remains unchanged.

Refresh the default value in month from column date:

=> UPDATE t SET month=DEFAULT;
 OUTPUT
--------
      1
(1 row)

=> COMMIT;
COMMIT
=> SELECT * FROM t;
    date    | state | month
------------+-------+-------
 2017-11-28 | MA    |    11
(1 row)

Derive a default column value from user-defined scalar function

This example shows a user-defined scalar function that adds two integer values. The function is called add2ints and takes two arguments.

Develop and deploy the function, as described in Scalar functions (UDSFs).

Create a sample table, t1, with two integer columns:

=> CREATE TABLE t1 ( x int, y int );
CREATE TABLE

Insert some values into t1:

=> insert into t1 values (1,2);
OUTPUT
--------
      1
(1 row)
=> insert into t1 values (3,4);
 OUTPUT
--------
      1
(1 row)

Use ALTER TABLE to add a column to t1, with the default column value derived from the UDSF add2ints:
```
alter table t1 add column z int default add2ints(x,y);
ALTER TABLE
```

List the new column:

select z from t1;
 z
----
  3
  7
(2 rows)

Table with a SET USING column that queries another table for its values

Define tables t1 and t2. Column t2.b is defined to get its data from column t1.b, through the query in its SET USING clause:
```
=> CREATE TABLE t1 (a INT PRIMARY KEY ENABLED, b INT);
CREATE TABLE

=> CREATE TABLE t2 (a INT, alpha VARCHAR(10),
      b INT SET USING (SELECT t1.b FROM t1 WHERE t1.a=t2.a))
      ORDER BY a SEGMENTED BY HASH(a) ALL NODES;
CREATE TABLE
```
Important

The definition for table t2 includes SEGMENTED BY and ORDER BY clauses that exclude SET USING column b. If these clauses are omitted, the database creates an auto-projection for this table that specifies column b in its SEGMENTED BY and ORDER BY clauses . Inclusion of a SET USING column in any projection's segmentation or sort order prevents function REFRESH_COLUMNS from populating this column. Instead, it returns with an error.

For details on this and other restrictions, see REFRESH_COLUMNS.

Populate the tables with data:

=> INSERT INTO t1 VALUES(1,11),(2,22),(3,33),(4,44);
=> INSERT INTO t2 VALUES (1,'aa'),(2,'bb');
=> COMMIT;
COMMIT

View the data in table t2: Column in SET USING column b is empty, pending invocation of function REFRESH_COLUMNS:
```
=> SELECT * FROM t2;
 a | alpha | b
---+-------+---
 1 | aa    |
 2 | bb    |
(2 rows)
```
Refresh the column data in table t2 by calling function REFRESH_COLUMNS:
```
=> SELECT REFRESH_COLUMNS ('t2','b', 'REBUILD');
      REFRESH_COLUMNS
---------------------------
 refresh_columns completed
(1 row)
```
In this example, REFRESH_COLUMNS is called with the optional argument REBUILD. This argument specifies to replace all data in SET USING column b. It is generally good practice to call REFRESH_COLUMNS with REBUILD on any new SET USING column. For details, see REFRESH_COLUMNS.
View data in refreshed column b, whose data is obtained from table t1 as specified in the column's SET USING query:
```
=> SELECT * FROM t2 ORDER BY a;
  a | alpha | b
---+-------+----
 1 | aa    | 11
 2 | bb    | 22
(2 rows)
```

Expressions with correlated subqueries

DEFAULT and SET USING expressions support subqueries that can obtain values from other tables, and use those with values in the current table to compute column values. The following example adds a column gmt_delivery_time to fact table customer_orders. The column specifies a DEFAULT expression to set values in the new column as follows:

Calls meta-function NEW_TIME, which performs the following tasks:
- Uses customer keys in customer_orders to query the customers dimension table for customer time zones.
- Uses the queried time zone data to convert local delivery times to GMT.
Populates the gmt_delivery_time column with the converted values.

=> CREATE TABLE public.customers(
    customer_key int,
    customer_name varchar(64),
    customer_address varchar(64),
    customer_tz varchar(5),
    ...);

=> CREATE TABLE public.customer_orders(
    customer_key int,
    order_number int,
    product_key int,
    product_version int,
    quantity_ordered int,
    store_key int,
    date_ordered date,
    date_shipped date,
    expected_delivery_date date,
    local_delivery_time timestamptz,
    ...);

=> ALTER TABLE customer_orders ADD COLUMN gmt_delivery_time timestamp
   DEFAULT NEW_TIME(customer_orders.local_delivery_time,
      (SELECT c.customer_tz FROM customers c WHERE (c.customer_key = customer_orders.customer_key)),
      'GMT');

OpenText Analytics Database 26.2.x – Managing table columns

Admin: Renaming columns

Admin: Changing scalar column data type

Supported data type conversions

Unsupported data type conversions

Admin: Adding a new field to a complex type column

Requirements and restrictions

Examples

Adding a field

Common error messages

Additional properties

Admin: Defining column values

DEFAULT

Note

Restrictions

SET USING

Restrictions

DEFAULT USING

Supported expressions

Expression restrictions

Note

Disambiguating predicate columns

Examples

Derive a column's default value from another column

Update default column values

Derive a default column value from user-defined scalar function

Table with a SET USING column that queries another table for its values

Important

Expressions with correlated subqueries