Admin: Setting a purge policy

Mon, 01 Jan 0001 00:00:00 +0000

The preferred method for purging data is to establish a policy that determines which deleted data is eligible to be purged. Eligible data is automatically purged when the Tuple Mover performs mergeout operations.

The database provides two methods for determining when deleted data is eligible to be purged:

Specifying the time for which delete data is saved
Specifying the number of epochs that are saved

Specifying the time for which delete data is saved

Specifying the time for which delete data is saved is the preferred method for determining which deleted data can be purged. By default, the database saves historical data only when nodes are down.

To change the specified time for saving deleted data, use the HistoryRetentionTime configuration parameter:

=> ALTER DATABASE DEFAULT SET HistoryRetentionTime = {seconds | -1};

In the above syntax:

seconds is the amount of time (in seconds) for which to save deleted data.
-1 indicates that you do not want to use the HistoryRetentionTime configuration parameter to determine which deleted data is eligible to be purged. Use this setting if you prefer to use the other method (HistoryRetentionEpochs) for determining which deleted data can be purged.

The following example sets the history epoch retention level to 240 seconds:

=> ALTER DATABASE DEFAULT SET HistoryRetentionTime = 240;

Specifying the number of epochs that are saved

Unless you have a reason to limit the number of epochs, it is recommended that you specify the time for which deleted data is saved.

To specify the number of historical epoch to save through the HistoryRetentionEpochs configuration parameter:

Turn off the HistoryRetentionTime configuration parameter:

=> ALTER DATABASE DEFAULT SET HistoryRetentionTime = -1;

Set the history epoch retention level through the HistoryRetentionEpochs configuration parameter:
```
=> ALTER DATABASE DEFAULT SET HistoryRetentionEpochs = {num_epochs | -1};
```
- num_epochs is the number of historical epochs to save.
- -1 indicates that you do not want to use the HistoryRetentionEpochs configuration parameter to trim historical epochs from the epoch map. By default, HistoryRetentionEpochs is set to -1.

The following example sets the number of historical epochs to save to 40:

=> ALTER DATABASE DEFAULT SET HistoryRetentionEpochs = 40;

Modifications are immediately implemented across all nodes within the database cluster. You do not need to restart the database.

Note

If both HistoryRetentionTime and HistoryRetentionEpochs are specified, HistoryRetentionTime takes precedence.

See Epoch management parameters for additional details. See Epochs for information about how epochs work.

Disabling purge

If you want to preserve all historical data, set the value of both historical epoch retention parameters to -1, as follows:

=> ALTER DABABASE mydb SET HistoryRetentionTime = -1;
=> ALTER DATABASE DEFAULT SET HistoryRetentionEpochs = -1;

Admin: Manually purging data

Mon, 01 Jan 0001 00:00:00 +0000

You manually purge deleted data as follows:

Set the cut-off date for purging deleted data. First, call one of the following functions to verify the current ancient history mark (AHM):
- GET_AHM_TIME returns a TIMESTAMP value of the AHM.
- GET_AHM_EPOCH returns the number of the epoch in which the AHM is located.
Set the AHM to the desired cut-off date with one of the following functions:
- SET_AHM_TIME sets the AHM to the epoch that includes the specified TIMESTAMP value on the initiator node.
- SET_AHM_EPOCH sets the AHM to the specified epoch.
- MAKE_AHM_NOW sets the AHM to the greatest allowable value. This lets you purge all deleted data.
If you call SET_AHM_TIME, keep in mind that the timestamp you specify is mapped to an epoch, which by default has a three-minute granularity. Thus, if you specify an AHM time of 2008-01-01 00:00:00.00, the database might purge data from the first three minutes of 2008, or retain data from last three minutes of 2007.

Note
You cannot advance the AHM beyond a point where the database is unable to recover data for a down node.
Purge deleted data from the desired projections with one of the following functions:
- PURGE purges all projections in the physical schema.
- PURGE_TABLE purges all projections anchored to the specified table.
- PURGE_PROJECTION purges the specified projection.
- PURGE_PARTITION purges a specified partition.
The tuple mover performs a mergeout operation to purge the data. The database periodically invokes the tuple mover to perform mergeout operations, as configured by tuple mover parameters. You can manually invoke the tuple mover by calling the function DO_TM_TASK.

Caution

Manual purge operations can take a long time.

See Epochs for additional information about how epochs work.

OpenText Analytics Database 26.2.x – Purging deleted data

Admin: Setting a purge policy

Specifying the time for which delete data is saved

Specifying the number of epochs that are saved

Note

Disabling purge

Admin: Manually purging data

Note

Caution