OpenText Analytics Database 26.2.x – Overview of UDx

Ot-Cad: UDx

Mon, 01 Jan 0001 00:00:00 +0000

OTCAD enables you to create, list, and delete custom UDx, so that you can integrate them with existing database tables. UDx allow you to reuse and share code that extends built-in functionality. Use UDx to perform specific tasks such as complex calculations, transformations, or custom data manipulations.

You can reuse a UDx several times. A function always returns a value explicitly by specifying an expression. It is a good choice to calculate and return a value. You can use UDFs to extend built-in functions or to encapsulate calculations that are standard for your organization. UDFs you create can be called in a way similar to built-in functions.

You write a UDF’s logic (it's handler) – in python, C++, or java.

About UDx

You can view and manage UDx. After logging in to OTCAD, the home page appears. Select the More options button and UDx. The User-defined extensions page appears with the Functions and UDx libraries tabs.

The Functions tab lists the functions that are available in the system. The Active SDK version label displays the current SDK version of the database. The following columns are displayed:

Schema name - The name of the schema.
Function name - The name of the function.
Function type - The type of the function. Choose from among Scalar, Aggregate, Analytic, and Transform.
SDK version - The version of the SDK used to create the library. This is applicable only for C++ and java. For python, this column is empty.
Language - The language in which the function is developed. For e.g, Python.
Library - The name of the library to which the UDx is associated.
Created by - The user ID of the person who created the function.
Actions - Option to delete the function. For more information about deleting a function, see Delete a function.

In the Search function name field, enter the name of the function. The function is displayed in the Functions tab of the User-defined extensions page.

The UDx libraries tab lists the UDx along with the status that are available in the system. The following columns are displayed:

Library - The name of the library containing the UDx.
Language - The language in which the library is developed. For e.g, Python.
Functions - The number of functions associated with the UDx.
Created by -The user ID of the person who created the library.
Creation date - The date and time of creation of the library.
Status - The present status of the UDx library.
Details - Details about the UDx in the system.
Actions - Option to delete the UDx library. For more information about deleting a UDx library, see Delete a UDx library.

In the Search library name field, enter the name of the UDx library. The UDx library is displayed in the UDX libraries tab of the User-defined extensions page.

Create a library

To create a library, do the following in the User-defined extensions page:

Select +Add new.

The Create a library page appears.
In the Library name field, enter the name of the library.
In the Schema list, choose the required schema.
In the Language list, choose the required programming language of the functions in the library - Python, Java, or C++.

In the File name field, enter the name of the file.

Note

  Ensure that the class names are present in this file.

Click Browse and choose the file that contains the file you selected in the previous step.

Note

  For python, you can upload only .zip files. Ensure that the file is available immediately in the root folder. For example, root/my_file.py. 
  For C++, you can upload only .so files. For java, you can upload JAR files. The maximum file size allowed for all languages is 300 MB. Uploaded files are quarantined for antivirus scanning. If no threats are found, OTCAD creates the function.

Note

   The uploading party is solely responsible for any code or content they submit. OpenText does not review or endorse uploads and assumes no liability for them. By uploading, users acknowledge and accept full responsibility for any consequences that may arise.

In the Function area, for the Name field, enter the name of the function.
In the Function area, for the Class field, enter the name of the class that is defined in the file selected in step 5. For e.g, MultiplyAndSumFactory.
From the Type list, select the UDx type:
- Scalar
- Aggregate
- Analytic
- Transform
For more information about the UDx types and the supported languages, see Developing user-defined extensions (UDx).
Click +Add to add another function. The newly added functions appear in the UDx libraries tab.
Note
```
   You need to have super user privilege to create the library.  
```
Click Create.

The library is created and displayed in the UDx libraries tab of the User-defined extensions page.

Delete a function

You can delete functions that are no longer used or needed.

In the Functions tab of the User defined extensions page, select ⋮ in the Action column.

Select Delete.

Note

   Deleting a function removes it from the database. However, references to the function are not deleted. For example, SQL query.

In the Confirmation dialog, select Delete.

Delete a UDx library

You can delete UDx libraries that are no longer used or needed.

In the UDx Libraries tab of the User defined extensions page, select ⋮ in the Action column.

Select Delete.

Note

   Deleting a UDx library also deletes the associated UDx function. This may affect all locations where UDx functions are used. You will not be able to retrieve the UDx function after it is deleted as backups will not be available.

In the Confirmation dialog, select Delete.

Ot-Cad: Developing user-defined extensions (UDx)

Mon, 01 Jan 0001 00:00:00 +0000

The primary strengths of UDx are:

You can use them wherever you use internal functions.
They take full advantage of the distributed computing feature of the system. The extensions usually execute in parallel on each node in the cluster.
The complicated aspects of developing a distributed piece of analytic code are handled by the system. Your main programming task is to read in data, process it, and then write it out using the SDK APIs.

Fenced mode

Fenced mode runs the UDx code outside of the main system process. Fenced UDx crashes do not impact the core system process. There is a small performance impact when running UDx code in fenced mode. On average, using fenced mode adds about 10% more time to execution.

Fenced mode is currently available for all UDx with the exception of user-defined aggregates. All UDx run in fenced mode, since the Python runtimes cannot run directly within the system process. Using fenced mode does not affect the development of your UDx. Fenced mode is enabled by default for all UDx.

OTCAD does not support the unfenced mode.

Developing with the database SDK

Before you develop your UDx, you need to configure your development environment. The development environment must use the same database version as the production environment. For guidance on obtaining OpenText SDK, contact Technical Support. For more information about configuring your development environment, see Developing with the OpenText Analytics Database SDK.

Database upgrade

When the database is upgraded, the SDK is also upgraded. Your UDx libraries depend on that SDK. If the SDK changes, your existing libraries may no longer work. You need to recompile your library using the new SDK, delete the old library, and upload the newly compiled one. If you do not recompile your library using the new SDK, the error "Library built with incompatible SDK version. Rebuild with SDK version [string] and recreate the library" appears when loading or running the UDx.

If you make any changes to your UDx code, you must compile a new version of the library and upload it again. This also requires removing the existing library and replacing it with your updated one. The database then runs the latest version of your UDx library.

UDx types

OTCAD supports four types of user-defined extensions:

User-defined aggregate functions (UDAFs)

UDAFs allow you to create custom Aggregate functions specific to your needs. They read one column of data and return one output column. UDAFs can be developed in C++. An Aggregate function performs an operation on a set of values and returns one value. The system provides standard built-in aggregate functions such as AVG, MAX, and MIN. For more information about UDAF, see Aggregate functions (UDAFs).

User-defined analytic functions (UDAnFs)

UDAnFs are similar to UDSFs, in that they read a row of data and return a single row. However, the function can read input rows independently of outputting rows, so that the output values can be calculated over several input rows. UDAnFs can be developed in C++ and Java and are used for analytics. UDAnFs must output a single value for each row of data read and can have no more than 9800 arguments. For more information about UDAnF, see Analytic functions (UDAnFs).

User-defined scalar functions (UDSFs)

UDSFs take in a single row of data and return a single value. UDSFs can be developed in C++, python, or java. A user-defined scalar function (UDSF) returns a single value for each row of data it reads. You can use a UDSF anywhere you can use a built-in database function. For more information about UDSF, see Scalar functions (UDSFs).

User-defined transform functions (UDTFs)

UDTFs operate on table partitions and return zero or more rows of data. The data they return can be an entirely new table, unrelated to the schema of the input table, with its own ordering and segmentation expressions. They can only be used in the SELECT list of a query. UDTFs can be developed in C++, python, and java.

To optimize query performance, you can use live aggregate projections to pre-aggregate the data that a UDTF returns. A user-defined transform function (UDTF) lets you transform a table of data into another table. It reads one or more arguments (as a row of data) and returns zero or more rows of data comprising one or more columns. For more information about UDTF, see Transform functions (UDTFs).

While each UDx type has a unique base class, developing them is similar in many ways. Different UDx types can also share the same library.

UDx types and supported languages

The UDx types and their supported languages are as follows:

UDx type	Java	C++	Python
User-defined scalar functions	Yes	Yes	Yes
User-defined aggregate functions	No	Yes	No
User-defined analytic functions	Yes	Yes	No
User-defined transform functions	Yes	Yes	Yes

For more information, see [Developing user-defined extensions (UDx)] Developing user-defined extensions (UDxs)