OpenText Core Analytics Database

• 1: OpenText Core Analytics Database Settings Overview
• 1.1: Connection Data Settings
• 1.2: CIDR
• 1.3: Connection Data Settings
• 2: Overview of Data Pipelines
• 2.1: Data Pipelines
• 3: Overview of Integrations
• 3.1: Integrations

1 - OpenText Core Analytics Database Settings Overview

You can configure the default database configuration settings in OpenText™ Core Analytics Database (OTCAD).

1. Log in to OTCAD.
2. Select Settings from More options. The Settings Overview page opens. In this page, you can configure the following OTCAD settings:
   • Connection data: You can configure the default connection data settings in OTCAD.
   • CIDR: CIDR is fundamental to subnetting, the process of dividing a large network into smaller, more manageable subnetworks. By using different prefix lengths, you can create subnets with varying numbers of host addresses, enabling efficient use of IP address space.
   • Scaling: Database scaling is the ability of a database to obtain or reduce resources depending on the system's load, establishing a direct correlation between the allocation of resources and their demand. Scaling ensures consistency in the database server quality while maximizing resource usage.
3. Select Connection Data.
4. Copy this information to connect your database with other applications or clients:
   • Host DNS: The DNS host provides the authoritative nameservers that answer or resolve queries for your website. This is how your site is found on the web.
   • Host Port: The host port is a network port on a computer or device that is used to establish a connection with another device or service, often for communication or data transfer.
   • Database Name: The unique name of the database.
   • Refresh Token: The refresh token is a special type of token used in authentication flows, particularly in OAuth 2.0, to obtain new access tokens when the current ones expire.
   • Access Token: An access token is a security credential that allows an application or user to access protected resources on a server. It acts as a digital key, verifying the user's or application's identity and authorization level, and is used to grant access without requiring the user to repeatedly provide their credentials.
5. Select Close to close the Connection Details page.

1.1 - Connection Data Settings

You can configure the default connection data settings in OTCAD. You can manage IP addresses with Classless Inter-Domain Routing (CIDR) and scale your plan to suit your requirements.

1.2 - CIDR

CIDR

Cloud infrastructure engineers use CIDR notation to manage IP address ranges for cloud-based networks. IP addresses enable resources in your Virtual Private Cloud (VPC) to communicate with each other and with resources over the Internet. CIDR notation is a way to represent an IP address and its network mask.

OpenText supports ranges of IP addresses using Classless Inter-Domain Routing (CIDR) notation. For example, 192.168.1.0/24 represents all IPv4 addresses in the range of 192.168.1.0 to 192.168.1.255.

The format of these addresses is as follows:

• An individual IPv4 address is 32 bits, with 4 groups of up to 3 decimal digits. For example, 10.0.1.0.
• An individual IPv6 address is 128 bits, with 8 groups of 4 hexadecimal digits. For example, 2001:0db8:85a3:0000:0000:8a2e:0370:7334.

Public IPv4 and IPv6 addresses

All subnets have an attribute that determines whether a network interface created in the subnet automatically receives a public IPv4 address. Therefore, when you launch an instance into a subnet that has this attribute enabled, a public IP address is assigned to the primary network interface that's created for the instance. You can control whether your instance receives a public IP address by doing the following:

• Modifying the public IP addressing attribute of your subnet.
• Enabling or disabling the public IP addressing feature during instance launch, which overrides the subnet's public IP addressing attribute.
• Unassigning a public IP address from your instance after launch by managing the IP addresses associated with a network interface.

Public IPv6 addresses are IPv6 addresses that can be configured to remain private or configured to be reachable over the Internet.

Private IPv4 and IPv6 addresses

Private IPv4 addresses are not reachable over the Internet and can be used for communication between the instances in your VPC. When you launch an instance into a VPC, a primary private IP address from the IPv4 address range of the subnet is assigned to the primary network interface (for example, eth0) of the instance. Each instance is also given a private (internal) DNS hostname that resolves to the private IP address of the instance.

You cannot share private IPv6 addresses on the Internet. You can use a private IPv6 address if you want your private networks to support IPv6 and you have no intention of routing traffic from these addresses to the Internet. You can connect to the internet from a resource that has a private IPv6 address. But you must route traffic through a resource in another subnet with a public IPv6 address.

Configure CIDR settings

To configure CIDR settings:

1. In the Settings Overview page, select View CIDR. The CIDR page displays.
2. Select Private.
3. To add an IP address, select Add IP.
4. From the Value drop-down list, select either IPv4 or IPv6 depending on the IP address you want to add. Do one of the following: a. Enter a valid IPv4 or IPv6 address. b. Copy a valid IPv4 or IPv6 address from another source and select Paste.
5. Select √.

1.3 - Connection Data Settings

Scaling

Workload scaling enhances application performance and resource efficiency in dynamic environments. Scaling ensures that applications can handle varying workloads without performance degradation. OpenText provides the ability to automatically scale resources up or down based on real-time metrics, allowing organizations to respond quickly to changes in traffic. This elasticity not only improves user experience but also optimizes resource utilization, helping to minimize costs associated with underused or overprovisioned resources.

You cam manage compute to your database to improve the performance of complex analytic queries that have been running for a long time. Adding compute helps these queries run faster.

Configure the compute of your database to suit your query requirements and workload performance needs.

To manage compute for your database:

1. In the Settings Overview page, select Scaling.
2. Select View Scaling. The horizontal database scaling options are displayed. You can scale the database either manually or automatically. Do one of the following: a. To manually scale the database, select Manual. b. Slide across the slider horizontally to scale the database depending on the size - small, small-medium, medium, medium-large, or large. c. Select Apply changes. d. In the View scaling log area, click the Audit log and Scaling log tabs to view the log details. <-Or-> a. To automatically scale the database, select Auto. Select this option only if you need to automatically scale the database based on defined parameters. b. From the Scale up to drop-down list, select either Large or Medium depending on the size of your database.

   Note

      Define the maximum compute beyond which the system cannot add anything and balance the loads automatically.
   

   c. Use the CPU usage toggle button to scale out the database when the CPU usage exceeds a value in the range 1% to 100%. d. Use the Memory usage toggle button to scale out the database when the memory usage exceeeds a value in the range 1% to 100%.

   Note

     When you apply both CPU usage and Memory usage, the system adds additional compute when either of the thresholds are reached.
   

   e. Select Apply changes. f. In the View scaling log area, click the Audit log and Scaling log tabs to view the log details. The Audit log provides information about the following:
   • Date & time
   • Event
   • Performed by The Scaling log provides information about the following:
   • Date & time
   • Scale event
   • Compute (units)
   • Triggered
   • Performed by

2 - Overview of Data Pipelines

Data pipelines allow you to define where, when, and how you need to load data into OpenText™ Analytics Database with minimal manual intervention.

2.1 - Data Pipelines

Data pipeline

A data pipeline also known as a data loader is a declarative, automated way to process data continuously from external sources like Amazon S3. They allow you to define where, when, and how you need to load data into OTCAD with minimal manual intervention.

Data pipelines UI tour

You can view and manage data pipelines. After logging in to the OTCAD application, you land on the home page. Select the More options button and Data Pipeline. The Data pipeline page appears as shown in this image:

This page displays the following information in the Overview card:

• Total number of pipelines - The Total Pipelines card displays the total number of pipelines configured in the system.
• Pipelines that failed execution - Select the Failed Execution drop-down list and the period for which you need to view the pipelines that failed to execute.
• Active pipelines - The Active Pipelines card displays the total number of pipelines that are in the Active status. The Pipelines area displays the following information:
• Pipeline name - The name of the data pipeline.
• Created by - The user ID of the person who created the data pipeline.
• Data source - The source location of the files that contain the data to be loaded.
• Schema - The tables, columns (fields), data types, and relationships among different tables in the database.
• Destination table - The database table where data is written after it has been processed or transformed from a source table or other data source.
• Last run on - The timestamp at which the data pipeline was last run.
• Last run status - Indicates the status of the data pipeline when it was last run.
• Pipeline status - Indicates the present status of the data pipeline.

Create a data pipeline

To create a data pipeline, do the following:

1. In the Data Pipelines page, select +Create a pipeline. The Create a pipeline page displays.
2. In the Pipeline name field, enter the name of the pipeline.
3. In the Access key ID field, enter your AWS account access key id.
4. In the Secret access key field, enter your AWS account secret access key.

   Note

   Provide valid AWS credentials for the Access key ID and Secret access key. Invalid AWS credentials do not allow you to create a data pipeline.
5. From the Region drop-down list, select the region (geography) of the S3 bucket where the files are present.
6. In the S3 Bucket/File/Folder path field, enter the name or the folder path where the files are present.
7. Select the Data is encrypted option to specify the following parameters:
   • Select either AWS Key Management Service Key or Customer managed keys if you wish to encrypt and load data into the S3 bucket.
   • Select Encryption key ID.
8. Select Next.
9. In the Retry limit field, specify the number of times for which the system needs to retry a failed file load.
10. In the Parameters field, specify the copy parameters. For more information, see Parameters.
11. Select Next.
12. From the Destination table drop-down list, select the destination table to which you need to load the data.
13. Select Next.
14. Specify the schedule at which the data pipeline needs to run. Do one of the following:
    • Select Schedule.
    • From the date pickers, select the Start date and End date.
    • In the Repeat every field, specify the duration at which the data pipeline needs to run.
    • From the Unit drop-down list, select the minute, hour, day, week, or month at which the data pipeline needs to run.
    • Select the On day radio button and specify the day on which the data pipeline needs to run.
    • Select the On radio button and specify the exact day and month on which the data pipeline needs to run.
    • Select the option Trigger when something is added to run the data pipeline when a file is added to the S3 bucket.
    • Enter the SQS credentials in the Access key ID, Secret access key, and Resource URL fields. <--Or-->
    • Select Execute once.
15. Select Finish. The data pipeline is created and displayed in the Data Pipelines page.

Filter a data pipeline

You can filter data pipelines based on certain criteria. Filter data pipelines in one of these ways:

1. A date range on which the data pipelines last ran.
   • Select the Filter icon in the Data Pipelines page.
   • Expand the Last run range drop-down list.
   • In the date picker, select a date range (start date and end date) for which you need to view the data pipelines. For example, select a range from 1 May to 31 May to view data pipelines that were created in the month of May.
2. Schema
   • Select the Filter icon in the Data Pipelines page.
   • Expand the Schema drop-down list.
   • Select the required schema.
3. Data source
   • Select the Filter icon in the Data Pipelines page.
   • Expand the Data source drop-down list.
   • Select the data source or data sources for which you wish to view the data pipeline.
4. Pipeline status
   • Select the Filter icon in the Data Pipelines page.
   • Expand the Pipeline status drop-down list.
   • Select Active to view the data pipelines that are active. A data pipeline is in the Active status when the schedule end date is a future date and there are one or more ingestions that are yet to complete.
   • Select Inactive to view the data pipelines that are inactive. A data pipeline is in the Inactive status either when the end date is past or there are no ingestions that are yet to complete.
5. Id of the person who created the pipeline
   • Select the Filter icon in the Data Pipelines page.
   • Expand the Created by drop-down list.
   • Select the user Id of the person who created the data pipeline.

Search a data pipeline

All data pipelines are displayed by default. You can search data pipelines using specific search criteria. To search data pipelines, do the following:

1. In the Data pipelines page, select +Search pipeline name or schema.
2. Enter one of the following search criteria:
   • Pipeline name
   • Owner of the data pipeline (Created by)
   • Schema
   • Destination table
3. You can sort the data pipeline by the pipeline name or the date on which it was last run. To sort the data pipelines in the ascending or descending order, select the Sort icon for the Pipeline name or Last run on column in the Data Pipelines page.

View pipeline details

You can view pipeline details in the Data pipelines page. To view the details of a data pipeline, do the following:

1. In the Data pipeline page, mouse over the Pipeline name column and select the +View details icon for a data pipeline. You can view the following details of the data pipeline in this page.
   • Overview - Displays the owner (user ID) of the data pipeline.
   • Configurations - Displays the source and destination paths of the data pipeline. In the Configurations card, click the Edit icon to edit the data source and destination paths of the data pipeline. For more information, see Creating a data pipeline.
   • Total instances - Displays the total number of jobs. Select the drop-down list and view information about the instances for the last 24 hours, last week, last month, last 6 months, last one year, or all time.
   • To view the reasons for the failure of a job, mouse over an instance with job status Failed and select the View error logs icon that appears for a file name. The error log provides information about the row that failed execution, reason for failure, and rejected data. You can troubleshoot the job with this information and ensure that this job is successfully executed.
   • Incident Overview - Displays information about the data pipeline job in a pie chart. Select the drop-down list and view information about the jobs for the last 24 hours, last week, last month, last 6 months, last one year, or all time.
2. Select Execute pipeline to execute the selected data pipeline. Executing a data pipeline loads all files that have not already been loaded and that have not reached the retry limit. Executing the data pipeline commits the transaction.
3. Select Edit pipeline to edit the data pipeline.
4. In the Access key ID field, enter your AWS account access key id.
5. In the Secret access key field, enter your AWS account secret access key. All the details about the data pipeline are populated, except the Access key ID and Secret access key.
   

   Note

       Provide valid AWS credentials for the **Access key ID** and **Secret access key**. Invalid AWS credentials do not allow you to edit a data pipeline.  
   

   For more information about editing a data pipeline, see Creating a data pipeline.

Edit a data pipeline

After creating a data pipeline, you can edit the details to suit your requirements. To edit a data pipeline, do the following:

1. In the Data Pipelines page, mouse over the Pipeline name column and select the +Edit pipeline icon for a data pipeline.
2. In the Access key ID field, enter your AWS account access key id.
3. In the Secret access key field, enter your AWS account secret access key. All the details about the data pipeline are populated, except the Access key ID and Secret access key.
   

   Note

        Provide valid AWS credentials for the **Access key ID** and **Secret access key**. Invalid AWS credentials do not allow you to edit a data pipeline.  
   

   For more information about editing a data pipeline, see Creating a data pipeline.

Clone a data pipeline

You can create a clone or replica of an existing data pipeline. The configurations of the existing data pipeline are copied to the cloned data pipeline. You can edit these configuration settings in the cloned data pipeline.

1. In the Data Pipelines page, mouse over the Pipeline name column and select the Clone pipeline icon for a data pipeline.
2. In the Confirmation dialog, select Confirm. The Create a pipeline page displays.
3. In the Pipeline name field, enter the name of the pipeline.
4. In the Access key ID field, enter your AWS account access key id.
5. In the Secret access key field, enter your AWS account secret access key.

   Note

        Provide valid AWS credentials for the **Access key ID** and **Secret access key**. Invalid AWS credentials do not allow you to edit a data pipeline.  
   

   Information in all other fields is pre-populated. You can edit this information.

Remove a data pipeline

You can delete a data pipeline that is no longer in use or required.

1. In the Data Pipelines page, mouse over the Pipeline name column and select the +Remove pipeline icon for a data pipeline.
2. In the Confirmation dialog, select Remove.

3 - Overview of Integrations

OTCAD integrates with third party applications. This topic provides information about OTCAD integration with third party applications.

3.1 - Integrations

OTCAD is a cloud solution that enhances the speed and scalability of an "OpenText™ Analytics Database without the need to manage hardware. Use OTCAD to build databases securely and scale them to large amounts of data in the cloud.

Integrations

When you log in to the application, the More options button is displayed.

1. Expand More options.
2. Select Integrations. The Integrations page displays. OTCAD supports machine learning with VerticaPy for data science and AI-driven data science applications. It also supports the following third-party applications:
   • IBM Cognos BI See Vertica Integration with IBM Cognos Analytics: Connection Guide
   • Grafana See Vertica Integration with Grafana
   • Informatica PowerCenter See Vertica QuickStart for Informatica PowerCenter
   • DbVisualizer See Vertica Integration with DbVisualizer: Connection Guide
   • DBeaver See Vertica Integration with DBeaver: Connection Guide
   • Domino Data Lab See End-to-End Machine Learning Solution with Vertica and Domino Data Lab Using VerticaPy
3. For each third-party application, select View Docs to view the Connection guide.