OpenText Analytics Database 26.2.x – Machine learning for predictive analytics

Data-Analysis: Download the machine learning example data

Mon, 01 Jan 0001 00:00:00 +0000

You need several data sets to run the machine learning examples. You can download these data sets from the Vertica GitHub repository.

Important

The GitHub examples are based on the latest OpenText™ Analytics Database version. If you note differences, please upgrade to the latest version.

You can download the example data in either of two ways:

Download the ZIP file. Extract the contents of the file into a directory.
Clone the Machine Learning GitHub repository. Using a terminal window, run the following command:
```
$ git clone https://github.com/vertica/Machine-Learning-Examples
```

Loading the example data

You can load the example data by doing one of the following. Note that models are not automatically dropped. You must either rerun the load_ml_data.sql script to drop models or manually drop them.

Copying and pasting the DDL and DML operations in load_ml_data.sql in a vsql prompt or another database client.
Running the following command from a terminal window within the data folder in the Machine-Learning-Examples directory:
```
$ /opt/vertica/bin/vsql -d <name of your database> -f load_ml_data.sql
```

You must also load the naive_bayes_data_prepration.sql script in the Machine-Learning-Examples directory:

$ /opt/vertica/bin/vsql -d <name of your database> -f ./naive_bayes/naive_bayes_data_preparation.sql

Example data descriptions

The repository contains the following data sets.

Name	Description
agar_dish	Synthetic data set meant to represent clustering of bacteria on an agar dish. Contains the following columns: id, x-coordinate, and y-coordinate.
agar_dish_2	125 rows sampled randomly from the original 500 rows of the agar_dish data set.
agar_dish_1	375 rows sampled randomly from the original 500 rows of the agar_dish data set.
baseball	Contains statistics from a fictional baseball league. The statistics included are: first name, last name, date of birth, team name, homeruns, hits, batting average, and salary.
daily-min-temperatures	Contains data on the daily minimum temperature in Melbourne, Australia from 1981 through 1990.
dem_votes	Contains data on the number of yes and no votes by Democrat members of U.S. Congress for each of the 16 votes in the house84 data set. The table must be populated by running the `naive_bayes_data_prepration.sql` script. Contains the following columns: vote, yes, no.
faithful	Wait times between eruptions and the duration of the eruption for the Old Faithful geyser in Yellowstone National Park, Wyoming, USA. Reference Härdle, W. (1991) Smoothing Techniques with Implementation in S. New York: Springer. Azzalini, A. and Bowman, A. W. (1990). A look at some data on the Old Faithful geyser. Applied Statistics 39, 357–365.
faithful_testing	Roughly 60% of the original 272 rows of the faithful data set.
faithful_training	Roughly 40% of the original 272 rows of the faithful data set.
house84	The house84 data set includes votes for each of the U.S. House of Representatives Congress members on 16 votes. Contains the following columns: id, party, vote1, vote2, vote3, vote4, vote5, vote6, vote7, vote8, vote9, vote10, vote11, vote12, vote13, vote14, vote15, vote16. Reference Congressional Quarterly Almanac, 98th Congress, 2nd session 1984, Volume XL: Congressional Quarterly Inc. Washington, D.C., 1985.
iris	The iris data set gives the measurements in centimeters of the variables sepal length and width and petal length and width, respectively, for 50 flowers from each of 3 species of iris. The species are Iris setosa, versicolor, and virginica. Reference Becker, R. A., Chambers, J. M. and Wilks, A. R. (1988) The New S Language. Wadsworth & Brooks/Cole.
iris1	90 rows sampled randomly from the original 150 rows in the iris data set.
iris2	60 rows sampled randomly from the original 150 rows in the iris data set.
mtcars	The data was extracted from the 1974 Motor Trend US magazine, and comprises fuel consumption and 10 aspects of automobile design and performance for 32 automobiles (1973–74 models). Reference Henderson and Velleman (1981), Building multiple regression models interactively. Biometrics, 37, 391–411.
rep_votes	Contains data on the number of yes and no votes by Republican members of U.S. Congress for each of the 16 votes in the house84 data set. The table must be populated by running the `naive_bayes_data_prepration.sql` script. Contains the following columns: vote, yes, no.
salary_data	Contains fictional employee data. The data included are: employee id, first name, last name, years worked, and current salary.
transaction_data	Contains fictional credit card transactions with a BOOLEAN column indicating whether there was fraud associated with the transaction. The data included are: first name, last name, store, cost, and fraud.
titanic_testing	Contains passenger information from the Titanic ship including sex, age, passenger class, and whether or not they survived.
titanic_training	Contains passenger information from the Titanic ship including sex, age, passenger class, and whether or not they survived.
world	Contains country-specific information about human development using HDI, GDP, and CO2 emissions.

Data-Analysis: Data preparation

Mon, 01 Jan 0001 00:00:00 +0000

Before you can analyze your data, you must prepare it. You can do the following data preparation tasks in OpenText™ Analytics Database:

Data-Analysis: Regression algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Regression is an important and popular machine learning tool that makes predictions from data by learning the relationship between some features of the data and an observed value response. Regression is used to make predictions about profits, sales, temperature, stocks, and more. For example, you could use regression to predict the price of a house based on the location, the square footage, the size of the lot, and so on. In this example, the house's value is the response, and the other factors, such as location, are the features.

The optimal set of coefficients found for the regression's equation is known as the model. The relationship between the outcome and the features is summarized in the model, which can then be applied to different data sets, where the outcome value is unknown.

Data-Analysis: Classification algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Classification is an important and popular machine learning tool that assigns items in a data set to different categories. Classification is used to predict risk over time, in fraud detection, text categorization, and more. Classification functions begin with a data set where the different categories are known. For example, suppose you want to classify students based on how likely they are to get into graduate school. In addition to factors like admission score exams and grades, you could also track work experience.

Binary classification means the outcome, in this case, admission, only has two possible values: admit or do not admit. Multiclass outcomes have more than two values. For example, low, medium, or high chance of admission. During the training process, classification algorithms find the relationship between the outcome and the features. This relationship is summarized in the model, which can then be applied to different data sets, where the categories are unknown.

Data-Analysis: Clustering algorithms

Mon, 01 Jan 0001 00:00:00 +0000

Clustering is an important and popular machine learning tool used to find clusters of items in a data set that are similar to one another. The goal of clustering is to create clusters with a high number of objects that are similar. Similar to classification, clustering segments the data. However, in clustering, the categorical groups are not defined.

Clustering data into related groupings has many useful applications. If you already know how many clusters your data contains, the K-means algorithm may be sufficient to train your model and use that model to predict cluster membership for new data points.

However, in the more common case, you do not know before analyzing the data how many clusters it contains. In these cases, the Bisecting k-means algorithm is much more effective at finding the correct clusters in your data.

Both k-means and bisecting k-means predict the clusters for a given data set. A model trained using either algorithm can then be used to predict the cluster to which new data points are assigned.

Clustering can be used to find anomalies in data and find natural groups of data. For example, you can use clustering to analyze a geographical region and determine which areas of that region are most likely to be hit by an earthquake. For a complete example, see Earthquake Cluster Analysis Using the KMeans Approach.

In OpenText™ Analytics Database, clustering is computed based on Euclidean distance. Through this computation, data points are assigned to the cluster with the nearest center.

Data-Analysis: Time series forecasting

Mon, 01 Jan 0001 00:00:00 +0000

Time series models are trained on stationary time series (that is, time series where the mean doesn't change over time) of stochastic processes with consistent time steps. These algorithms forecast future values by taking into account the influence of values at some number of preceding timesteps (lags).

Examples of applicable datasets include those for temperature, stock prices, earthquakes, product sales, etc.

To normalize datasets with inconsistent timesteps, see Gap filling and interpolation (GFI).

Data-Analysis: Model management

Mon, 01 Jan 0001 00:00:00 +0000

OpenText™ Analytics Database provides a number of tools to manage existing models. You can view model summaries and attributes, alter model characteristics like name and privileges, drop models, and version models.

Data-Analysis: Using external models

Mon, 01 Jan 0001 00:00:00 +0000

To give you the utmost in machine learning flexibility and scalability, OpenText™ Analytics Database supports importing, exporting, and predicting with PMML and TensorFlow models.

The machine learning configuration parameter MaxModelSizeKB sets the maximum size of a model that can be imported into the database.

Support for PMML models

OpenText™ Analytics Database supports the import and export of machine learning models in Predictive Model Markup Language (PMML) format. Support for this platform-independent model format allows you to use models trained on other platforms to predict on data stored in your database. You can also use the database as your model repository. OpenText™ Analytics Database supports PMML version 4.4.1.

With the PREDICT_PMML function, you can use an archived PMML model to run prediction on data stored in the database. For more information, see Using PMML models.

For details on the PMML models, tags, and attributes that OpenText™ Analytics Database supports, see PMML features and attributes.

Support for TensorFlow models

OpenText™ Analytics Database now supports importing trained TensorFlow models, and using those models to do prediction on data stored in the database. OpenText™ Analytics Database supports TensorFlow models trained in TensorFlow version 1.15.

The PREDICT_TENSORFLOW and PREDICT_TENSORFLOW_SCALAR functions let you predict on data with TensorFlow models.

For additional information, see TensorFlow models.

Additional external model support

The following functions support both PMML and TensorFlow models: