Combining cooperative parse and apportioned load

You can enable both Cooperative parse and Apportioned load in the same parser, allowing OpenText™ Analytics Database to decide how to load data.

Deciding how to divide a load

OpenText™ Analytics Database uses apportioned load, where possible, at query-planning time. It decides whether to also use cooperative parse at execution time.

Apportioned load requires SourceFactory support. Given a suitable UDSource, at planning time the database calls the isParserApportionable() method on the ParserFactory. If this method returns true, the database apportions the load.

If isParserApportionable() returns false but isChunkerApportionable() returns true, then a chunker is available for cooperative parse and that chunker supports apportioned load. The database apportions the load.

If neither of these methods returns true, the database does not apportion the load.

At execution time, the database first checks whether the load is running in unfenced mode and proceeds only if it is. Cooperative parse is not supported in fenced mode.

If the load is not apportioned, and more than one thread is available, the database uses cooperative parse.

If the load is apportioned, and exactly one thread is available, the database uses cooperative parse if and only if the parser is not apportionable. In this case, the chunker is apportionable but the parser is not.

If the load is apportioned, and more than one thread is available, and the chunker is apportionable, the database uses cooperative parse.

If the database uses cooperative parse but prepareChunker() does not return a UDChunker instance, the database reports an error.

Executing apportioned, cooperative loads

If a load uses both apportioned load and cooperative parse, the database uses the SourceFactory to break the input into portions. It then assigns the portions to execution nodes. See How OpenText™ Analytics Database Apportions a Load.

On the execution node, the database calls the chunker's alignPortion() method to align the input with portion boundaries. (This step is skipped for the first portion, which by definition is already aligned at the beginning.) This step is necessary because a parser using apportioned load sometimes has to read beyond the end of the portion, so a chunker needs to find the end point.

After aligning the portion, the database calls the chunker's process() method repeatedly. See How OpenText™ Analytics Database Divides a Load.

The chunks found by the chunker are then sent to the parser's process() method for processing in the usual way.