C++ SDK Documentation
24.2.0
|
Base class for UDSource. Use, with IterativeSourceFactory, if computing the size of a source up front would be prohibitively expensive, or if the number of distinct sources would be prohibitively large to use the standard API. Typically you should use UDSource instead. More...
Public Member Functions | |
void | cancelUDX (ServerInterface &srvInterface) |
virtual void | destroy (ServerInterface &srvInterface) |
virtual void | destroy (ServerInterface &srvInterface, SessionParamWriterMap &udSessionParams) |
virtual Portion | getPortion ()=0 |
virtual std::string | getUri () |
bool | isCanceled () const |
virtual StreamState | process (ServerInterface &srvInterface, DataBuffer &output)=0 |
virtual StreamState | processWithMetadata (ServerInterface &srvInterface, DataBuffer &output, LengthBuffer &output_lengths) |
virtual size_t | requestMinBufferSize (ServerInterface &srvInterface) |
virtual void | setup (ServerInterface &srvInterface) |
virtual bool | useSideChannel () |
Static Public Attributes | |
static const size_t | DEFAULT_MIN_BUFFER_SIZE = 1024 * 1024 |
Protected Member Functions | |
virtual void | cancel (ServerInterface &srvInterface) |
Base class for UDSource. Use, with IterativeSourceFactory, if computing the size of a source up front would be prohibitively expensive, or if the number of distinct sources would be prohibitively large to use the standard API. Typically you should use UDSource instead.
Not intended or optimized for typical applications.
Note that it is UNSAFE to maintain pointers or references to any of these arguments (or any other argument passed by reference into any other function in this API) beyond the scope of the function call in question. For example, do not store a reference to the server interface or the input block on an instance variable. Vertica may free and replace these objects.
|
inlineprotectedvirtualinherited |
Cancel callback to be overridden by the UDX. Called when the query running the UDX has been canceled.
Referenced by Vertica::UDXObject::cancelUDX().
|
inlineinherited |
Cancel callback invoked when the query running the UDX has been canceled. See cancel().
|
pure virtual |
Return the portion this source object is responsible for (if any).
This can be used by other elements of the load stack to determine how much data to process. It also will uniquely identify a source in the load_sources view. As such, this will be called BEFORE setup() (for load_sources) and again after setup() (for data processing).
Implemented in Vertica::UDSource.
|
inlinevirtual |
Return the URI of the current source of data.
This function will be invoked during execution to fill in monitoring information.
|
inlineinherited |
|
inlinevirtual |
UnsizedUDSource::processWithMetadata()
Reads data from the input source and record length metadata from the side channel and processes it. To implement processWithMetadata(), you must override useSideChannel() to return true
. Vertica invokes this method repeatedly until it returns DONE or the query is canceled by the user.
Input: an external data source.
Output: a stream of data bytes, and a stream of bytes containing message length metadata from the data source.
On each invocation, processWithMetadata() should acquire more data and write that data to the buffer specified by output
, and write the message lengths to output_lengths
.
For the DataBuffer, processWithMetadata() should set output.offset
(an output parameter) to the number of bytes that were written to the output
buffer. It is common, though not necessary, for this to be the same as output.size
(an input parameter). When processWithMetadata() is called, output.offset
is uninitialized. To indicate that the buffer is too small to hold a record, processWithMetadata() should set output.offset
and output_length.offset
to 0 and return OUTPUT_NEEDED. Then processWithMetadata() is called again with a larger buffer.
For the LengthBuffer, processWithMetadata() should set output_lengths.offset
to the number of length values that were written to the output_lengths
buffer. If output.offset
is set to 0, then output_lengths.offset
should also be set to 0.
In general, processWithMetadata() code should assume that data buffers start at output.buf[output.offset]
and length buffers start at output_lengths.buf[output_lengths.offset]
.
As a performance optimization, upstream operators might start processing emitted data (data between output.buf[0] and output.buf[output.offset] in the DataBuffer, and between output_lengths.buf[0] and output_lengths.buf[output.offset] in the LengthBuffer) before OUTPUT_NEEDED is returned. For this reason, output.offset
and output_lengths.offset
must be strictly increasing.
|
inlinevirtual |
Request a minimum buffer size into which this source will read data. This will be called after setup(), before the first call to process().
|
inlinevirtual |
UnsizedUDSource::useSideChannel()
Provides access to the side channel containing record length metadata, when the UnsizedUDSource has metadata about record boundaries available in a structured format that is separate from the data payload.
Override and return true
to indicate that processWithMetadata() should be called instead of process().
Return false
to implement process().
false
by default.