Java SDK Documentation  23.3.0
Public Member Functions | Protected Member Functions | List of all members
com.vertica.sdk.UDSource Class Referenceabstract
Inheritance diagram for com.vertica.sdk.UDSource:
Inheritance graph
Collaboration diagram for com.vertica.sdk.UDSource:
Collaboration graph

Public Member Functions

final void cancelUDX (ServerInterface srvInterface)
 
void destroy (ServerInterface srvInterface) throws UdfException
 
void destroy (ServerInterface srvInterface, SessionParamWriterMap udSessionParams) throws UdfException
 
Portion getPortion ()
 
Integer getSize ()
 
String getUri ()
 
boolean isCanceled ()
 
abstract StreamState process (ServerInterface srvInterface, DataBuffer output) throws UdfException
 
void setup (ServerInterface srvInterface) throws UdfException
 

Protected Member Functions

void cancel (ServerInterface srvInterface)
 

Detailed Description

Acquires data from an external source (such as a file or URL) and produces that data in a streaming manner.

The output of a UDSource can be sent through one or more UDFilters followed by one UDParser.

Member Function Documentation

void com.vertica.sdk.UDXObject.cancel ( ServerInterface  srvInterface)
protectedinherited

Cancel callback to be overridden by the UDX implementation. Called when the query running the UDX has been canceled.

  • This method will be invoked at most once per UDX object. Once a UDX object has been canceled, it will never be un-canceled.
  • This method may be called from a separate thread, concurrently with other methods of this UDX object (but never the constructor). Implementations must be thread-safe with all methods of this UDX.
  • This method will be invoked for either an explicit user cancel, or in the event of an error during query execution.
Parameters
srvInterfacea ServerInterface object used to communicate with Vertica

Referenced by com.vertica.sdk.UDXObject.cancelUDX().

final void com.vertica.sdk.UDXObject.cancelUDX ( ServerInterface  srvInterface)
inherited

Cancel callback invoked when the query running the UDX has been canceled.

See cancel().

Parameters
srvInterfacea ServerInterface object used to communicate with Vertica
void com.vertica.sdk.UDSource.destroy ( ServerInterface  srvInterface) throws UdfException

UDSource::destroy()

Will be invoked during query execution, after the last time that process() is called on this UDSource instance.

May write UD Session Parameters for namespaces public and library

May optionally be overridden to perform tear-down/destruction.

Parameters
srvInterfacea ServerInterface object used to communicate with Vertica
Exceptions
UdfExceptionUDF problem
Portion com.vertica.sdk.UDSource.getPortion ( )

UDSource::getPortion()

Returns
Portion information associated with this source, if available
Integer com.vertica.sdk.UDSource.getSize ( )

UDSource::getSize()

Returns the estimated number of bytes that process() will return.

This value is treated as advisory only. It is used to indicate the file size in the LOAD_STREAMS table.

IMPORTANT: getSize() can be called at any time, even before setup() is called! (Though not before or during the constructor.)

In the case of Sources whose factories can potentially produce many UDSource instances, getSize() should avoid acquiring resources that last for the life of the object. Doing otherwise can defeat Vertica's attempts to limit the maximum number of Sources that are consuming system resources at any given time. For example, if it opens a file handle and leaves that file handle open for use by process(), and if a large number of UDSources are loaded in a single statement, the query may exceed the operating system limit on file handles and crash, even though Vertica only operates on a small number of files at once. This doesn't apply to singleton Sources, Sources whose factory will only ever produce one UDSource instance.

Returns
the size
String com.vertica.sdk.UnsizedUDSource.getUri ( )
inherited

UnsizedUDSource::getUri()

This function will be invoked during execution to fill in monitoring information.

Returns
the URI of the current source of data.
boolean com.vertica.sdk.UDXObject.isCanceled ( )
inherited
Returns
true if execution was canceled.

Referenced by com.vertica.sdk.UDXObject.cancelUDX().

abstract StreamState com.vertica.sdk.UDSource.process ( ServerInterface  srvInterface,
DataBuffer  output 
) throws UdfException
pure virtual

UDSource::process()

Will be invoked repeatedly during query execution, until it returns DONE or until the query is canceled by the user.

On each invocation, process() should acquire more data and write that data to the buffer specified by output.

process() must set output.offset to the number of bytes that were written to the output buffer. It is common, though not necessary, for this to be the same as output.size. output.offset is initially uninitialized. If it is set to 0, this indicates that the output buffer is too small for process() to be able to write a unit of input to it.

Note that, unless OUTPUT_NEEDED is returned, output will be UNMODIFIED the next time process() is called. This means that pointers into the buffer will continue to be valid. It also means that output.offset may be set. So, in general, process() code should assume that buffers start at output.buf[output.offset]. Note also that, as a performance optimization, upstream operators may start processing emitted data (data between output.buf[0] and output.buf[output.offset]) before OUTPUT_NEEDED is returned. For this reason, output.offset must be strictly increasing.

process() must not block indefinitely. If it cannot proceed for an extended period of time, it should return KEEP_GOING. It will be called again shortly. Failure to do this will, among other things, prevent the query from being canceled by the user.

Note that it is UNSAFE to maintain pointers or references to any of these arguments (or any other argument passed by reference into any other function in this API) beyond the scope of the function call in question. For example, do not store a reference to the server interface or the input block on an instance variable. Vertica may free and replace these objects.

Parameters
srvInterfacea ServerInterface object used to communicate with Vertica
Returns
OUTPUT_NEEDED if this UDSource has more data to produce; DONE if has no more data to produce.
Exceptions
UdfExceptionUDF problem

Implements com.vertica.sdk.UnsizedUDSource.

void com.vertica.sdk.UDSource.setup ( ServerInterface  srvInterface) throws UdfException

UDSource::setup()

Will be invoked during query execution, prior to the first time that process() is called on this UDSource instance.

May optionally be overridden to perform setup/initialization.

Parameters
srvInterfacea ServerInterface object used to communicate with Vertica
Exceptions
UdfExceptionUDF problem