com.vertica.jdbc

Class VerticaCopyStream

java.lang.Object

com.vertica.jdbc.VerticaCopyStream

public class VerticaCopyStream
extends java.lang.Object

A bulk loading utility that sends COPY FROM LOCAL STDIN data from a JDBC client to the Vertica database. To bulk load data, provide any InputStream(s) and a COPY FROM LOCAL STDIN statement that describes the stream's format and a destination table.

The start() method begins a copy statement. If any streams have already been provided, they are sent at this point. The execute() method can optionally continue the execution. It sends any streams that have been added via addStream(InputStream) since the last start() or execute() operation, Finally, to end the copy statement, and get the total number of rows added during the entire copy operation, call the finish() method.

Though the start(), execute() and finish() methods are designed to work on a single copy statement open over multiple method calls, there are cases when the driver must split what is logically a single operation into multiple distinct commands.

This utility honors the connection's autocommit setting, meaning that upon returning from the start(), execute() or finish() methods, the driver will have committed the transaction when autocommit is enabled. It is not illegal to add data streams to a VerticaCopyStream across transactions, but this will result in degraded performance, as separate ROS containers will be created on the server for each transaction's copy operation. For best performance, disable autocommit when using this utility.

Similarly, because the driver can only allow one statement to run on a connection at a time, interleaving other statement executions between calls to start(), execute() or finish() will force the open copy statement to end, so that the other statement may execute. In this case, a new copy operation will begin upon the next call to execute() or finish(). This will create multiple ROS containers and degrade performance. Furthermore, if the copy statement does not have the NO COMMIT option, then each interleaved statement will in effect end the transaction, even when autocommit is turned off.

To track the progress of the load, the getRejects() method returns a list of' row numbers that the server rejected. Each call to start(), execute() or finish() will send over any stream(s) added via addStream(InputStream) that have not yet been sent to the server. The rejections from the streams(s) sent during the last start(), execute() or finish() operation will be available via getRejects(). This list is cleared with each new call to start(), execute() or finish().

This class also implements Exceptions and Rejections files where, if specified in the Copy statement, the rejected rows will be written to the file designated as REJECTED DATA and the reason for rejection will be written to the designated EXCEPTIONS file. If these are specified, the driver will return an empty list with any calls to getRejects(). The rejection files will contain more detailed rejection information and can be checked instead.

Constructor Summary

Constructors

Constructor and Description
VerticaCopyStream(VerticaConnection conn, java.lang.String copyStatement) Creates a new VerticaCopyStream to execute the specified statement.
VerticaCopyStream(VerticaConnection conn, java.lang.String copyStatement, java.io.InputStream source) Creates a new VerticaCopyStream to execute the specified statement.
VerticaCopyStream(VerticaConnection conn, java.lang.String copyStatement, java.util.List<java.io.InputStream> sources) Creates a new VerticaCopyStream to execute the specified statement.

Method Summary

Modifier and Type	Method and Description
void	addStream(java.io.InputStream input) Adds a stream to the list of streams to send in the copy statement.
void	execute()
long	finish() Finishes the copy statement.
java.util.List<java.lang.Long>	getRejects() Returns a list of row numbers corresponding to the rows in the input streams that were rejected by the server.
long	getRowCount() Gets the final row count for the entire copy operation.
void	start() Starts the copy operation, optionally sending over any InputStream data that has been added to this VerticaCopyStream thus far (via execute()).

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VerticaCopyStream

public VerticaCopyStream(VerticaConnection conn,
                         java.lang.String copyStatement)
                  throws java.sql.SQLException

Creates a new VerticaCopyStream to execute the specified statement.

Parameters:

conn - The connection to run the copy statement on.

copyStatement - A COPY FROM STDIN statement to execute.

Throws:

java.sql.SQLException - If creation of new copy stream fails.

VerticaCopyStream

public VerticaCopyStream(VerticaConnection conn,
                         java.lang.String copyStatement,
                         java.io.InputStream source)
                  throws java.sql.SQLException

Creates a new VerticaCopyStream to execute the specified statement.

Parameters:

conn - The connection to run the copy statement on.

copyStatement - A COPY FROM STDIN statement to execute.

source - A data stream in the format expected by the copy statement.

Throws:

java.sql.SQLException - If creation of new copy stream fails.

VerticaCopyStream

public VerticaCopyStream(VerticaConnection conn,
                         java.lang.String copyStatement,
                         java.util.List<java.io.InputStream> sources)
                  throws java.sql.SQLException

Creates a new VerticaCopyStream to execute the specified statement.

Parameters:

conn - The connection to run the copy statement on.

copyStatement - A COPY FROM STDIN statement to execute.

sources - A list of data streams, each in the format expected by the copy statement. The streams will be copied sequentially into the database.

Throws:

java.sql.SQLException - If creation of new copy stream fails.

Method Detail

start

public void start()
           throws java.sql.SQLException

Starts the copy operation, optionally sending over any InputStream data that has been added to this VerticaCopyStream thus far (via execute()). The statement is left open so that more data streams can be added.

Throws:

java.sql.SQLException - If start operation fails.

execute

public void execute()
             throws java.sql.SQLException

Throws:

java.sql.SQLException - If this COPY operation has not yet been started, or a database error occurs.

addStream

public void addStream(java.io.InputStream input)
               throws java.sql.SQLException

Adds a stream to the list of streams to send in the copy statement. The data will be sent to the database on the next call to start(), execute() or finish().

Parameters:

input - The stream to add.

Throws:

java.sql.SQLException - If addStream operation fails.

finish

public long finish()
            throws java.sql.SQLException

Finishes the copy statement. If streams have been added but not yet executed, they will be executed.

Returns:

The total number of rows that were added to the table during this COPY operation.

Throws:

java.sql.SQLException - If finish operation fails.

getRejects

public java.util.List<java.lang.Long> getRejects()

Returns a list of row numbers corresponding to the rows in the input streams that were rejected by the server. This should be checked after each call to start(), execute() or finish() to see which rows were rejected. When a new execution starts, the previous list of rejected rows is discarded. This list will be empty if EXCEPTIONS or REJECTED DATA files are specified

Returns:

A list of row numbers that were rejected by the server during the last execution.

getRowCount

public long getRowCount()

Gets the final row count for the entire copy operation. This will be remain zero until finish() has been called.

Returns:

The total number of rows successfully copied to the database.