Google Cloud Storage (GCS) object store

File system using the Google Cloud Storage platform.

File system using the Google Cloud Storage platform.

URI format

gs://bucket/path

Authentication

To access data in Google Cloud Storage (GCS) you must first do the following tasks:

  • Create a default project, obtain a developer key, and enable S3 interoperability mode as described in the GCS documentation.

  • Set the GCSAuth configuration parameter as in the following example.

    => ALTER SESSION SET GCSAuth='id:secret';

Configuration parameters

The following database configuration parameters apply to the GCS file system. You can set parameters at different levels with the appropriate ALTER statement, such as ALTER SESSION...SET PARAMETER. Query the CONFIGURATION_PARAMETERS system table to determine what levels (node, session, user, database) are valid for a given parameter. For information about all parameters related to GCS, see Google Cloud Storage parameters.

For external tables using highly partitioned data in an object store, see the ObjectStoreGlobStrategy configuration parameter and Partitions on Object Stores.

GCSAuth
An ID and secret key to authenticate to GCS. For extra security, do not store credentials in the database; instead, use ALTER SESSION...SET PARAMETER to set this value for the current session only.
GCSEnableHttps
Boolean, whether to use the HTTPS protocol when connecting to GCS, can be set only at the database level with ALTER DATABASE...SET PARAMETER.

Default: 1 (enabled)

GCSEndpoint
The connection endpoint address.

Default: storage.googleapis.com

Examples

The following example loads data from GCS:

=> ALTER SESSION SET GCSAuth='my_id:my_secret_key';

=> COPY t FROM 'gs://DataLake/clicks.parquet' PARQUET;