UDFS_STATISTICS
Records aggregate information about operations on file systems and object-stores.
Records aggregate information about operations on file systems and object stores.
An operation can be made up of many individual read, write, or retry requests. SUCCESSFUL_OPERATIONS and FAILED_OPERATIONS count operations; the other counters count individual requests. When an operation finishes, one of the OPERATIONS counters is incremented once, but several other counters could be incremented several times each.
| Column Name | Data Type | Description |
|---|---|---|
| FILESYSTEM | VARCHAR | Name of the file system, such as S3 or WebHDFS. |
| SUCCESSFUL_OPERATIONS | INTEGER | Number of successful operations. |
| FAILED_OPERATIONS | INTEGER | Number of failed operations. |
| RETRIES | INTEGER | Number of retry events. |
| METADATA_READS | INTEGER | Number of requests to read metadata. For example, S3 list bucket and HEAD requests are metadata reads. |
| METADATA_WRITES | INTEGER | Number of requests to write metadata. For example, S3 POST and DELETE requests are metadata writes. |
| DATA_READS | INTEGER | Number of read operations, such as S3 GET requests to download files. |
| DATA_WRITES | INTEGER | Number of write operations, such as S3 PUT requests to upload files. |
| DOWNSTREAM_BYTES | INTEGER | Number of bytes received. |
| UPSTREAM_BYTES | INTEGER | Number of bytes sent. |
| OPEN_FILES | INTEGER | Number of files that are currently open. |
| MAPPED_FILES | INTEGER | Number of files that are currently mapped. On S3 file systems, this is the number of streaming connections for reading data. On other file systems, this value is 0. |
| READING | INTEGER | The number of read operations that are currently running. |
| WRITING | INTEGER | The number of write operations that are currently running. |
| TOTAL_REQUEST_DURATION_MS | INTEGER | Accumulated HTTP request duration, in milliseconds. This information is recorded only for S3 and GCS. |
| OVERALL_AVERAGE_LATENCY_MS | INTEGER | Average HTTP request duration, in milliseconds. This value is TOTAL_REQUEST_DURATION_MS divided by the sum of SUCCESSFUL_OPERATIONS and FAILED_OPERATIONS (rounded). This value is set only for S3 and GCS. |
| OVERALL_DOWNSTREAM_THROUGHPUT_MB_S | FLOAT | HTTP request downstream throughput in megabytes per second (MB/s). This value is DOWNSTREAM_BYTES divided by TOTAL_REQUEST_DURATION_MS. This value is set only for S3 and GCS. |
| OVERALL_UPSTREAM_THROUGHPUT_MB_S | FLOAT | HTTP request upstream throughput in megabytes per second (MB/s). This value is UPSTREAM_BYTES divided by TOTAL_REQUEST_DURATION_MS. This value is set only for S3 and GCS. |
Examples
The following query gets the total number of metadata RPCs for HDFS operations:
=> SELECT SUM(metadata_reads) FROM UDFS_STATISTICS WHERE filesystem = 'HDFS';