Write partitioning properties#
Enable preferred write partitioning. When set to
true and more than the
minimum number of partitions, set in
are written, each partition is written by a separate writer. As a result, for some connectors such as the
Hive connector, only a single new file is written per partition, instead of
multiple files. Partition writer assignments are distributed across worker
nodes for parallel processing.
use-preferred-write-partitioning can be
specified on a per-query basis using the
Use the connector’s preferred write partitioning when the optimizer’s estimate
of the number of partitions that will be written by the query is greater than
the configured value. If the number of partitions cannot be estimated from the
statistics, then preferred write partitioning is not used.
If the threshold value is
1 then preferred write partitioning is always used.
preferred-write-partitioning-min-number-of-partitions can be specified on a
per-query basis using the