Package org.apache.spark.sql.connector.read.partitioning

Class KeyGroupedPartitioning

Object

org.apache.spark.sql.connector.read.partitioning.KeyGroupedPartitioning

All Implemented Interfaces:

Partitioning

@Evolving
public class KeyGroupedPartitioning
extends Object
implements Partitioning

Represents a partitioning where rows are split across partitions based on the partition transform expressions returned by keys.

Note: Data source implementations should make sure for a single partition, all of its rows must be evaluated to the same partition value after being applied by keys expressions. Different partitions can share the same partition value: Spark will group these into a single logical partition during planning phase.

Since:

3.3.0

Constructor Summary

Constructors

Constructor	Description
KeyGroupedPartitioning(Expression[] keys, int numPartitions)

Method Summary

Modifier and Type	Method	Description
Expression[]	keys()	Returns the partition transform expressions for this partitioning.
int	numPartitions()	Returns the number of partitions that the data is split across.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

KeyGroupedPartitioning

public KeyGroupedPartitioning(Expression[] keys,
 int numPartitions)

Method Details

keys

public Expression[] keys()

Returns the partition transform expressions for this partitioning.

Spark resolves the column references in these expressions against the table columns, including columns pruned from the scan output. If any of them cannot be resolved, Spark ignores the whole reported partitioning and logs a warning.

numPartitions

public int numPartitions()

Description copied from interface: Partitioning

Returns the number of partitions that the data is split across.

Specified by:

numPartitions in interface Partitioning