Package org.apache.spark.sql.connector.read

Interface SupportsReportPartitioning

All Superinterfaces:

Scan

@Evolving
public interface SupportsReportPartitioning
extends Scan

A mix in interface for Scan. Data sources can implement this interface to report data partitioning and try to avoid shuffle at Spark side.

Note that, when a Scan implementation creates exactly one InputPartition, Spark may avoid adding a shuffle even if the reader does not implement this interface.

Since:

3.0.0

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.sql.connector.read.Scan

Scan.ColumnarSupportMode

Method Summary

Modifier and Type	Method	Description
Partitioning	outputPartitioning()	Returns the output data partitioning that this reader guarantees.

Methods inherited from interface org.apache.spark.sql.connector.read.Scan

columnarSupportMode, description, readSchema, reportDriverMetrics, supportedCustomMetrics, toBatch, toContinuousStream, toMicroBatchStream

Method Details

outputPartitioning

Partitioning outputPartitioning()

Returns the output data partitioning that this reader guarantees.