Package org.apache.spark.sql.connector.read

Interface SupportsPushDownJoin

All Superinterfaces:

ScanBuilder

@Evolving
public interface SupportsPushDownJoin
extends ScanBuilder

A mix-in interface for ScanBuilder. Data sources can implement this interface to push down join operators.

Since:

4.1.0

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static final record	SupportsPushDownJoin.ColumnWithAlias	A helper class used when there are duplicated names coming from 2 sides of the join operator.

Method Summary

Modifier and Type	Method	Description
boolean	isOtherSideCompatibleForJoin(SupportsPushDownJoin other)	Returns true if the other side of the join is compatible with the current SupportsPushDownJoin for a join push down, meaning both sides can be processed together within the same underlying data source.
boolean	pushDownJoin(SupportsPushDownJoin other, JoinType joinType, SupportsPushDownJoin.ColumnWithAlias[] leftSideRequiredColumnsWithAliases, SupportsPushDownJoin.ColumnWithAlias[] rightSideRequiredColumnsWithAliases, Predicate condition)	Pushes down the join of the current SupportsPushDownJoin and the other side of join SupportsPushDownJoin.

Methods inherited from interface org.apache.spark.sql.connector.read.ScanBuilder

build

Method Details

isOtherSideCompatibleForJoin

boolean isOtherSideCompatibleForJoin(SupportsPushDownJoin other)

Returns true if the other side of the join is compatible with the current SupportsPushDownJoin for a join push down, meaning both sides can be processed together within the same underlying data source.

For example, JDBC connectors are compatible if they use the same host, port, username, and password.

pushDownJoin

boolean pushDownJoin(SupportsPushDownJoin other,
 JoinType joinType,
 SupportsPushDownJoin.ColumnWithAlias[] leftSideRequiredColumnsWithAliases,
 SupportsPushDownJoin.ColumnWithAlias[] rightSideRequiredColumnsWithAliases,
 Predicate condition)

Pushes down the join of the current SupportsPushDownJoin and the other side of join SupportsPushDownJoin.

Parameters:

other - SupportsPushDownJoin that this SupportsPushDownJoin gets joined with.

joinType - the type of join.

leftSideRequiredColumnsWithAliases - required output of the left side SupportsPushDownJoin

rightSideRequiredColumnsWithAliases - required output of the right side SupportsPushDownJoin

condition - join condition. Columns are named after the specified aliases in leftSideRequiredColumnWithAliases and rightSideRequiredColumnWithAliases

Returns:

True if join has been successfully pushed down.