pyspark.RDD.collectWithJobGroup#

RDD.collectWithJobGroup(groupId, description, interruptOnCancel=False)[source]#

When collect rdd, use this method to specify job group.

Added in version 3.0.0.

Deprecated since version 3.1.0: Use pyspark.InheritableThread with the pinned thread mode enabled.

Parameters:

groupIdstr: The group ID to assign.
descriptionstr: The description to set for the job group.
interruptOnCancelbool, optional, default False: whether to interrupt jobs on job cancellation.

Returns:

list: a list containing all the elements

See also

RDD.collect()
SparkContext.setJobGroup()