org.apache.iceberg

Interface DataFile

All Superinterfaces:

ContentFile<DataFile>

All Known Implementing Classes:

SparkDataFile

public interface DataFile
extends ContentFile<DataFile>

Interface for data files listed in a table manifest.

Field Summary

Fields

Modifier and Type	Field and Description
static Types.NestedField	COLUMN_SIZES
static Types.NestedField	CONTENT
static Types.NestedField	EQUALITY_IDS
static Types.NestedField	FILE_FORMAT
static Types.NestedField	FILE_PATH
static Types.NestedField	FILE_SIZE
static Types.NestedField	KEY_METADATA
static Types.NestedField	LOWER_BOUNDS
static Types.NestedField	NAN_VALUE_COUNTS
static Types.NestedField	NULL_VALUE_COUNTS
static java.lang.String	PARTITION_DOC
static int	PARTITION_ID
static java.lang.String	PARTITION_NAME
static Types.NestedField	RECORD_COUNT
static Types.NestedField	SORT_ORDER_ID
static Types.NestedField	SPEC_ID
static Types.NestedField	SPLIT_OFFSETS
static Types.NestedField	UPPER_BOUNDS
static Types.NestedField	VALUE_COUNTS

Method Summary

Modifier and Type	Method and Description
default FileContent	content() Returns type of content stored in the file; one of DATA, POSITION_DELETES, or EQUALITY_DELETES.
default java.util.List<java.lang.Integer>	equalityFieldIds() Returns the set of field IDs used for equality comparison, in equality delete files.
static Types.StructType	getType(Types.StructType partitionType)

Methods inherited from interface org.apache.iceberg.ContentFile

columnSizes, copy, copy, copyWithoutStats, fileSizeInBytes, format, keyMetadata, lowerBounds, nanValueCounts, nullValueCounts, partition, path, pos, recordCount, sortOrderId, specId, splitOffsets, upperBounds, valueCounts

Field Detail

CONTENT

static final Types.NestedField CONTENT

FILE_PATH

static final Types.NestedField FILE_PATH

FILE_FORMAT

static final Types.NestedField FILE_FORMAT

RECORD_COUNT

static final Types.NestedField RECORD_COUNT

FILE_SIZE

static final Types.NestedField FILE_SIZE

COLUMN_SIZES

static final Types.NestedField COLUMN_SIZES

VALUE_COUNTS

static final Types.NestedField VALUE_COUNTS

NULL_VALUE_COUNTS

static final Types.NestedField NULL_VALUE_COUNTS

NAN_VALUE_COUNTS

static final Types.NestedField NAN_VALUE_COUNTS

LOWER_BOUNDS

static final Types.NestedField LOWER_BOUNDS

UPPER_BOUNDS

static final Types.NestedField UPPER_BOUNDS

KEY_METADATA

static final Types.NestedField KEY_METADATA

SPLIT_OFFSETS

static final Types.NestedField SPLIT_OFFSETS

EQUALITY_IDS

static final Types.NestedField EQUALITY_IDS

SORT_ORDER_ID

static final Types.NestedField SORT_ORDER_ID

SPEC_ID

static final Types.NestedField SPEC_ID

PARTITION_ID

static final int PARTITION_ID

See Also:

Constant Field Values

PARTITION_NAME

static final java.lang.String PARTITION_NAME

See Also:

Constant Field Values

PARTITION_DOC

static final java.lang.String PARTITION_DOC

See Also:

Constant Field Values

Method Detail

getType

static Types.StructType getType(Types.StructType partitionType)

content

default FileContent content()

Description copied from interface: ContentFile

Returns type of content stored in the file; one of DATA, POSITION_DELETES, or EQUALITY_DELETES.

Specified by:

content in interface ContentFile<DataFile>

Returns:

the content stored in the file; one of DATA, POSITION_DELETES, or EQUALITY_DELETES

equalityFieldIds

default java.util.List<java.lang.Integer> equalityFieldIds()

Description copied from interface: ContentFile

Returns the set of field IDs used for equality comparison, in equality delete files.

An equality delete file may contain additional data fields that are not used by equality comparison. The subset of columns in a delete file to be used in equality comparison are tracked by ID. Extra columns can be used to reconstruct changes and metrics from extra columns are used during job planning.

Specified by:

equalityFieldIds in interface ContentFile<DataFile>

Returns:

IDs of the fields used in equality comparison with the records in this delete file