Delta Lake Feature Support Status in Apache Gluten (Velox Backend)
This document summarizes the support status of Delta Lake table features when used with Apache Gluten (Velox backend).
Supported Spark / Delta combinations
| Spark profile | Spark version | Scala version | Delta Lake version | Status |
|---|---|---|---|---|
spark-3.5 | Spark 3.5.x | 2.12 | 3.3.x | Supported |
spark-4.0 | Spark 4.0.x | 2.13 | 4.0.x | Supported |
Native Delta write is supported in both Spark 3.5 and Spark 4.0 profiles. The difference between the two rows above is the Spark/Delta compatibility target (Spark 3.5 + Delta 3.3 vs Spark 4.0 + Delta 4.0), not a native-write capability gap.
Build and runtime notes
Build Gluten with Delta support by enabling -Pdelta together with the Velox backend profile and a Spark profile.
- Spark 3.5 build example:
mvn clean package -Pbackends-velox -Pdelta -Pspark-3.5 -DskipTests
- Spark 4.0 build example:
mvn clean package -Pbackends-velox -Pdelta -Pspark-4.0 -Pscala-2.13 -Pjava-17 -DskipTests
Native Delta write is controlled by:
spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite- Default:
false - Type: experimental
- Default:
| Feature | Delta minWriterVersion | Delta minReaderVersion | Iceberg format-version | Feature type | Supported by Gluten (Velox) |
|---|---|---|---|---|---|
| Basic functionality | 2 | 1 | 1 | Writer | Yes |
| CHECK constraints | 3 | 1 | N/A | Writer | No |
| Change data feed | 4 | 1 | N/A | Writer | Yes |
| Generated columns | 4 | 1 | N/A | Writer | Partial |
| Column mapping | 5 | 2 | N/A | Reader and writer | Yes |
| Identity columns | 6 | 1 | N/A | Writer | Yes |
| Row tracking | 7 | 1 | 3 | Writer | Partial |
| Deletion vectors | 7 | 3 | 3 | Reader and writer | Partial |
| TimestampNTZ | 7 | 3 | 1 | Reader and writer | No |
| Liquid clustering | 7 | 3 | 1 | Reader and writer | Yes |
| Iceberg readers (UniForm) | 7 | 2 | N/A | Writer | Not tested |
| Type widening | 7 | 3 | N/A | Reader and writer | Partial |
| Variant | 7 | 3 | 3 | Reader and writer | Not tested |
| Variant shredding | 7 | 3 | 3 | Reader and writer | Not tested |
| Collations | 7 | 3 | N/A | Reader and writer | Not tested |
| Protected checkpoints | 7 | 1 | N/A | Writer | Not tested |