The REST API for Blob Storage defines HTTP operations against the storage account, containers(filesystems), and blobs(files). The API includes the operations listed in the following table.
| Operation | Resource Type | Description |
|---|---|---|
| Create Container | Filesystem | Creates a new azure storage container to be used as an hadoop filesystem. |
| Delete Container | Filesystem | Deletes the specified container acting as hadoop filesystem. |
| Set Container Metadata | Filesystem | Sets the metadata of the specified container acting as hadoop filesystem. |
| Get Container Properties | Filesystem | Gets the metadata of the specified container acting as hadoop filesystem. |
| List Blobs | Filesystem | Lists the paths under the specified directory inside container acting as hadoop filesystem. |
| Put Blob | Path | Creates a new path or updates an existing path under the specified filesystem (container). |
| Lease Blob | Path | Establishes and manages a lease on the specified path. |
| Put Block | Path | Appends Data to an already created blob at specified path. |
| Put Block List | Path | Flushes The Appended Data to the blob at specified path. |
| Set Blob Metadata | Path | Sets the user-defined attributes of the blob at specified path. |
| Get Blob Properties | Path | Gets the user-defined attributes of the blob at specified path. |
| Get Blob | Path | Reads data from the blob at specified path. |
| Delete Blob | Path | Deletes the blob at specified path. |
| Get Block List | Path | Retrieves the list of blocks that have been uploaded as part of a block blob. |
| Copy Blob | Path | Copies a blob to a destination within the storage account. |
| Append Block | Path | Commits a new block of data to the end of an existing append blob. |
The Create Container operation creates a new container under the specified account. If the container with the same name already exists, the operation fails. Rest API Documentation: Create Container
The Delete Container operation marks the specified container for deletion. The container and any blobs contained within it. Rest API Documentation: Delete Container
The Set Container Metadata operation sets user-defined metadata for the specified container as one or more name-value pairs. Rest API Documentation: Set Container Metadata
The Get Container Properties operation returns all user-defined metadata and system properties for the specified container. The returned data doesn’t include the container’s list of blobs. Rest API Documentation: Get Container Properties
The List Blobs operation returns a list of the blobs under the specified container. Rest API Documentation: List Blobs
By default the List Blobs response is parsed as XML. When Photon is enabled the driver additionally advertises the Apache Arrow IPC stream media type in the Accept header (application/vnd.apache.arrow.stream,application/xml) so the service may return a compact Apache Arrow response instead of XML. The response format is detected from the returned Content-Type: an Arrow response is parsed by the Arrow parser while any other (or missing) content type falls back to the existing XML parser. Both paths produce identical FileStatus results, so the public filesystem APIs are unchanged.
Photon is controlled by a single configuration and is disabled by default:
<property> <name>fs.azure.photon.enabled</name> <value>false</value> </property>
Photon applies only to non-HNS (flat namespace) accounts; on hierarchical namespace (HNS) accounts the Arrow request is not sent and listing stays on the XML path.
--add-opens)Apache Arrow’s off-heap allocator reflectively accesses java.nio internals, which the Java module system closes by default from Java 9 onwards. On Java 17+ the JVM running the driver (the client application, e.g. Spark or Hive executors, not just the test harness) must therefore open the java.nio package to Arrow, otherwise Arrow fails to initialise and any listing that receives an Arrow response fails:
--add-opens=java.base/java.nio=ALL-UNNAMED
Add this flag to the launching JVM (for example via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions, HADOOP_OPTS, or the container’s JAVA_TOOL_OPTIONS). If you cannot guarantee the flag is present on every JVM that runs the driver, keep fs.azure.photon.enabled set to false so listing uses the always-available XML path.
The Put Blob operation creates a new block blob, or updates the content of an existing block blob. The Put Blob operation will overwrite all contents of an existing blob with the same name. When you update an existing block blob, you overwrite any existing metadata on the blob. The content of the existing blob is overwritten with the content of the new blob. Partial updates are not supported with Put Blob Rest API Documentation: Put Blob
The Lease Blob operation creates and manages a lock on a blob for write and delete operations. The lock duration can be 15 to 60 seconds, or can be infinite. Rest API Documentation: Lease Blob
The Put Block operation creates a new block to be committed as part of a blob. Rest API Documentation: Put Block
The Put Block List operation writes a blob by specifying the list of block IDs that make up the blob. To be written as part of a blob, a block must have been successfully written to the server in an earlier Put Block operation. You can call Put Block List to update a blob by uploading only those blocks that have changed and then committing the new and existing blocks together. Rest API Documentation: Put Block List
The Set Blob Metadata operation sets user-defined metadata for the specified blob as one or more name-value pairs. Rest API Documentation: Set Blob Metadata
The Get Blob Properties operation returns all user-defined metadata, standard HTTP properties, and system properties for the blob. Rest API Documentation: Get Blob Properties
The Get Blob operation reads or downloads a blob from the system, including its metadata and properties. Rest API Documentation: Get Blob
The Delete Blob operation marks the specified blob for deletion. The blob is later deleted during garbage collection. Rest API Documentation: Delete Blob
The Get Block List operation retrieves the list of blocks that have been uploaded as part of a block blob. Rest API Documentation: Get Block List
The Copy Blob operation copies a blob to a destination within the storage account. Rest API Documentation: Copy Blob
The Append Block operation commits a new block of data to the end of an existing append blob. Rest API Documentation: Append Block