Package org.apache.hadoop.fs.azure.metrics

Class AzureFileSystemInstrumentation

java.lang.Object
org.apache.hadoop.fs.azure.metrics.AzureFileSystemInstrumentation
All Implemented Interfaces:
MetricsSource
@Metrics(about="Metrics for WASB", context="azureFileSystem") @Public @Evolving public final class AzureFileSystemInstrumentation extends Object implements MetricsSource
A metrics source for the WASB file system to track all the metrics we care about for getting a clear picture of the performance/reliability/interaction of the Hadoop cluster with Azure Storage.

  • Field Details

  • Constructor Details

    • AzureFileSystemInstrumentation

      public AzureFileSystemInstrumentation(Configuration conf)

  • Method Details

    • getFileSystemInstanceId

      public UUID getFileSystemInstanceId()
      The unique identifier for this file system in the metrics.
      Returns:
      The unique identifier.

    • getMetricsRegistryInfo

      public MetricsInfo getMetricsRegistryInfo()
      Get the metrics registry information.
      Returns:
      The metrics registry information.

    • setAccountName

      public void setAccountName(String accountName)
      Sets the account name to tag all the metrics with.
      Parameters:
      accountName - The account name.

    • setContainerName

      public void setContainerName(String containerName)
      Sets the container name to tag all the metrics with.
      Parameters:
      containerName - The container name.

    • webResponse

      public void webResponse()
      Indicate that we just got a web response from Azure Storage. This should be called for every web request/response we do (to get accurate metrics of how we're hitting the storage service).

    • getCurrentWebResponses

      public long getCurrentWebResponses()
      Gets the current number of web responses obtained from Azure Storage.
      Returns:
      The number of web responses.

    • fileCreated

      public void fileCreated()
      Indicate that we just created a file through WASB.

    • fileDeleted

      public void fileDeleted()
      Indicate that we just deleted a file through WASB.

    • directoryCreated

      public void directoryCreated()
      Indicate that we just created a directory through WASB.

    • directoryDeleted

      public void directoryDeleted()
      Indicate that we just deleted a directory through WASB.

    • updateBytesWrittenInLastSecond

      public void updateBytesWrittenInLastSecond(long currentBytesWritten)
      Sets the current gauge value for how many bytes were written in the last second.
      Parameters:
      currentBytesWritten - The number of bytes.

    • updateBytesReadInLastSecond

      public void updateBytesReadInLastSecond(long currentBytesRead)
      Sets the current gauge value for how many bytes were read in the last second.
      Parameters:
      currentBytesRead - The number of bytes.

    • currentUploadBytesPerSecond

      public void currentUploadBytesPerSecond(long bytesPerSecond)
      Record the current bytes-per-second upload rate seen.
      Parameters:
      bytesPerSecond - The bytes per second.

    • currentDownloadBytesPerSecond

      public void currentDownloadBytesPerSecond(long bytesPerSecond)
      Record the current bytes-per-second download rate seen.
      Parameters:
      bytesPerSecond - The bytes per second.

    • rawBytesUploaded

      public void rawBytesUploaded(long numberOfBytes)
      Indicate that we just uploaded some data to Azure storage.
      Parameters:
      numberOfBytes - The raw number of bytes uploaded (including overhead).

    • rawBytesDownloaded

      public void rawBytesDownloaded(long numberOfBytes)
      Indicate that we just downloaded some data to Azure storage.
      Parameters:
      numberOfBytes - The raw number of bytes downloaded (including overhead).

    • blockUploaded

      public void blockUploaded(long latency)
      Indicate that we just uploaded a block and record its latency.
      Parameters:
      latency - The latency in milliseconds.

    • blockDownloaded

      public void blockDownloaded(long latency)
      Indicate that we just downloaded a block and record its latency.
      Parameters:
      latency - The latency in milliseconds.

    • clientErrorEncountered

      public void clientErrorEncountered()
      Indicate that we just encountered a client-side error.

    • serverErrorEncountered

      public void serverErrorEncountered()
      Indicate that we just encountered a server-caused error.

    • getBlockUploadLatency

      public long getBlockUploadLatency()
      Get the current rolling average of the upload latency.
      Returns:
      rolling average of upload latency in milliseconds.

    • getBlockDownloadLatency

      public long getBlockDownloadLatency()
      Get the current rolling average of the download latency.
      Returns:
      rolling average of download latency in milliseconds.

    • getCurrentMaximumUploadBandwidth

      public long getCurrentMaximumUploadBandwidth()
      Get the current maximum upload bandwidth.
      Returns:
      maximum upload bandwidth in bytes per second.

    • getCurrentMaximumDownloadBandwidth

      public long getCurrentMaximumDownloadBandwidth()
      Get the current maximum download bandwidth.
      Returns:
      maximum download bandwidth in bytes per second.

    • getMetrics

      public void getMetrics(MetricsCollector builder, boolean all)
      Description copied from interface: MetricsSource
      Get metrics from the metrics source
      Specified by:
      getMetrics in interface MetricsSource
      Parameters:
      builder - to contain the resulting metrics snapshot
      all - if true, return all metrics even if unchanged.