Package org.apache.hadoop.fs.impl

Class AbstractFSBuilderImpl<S,B extends FSBuilder<S,B>>

java.lang.Object
org.apache.hadoop.fs.impl.AbstractFSBuilderImpl<S,B>
Type Parameters:
S - Return type on the FSBuilder.build() call.
B - type of builder itself.
All Implemented Interfaces:
FSBuilder<S,B>
Direct Known Subclasses:
FSDataOutputStreamBuilder, FutureDataInputStreamBuilderImpl, MultipartUploaderBuilderImpl
@Public @Unstable public abstract class AbstractFSBuilderImpl<S,B extends FSBuilder<S,B>> extends Object implements FSBuilder<S,B>
Builder for filesystem/filecontext operations of various kinds, with option support. .opt("fs.s3a.open.option.caching", true) .opt("fs.option.openfile.read.policy", "random, adaptive") .opt("fs.s3a.open.option.etag", "9fe4c37c25b") .optLong("fs.option.openfile.length", 1_500_000_000_000) .must("fs.option.openfile.buffer.size", 256_000) .mustLong("fs.option.openfile.split.start", 256_000_000) .mustLong("fs.option.openfile.split.end", 512_000_000) .build(); Configuration keys declared in an opt() may be ignored by a builder which does not recognise them. Configuration keys declared in a must() function set must be understood by the implementation or a IllegalArgumentException will be thrown.

  • Field Details

  • Constructor Details

    • AbstractFSBuilderImpl

      protected AbstractFSBuilderImpl(@Nonnull Optional<Path> optionalPath, @Nonnull Optional<PathHandle> optionalPathHandle)
      Constructor with both optional path and path handle. Either or both argument may be empty, but it is an error for both to be defined.
      Parameters:
      optionalPath - a path or empty
      optionalPathHandle - a path handle/empty
      Throws:
      IllegalArgumentException - if both parameters are set.

    • AbstractFSBuilderImpl

      protected AbstractFSBuilderImpl(@Nonnull Path path)

    • AbstractFSBuilderImpl

      protected AbstractFSBuilderImpl(@Nonnull PathHandle pathHandle)

  • Method Details

    • getThisBuilder

      public B getThisBuilder()
      Get the cast builder.
      Returns:
      this object, typecast

    • getOptionalPath

      public Optional<Path> getOptionalPath()
      Get the optional path; may be empty.
      Returns:
      the optional path field.

    • getPath

      public Path getPath()
      Get the path: only valid if constructed with a path.
      Returns:
      the path
      Throws:
      NoSuchElementException - if the field is empty.

    • getOptionalPathHandle

      public Optional<PathHandle> getOptionalPathHandle()
      Get the optional path handle; may be empty.
      Returns:
      the optional path handle field.

    • getPathHandle

      public PathHandle getPathHandle()
      Get the PathHandle: only valid if constructed with a PathHandle.
      Returns:
      the PathHandle
      Throws:
      NoSuchElementException - if the field is empty.

    • opt

      public B opt(@Nonnull String key, @Nonnull String value)
      Set optional Builder parameter.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.

    • opt

      public B opt(@Nonnull String key, boolean value)
      Set optional boolean parameter for the Builder.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • opt

      public B opt(@Nonnull String key, int value)
      Set optional int parameter for the Builder.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • opt

      public B opt(@Nonnull String key, long value)
      Description copied from interface: FSBuilder
      Set optional long parameter for the Builder.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.

    • optLong

      public B optLong(@Nonnull String key, long value)
      Description copied from interface: FSBuilder
      Set optional long parameter for the Builder.
      Specified by:
      optLong in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • opt

      public B opt(@Nonnull String key, float value)
      Set optional float parameter for the Builder.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • opt

      public B opt(@Nonnull String key, double value)
      Set optional double parameter for the Builder.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • optDouble

      public B optDouble(@Nonnull String key, double value)
      Set optional double parameter for the Builder.
      Specified by:
      optDouble in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • opt

      public B opt(@Nonnull String key, @Nonnull String... values)
      Set an array of string values as optional parameter for the Builder.
      Specified by:
      opt in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      values - values.
      Returns:
      generic type B.
      See Also:

    • must

      public B must(@Nonnull String key, @Nonnull String value)
      Set mandatory option to the Builder. If the option is not supported or unavailable on the FileSystem, the client should expect FSBuilder.build() throws IllegalArgumentException.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.

    • must

      public B must(@Nonnull String key, boolean value)
      Set mandatory boolean option.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • mustLong

      public B mustLong(@Nonnull String key, long value)
      Description copied from interface: FSBuilder
      Set mandatory long parameter for the Builder.
      Specified by:
      mustLong in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • mustDouble

      public B mustDouble(@Nonnull String key, double value)
      Set optional double parameter for the Builder.
      Specified by:
      mustDouble in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • must

      public B must(@Nonnull String key, int value)
      Set mandatory int option.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • must

      public B must(@Nonnull String key, long value)
      Description copied from interface: FSBuilder
      Set mandatory long option.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • must

      public B must(@Nonnull String key, float value)
      Description copied from interface: FSBuilder
      This parameter is converted to a long and passed to FSBuilder.mustLong(String, long) -all decimal precision is lost.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.

    • must

      public B must(@Nonnull String key, double value)
      Description copied from interface: FSBuilder
      Set mandatory long option, despite passing in a floating point value.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      value - value.
      Returns:
      generic type B.
      See Also:

    • must

      public B must(@Nonnull String key, @Nonnull String... values)
      Set a string array as mandatory option.
      Specified by:
      must in interface FSBuilder<S,B extends FSBuilder<S,B>>
      Parameters:
      key - key.
      values - values.
      Returns:
      generic type B.
      See Also:

    • getOptions

      public Configuration getOptions()
      Get the mutable option configuration.
      Returns:
      the option configuration.

    • getMandatoryKeys

      public Set<String> getMandatoryKeys()
      Get all the keys that are set as mandatory keys.
      Returns:
      mandatory keys.

    • getOptionalKeys

      public Set<String> getOptionalKeys()
      Get all the keys that are set as optional keys.
      Returns:
      optional keys.

    • rejectUnknownMandatoryKeys

      protected void rejectUnknownMandatoryKeys(Collection<String> knownKeys, String extraErrorText) throws IllegalArgumentException
      Reject a configuration if one or more mandatory keys are not in the set of mandatory keys. The first invalid key raises the exception; the order of the scan and hence the specific key raising the exception is undefined.
      Parameters:
      knownKeys - a possibly empty collection of known keys
      extraErrorText - extra error text to include.
      Throws:
      IllegalArgumentException - if any key is unknown.

    • rejectUnknownMandatoryKeys

      public static void rejectUnknownMandatoryKeys(Set<String> mandatory, Collection<String> knownKeys, String extraErrorText) throws IllegalArgumentException
      Reject a configuration if one or more mandatory keys are not in the set of mandatory keys. The first invalid key raises the exception; the order of the scan and hence the specific key raising the exception is undefined.
      Parameters:
      mandatory - the set of mandatory keys
      knownKeys - a possibly empty collection of known keys
      extraErrorText - extra error text to include.
      Throws:
      IllegalArgumentException - if any key is unknown.