Type Members

1.  trait CredentialProvider extends AnyRef

   :: DeveloperApi :: Service Provider Interface for credential resolution in the OIDC credential propagation framework.

   :: DeveloperApi :: Service Provider Interface for credential resolution in the OIDC credential propagation framework.

   Implementations exchange a user's identity (represented by UserContext) for a short-lived ServiceCredential scoped to a target URI. Providers are discovered via java.util.ServiceLoader and selected based on the URI scheme.

   Implementations must be thread-safe: resolve() may be called concurrently from multiple threads after init() completes.

   Annotations

   @DeveloperApi()

   Since

   4.3.0

2.  final class CredentialProviderLoader extends AnyRef

   :: Private :: Discovers CredentialProvider implementations via ServiceLoader and selects the appropriate provider for a given URI scheme using Binding Policy A (explicit selection).

   :: Private :: Discovers CredentialProvider implementations via ServiceLoader and selects the appropriate provider for a given URI scheme using Binding Policy A (explicit selection).

   Provider discovery happens once (lazily on first call) and the list is cached. Each provider is initialized exactly once per provider instance via CredentialProvider#init(Map) with the configuration from the first call that selects it (first-conf-wins semantics); subsequent resolutions reuse the already-initialized instance without re-calling init.

   When the configuration key spark.security.oidc.provider. is set (non-empty), the loader validates the configured fully-qualified class name against the discovered candidates for that scheme regardless of candidate count. If the configured class does not match any candidate, an IllegalArgumentException is thrown listing the scheme, the configured class, and the available candidates. Only when the configuration key is unset (or empty) do the count-based rules apply: a single candidate is auto-selected; multiple candidates produce an ambiguity error; no candidates produce Optional.empty().

   Thread-safety: This class uses synchronized access to the cached provider list and initialization tracking, and callers may invoke Map) from multiple threads. A provider instance is cached and shared across callers; per the CredentialProvider contract, implementations must be thread-safe, so a returned instance may be used concurrently.

   Annotations

   @Private()

   Since

   4.3.0

3.  class CredentialResolutionException extends Exception

   :: DeveloperApi :: Thrown when a CredentialProvider fails to resolve credentials for a target URI.

   :: DeveloperApi :: Thrown when a CredentialProvider fails to resolve credentials for a target URI.

   This is a checked exception to ensure callers handle credential resolution failures explicitly (e.g., retry, fail the job, or fall back to another mechanism).

   Annotations

   @DeveloperApi()

   Since

   4.3.0

4.  class FileTokenIngestor extends TokenIngestor

   A TokenIngestor that reads an OIDC identity token from a file.

   A TokenIngestor that reads an OIDC identity token from a file.

   The file path is typically a Kubernetes projected service account token (e.g., /var/run/secrets/tokens/spark-identity) or a path configured via spark.security.oidc.identityToken.file.

   This implementation:

   • Detects file rotation via mtime change (only re-parses when the file changes)
   • Parses JWT claims by Base64-decoding the payload segment directly, without signature verification, since the token is trusted from the local filesystem and works with both signed (RS256, ES256) and unsigned tokens
   • Handles errors gracefully: malformed JWT, missing file, or empty content returns empty rather than throwing

   Annotations

   @Private()

   Since

   4.3.0

5.  trait GroupMappingServiceProvider extends AnyRef

   This Spark trait is used for mapping a given userName to a set of groups which it belongs to.

   This Spark trait is used for mapping a given userName to a set of groups which it belongs to. This is useful for specifying a common group of admins/developers to provide them admin, modify and/or view access rights. Based on whether access control checks are enabled using spark.acls.enable, every time a user tries to access or modify the application, the SecurityManager gets the corresponding groups a user belongs to from the instance of the groups mapping provider specified by the entry spark.user.groups.mapping.

6.  trait HadoopDelegationTokenProvider extends AnyRef

   ::DeveloperApi:: Hadoop delegation token provider.

   ::DeveloperApi:: Hadoop delegation token provider.

   Annotations

   @DeveloperApi()

7.  final class ServiceCredential extends Serializable

   :: DeveloperApi :: A short-lived, service-specific credential derived from the user's identity token.

   :: DeveloperApi :: A short-lived, service-specific credential derived from the user's identity token.

   Instances are produced by credential providers on the driver and transmitted to executors via UserCredentials. The properties map holds service-specific key-value pairs (e.g., temporary AWS credentials).

   This class is immutable and Serializable.

   Annotations

   @DeveloperApi()

   Since

   4.3.0

8.  trait TokenIngestor extends AnyRef

   :: DeveloperApi :: Read an OIDC identity token and produces a UserContext.

   :: DeveloperApi :: Read an OIDC identity token and produces a UserContext.

   Implementation should be stateless with respect to Spark configuration; configuration is passed at construction time.

   Annotations

   @DeveloperApi()

   Since

   4.3.0

9.  final class UserContext extends AnyRef

   :: DeveloperApi :: Represents the authenticated user's identity context on the driver side.

   :: DeveloperApi :: Represents the authenticated user's identity context on the driver side.

   This class holds the OIDC token information used to derive short-lived ServiceCredential instances via credential providers. It is intentionally not java.io.Serializable and must never be transmitted to executors. The rawToken field is always redacted in #toString().

   Annotations

   @DeveloperApi()

   Since

   4.3.0

10.  final class UserCredentials extends Serializable

    :: DeveloperApi :: A bundle of ServiceCredential instances keyed by scheme (e.g., "s3a", "abfss").

    :: DeveloperApi :: A bundle of ServiceCredential instances keyed by scheme (e.g., "s3a", "abfss").

    Scheme keys are normalized to lowercase (Locale#ROOT) at construction time, and lookups via #forScheme(String) are case-insensitive. If the supplied map contains keys that differ only by case, an IllegalArgumentException is thrown.

    This class is transmitted to executors and does not contain any reference to UserContext or raw identity tokens. It is immutable and Serializable.

    Annotations

    @DeveloperApi()

    Since

    4.3.0