Package org.apache.hadoop.security.token.delegation

Class AbstractDelegationTokenSecretManager<TokenIdent extends AbstractDelegationTokenIdentifier>

java.lang.Object
org.apache.hadoop.security.token.SecretManager<TokenIdent>
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager<TokenIdent>
@Public @Evolving public abstract class AbstractDelegationTokenSecretManager<TokenIdent extends AbstractDelegationTokenIdentifier> extends SecretManager<TokenIdent>

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static class 
    org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation
    Class to encapsulate a token's renew date and password.

    Nested classes/interfaces inherited from class org.apache.hadoop.security.token.SecretManager

    org.apache.hadoop.security.token.SecretManager.InvalidToken

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final Map<Integer,org.apache.hadoop.security.token.delegation.DelegationKey>
    allKeys
    Access to allKeys is protected by this object lock
    protected int
    currentId
    Access to currentId is protected by this object lock.
    protected Map<TokenIdent,org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation>
    currentTokens
    Cache of currently valid tokens, mapping from DelegationTokenIdentifier to DelegationTokenInformation.
    protected int
    delegationTokenSequenceNumber
    Sequence number to create DelegationTokenIdentifier.
    protected Object
    noInterruptsLock
    If the delegation token update thread holds this lock, it will not get interrupted.
    protected boolean
    running
     
    protected boolean
    storeTokenTrackingId
    Whether to store a token's tracking ID in its TokenInformation.
    protected final Map<String,Long>
    tokenOwnerStats
    Map of token real owners to its token count.

  • Constructor Summary

    Constructors
    Constructor
    Description
    AbstractDelegationTokenSecretManager(long delegationKeyUpdateInterval, long delegationTokenMaxLifetime, long delegationTokenRenewInterval, long delegationTokenRemoverScanInterval)
    Create a secret manager

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addKey(org.apache.hadoop.security.token.delegation.DelegationKey key)
    Add a previously used master key to cache (when NN restarts), should be called before activate().
    void
    addPersistedDelegationToken(TokenIdent identifier, long renewDate)
    This method is intended to be used for recovering persisted delegation tokens.
    protected void
    addTokenForOwnerStats(TokenIdent id)
    Add token stats to the owner to token count mapping.
    TokenIdent
    cancelToken(Token<TokenIdent> token, String canceller)
    Cancel a token by removing it from cache.
    protected org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation
    checkToken(TokenIdent identifier)
    Find the DelegationTokenInformation for the given token id, and verify that if the token is expired.
    protected byte[]
    createPassword(TokenIdent identifier)
    Create the password for the given identifier.
    static SecretKey
    createSecretKey(byte[] key)
    Convert the byte[] to a secret key
    TokenIdent
    decodeTokenIdentifier(Token<TokenIdent> token)
    Decode the token identifier.
    org.apache.hadoop.security.token.delegation.DelegationKey[]
    getAllKeys()
     
    protected Map<TokenIdent,org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation>
    getCandidateTokensForCleanup()
     
    protected int
    getCurrentKeyId()
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    long
    getCurrentTokensSize()
    Total count of active delegation tokens.
    protected org.apache.hadoop.security.token.delegation.DelegationKey
    getDelegationKey(int keyId)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected int
    getDelegationTokenSeqNum()
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenSecretManagerMetrics
    getMetrics()
     
    protected org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation
    getTokenInfo(TokenIdent ident)
    For subclasses externalizing the storage, for example Zookeeper based implementations
    protected long
    getTokenRenewInterval()
    Interval for tokens to be renewed.
    String
    getTokenTrackingId(TokenIdent identifier)
     
    List<org.apache.hadoop.metrics2.util.Metrics2Util.NameValuePair>
    getTopTokenRealOwners(int n)
    Return top token real owners list as well as the tokens count.
    protected String
    getTrackingIdIfEnabled(TokenIdent ident)
     
    protected int
    incrementCurrentKeyId()
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected int
    incrementDelegationTokenSeqNum()
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    boolean
    isRunning()
    is secretMgr running
    protected void
    logExpireToken(TokenIdent ident)
     
    protected void
    logExpireTokens(Collection<TokenIdent> expiredTokens)
     
    protected void
    logUpdateMasterKey(org.apache.hadoop.security.token.delegation.DelegationKey key)
     
    protected void
    removeExpiredStoredToken(TokenIdent ident)
     
    protected void
    removeStoredMasterKey(org.apache.hadoop.security.token.delegation.DelegationKey key)
     
    protected void
    removeStoredToken(TokenIdent ident)
     
    long
    renewToken(Token<TokenIdent> token, String renewer)
    Renew a delegation token.
    void
    reset()
    Reset all data structures and mutable state.
    byte[]
    retrievePassword(TokenIdent identifier)
    Retrieve the password for the given token identifier.
    protected void
    rollMasterKey()
    Update the current master key for generating delegation tokens It should be called only by tokenRemoverThread.
    protected void
    setCurrentKeyId(int keyId)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected void
    setDelegationTokenSeqNum(int seqNum)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    void
    startThreads()
    should be called before this object is used.
    void
    stopThreads()
     
    protected void
    storeDelegationKey(org.apache.hadoop.security.token.delegation.DelegationKey key)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected void
    storeNewMasterKey(org.apache.hadoop.security.token.delegation.DelegationKey key)
     
    protected void
    storeNewToken(TokenIdent ident, long renewDate)
     
    protected void
    storeToken(TokenIdent ident, org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation tokenInfo)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected void
    syncTokenOwnerStats()
    This method syncs token information from currentTokens to tokenOwnerStats.
    protected void
    updateDelegationKey(org.apache.hadoop.security.token.delegation.DelegationKey key)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    protected void
    updateStoredToken(TokenIdent ident, long renewDate)
     
    protected void
    updateToken(TokenIdent ident, org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation tokenInfo)
    For subclasses externalizing the storage, for example Zookeeper based implementations.
    void
    verifyToken(TokenIdent identifier, byte[] password)
    Verifies that the given identifier and password are valid and match.

    Methods inherited from class org.apache.hadoop.security.token.SecretManager

    checkAvailableForRead, createIdentifier, createPassword, generateSecret, retriableRetrievePassword, update, validateSecretKeyLength

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • currentTokens

      protected Map<TokenIdent extends AbstractDelegationTokenIdentifier,org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation> currentTokens
      Cache of currently valid tokens, mapping from DelegationTokenIdentifier to DelegationTokenInformation. Protected by this object lock.

    • tokenOwnerStats

      protected final Map<String,Long> tokenOwnerStats
      Map of token real owners to its token count. This is used to generate metrics of top users by owned tokens.

    • delegationTokenSequenceNumber

      protected int delegationTokenSequenceNumber
      Sequence number to create DelegationTokenIdentifier. Protected by this object lock.

    • allKeys

      protected final Map<Integer,org.apache.hadoop.security.token.delegation.DelegationKey> allKeys
      Access to allKeys is protected by this object lock

    • currentId

      protected int currentId
      Access to currentId is protected by this object lock.

    • storeTokenTrackingId

      protected boolean storeTokenTrackingId
      Whether to store a token's tracking ID in its TokenInformation. Can be overridden by a subclass.

    • running

      protected volatile boolean running

    • noInterruptsLock

      protected Object noInterruptsLock
      If the delegation token update thread holds this lock, it will not get interrupted.

  • Constructor Details

    • AbstractDelegationTokenSecretManager

      public AbstractDelegationTokenSecretManager(long delegationKeyUpdateInterval, long delegationTokenMaxLifetime, long delegationTokenRenewInterval, long delegationTokenRemoverScanInterval)
      Create a secret manager
      Parameters:
      delegationKeyUpdateInterval - the number of milliseconds for rolling new secret keys.
      delegationTokenMaxLifetime - the maximum lifetime of the delegation tokens in milliseconds
      delegationTokenRenewInterval - how often the tokens must be renewed in milliseconds
      delegationTokenRemoverScanInterval - how often the tokens are scanned for expired tokens in milliseconds

  • Method Details

    • startThreads

      public void startThreads() throws IOException
      should be called before this object is used.
      Throws:
      IOException - raised on errors performing I/O.

    • reset

      public void reset()
      Reset all data structures and mutable state.

    • getCurrentTokensSize

      public long getCurrentTokensSize()
      Total count of active delegation tokens.
      Returns:
      currentTokens.size.

    • getTokenRenewInterval

      protected long getTokenRenewInterval()
      Interval for tokens to be renewed.
      Returns:
      Renew interval in milliseconds.

    • addKey

      public void addKey(org.apache.hadoop.security.token.delegation.DelegationKey key) throws IOException
      Add a previously used master key to cache (when NN restarts), should be called before activate().
      Parameters:
      key - delegation key.
      Throws:
      IOException - raised on errors performing I/O.

    • getAllKeys

      public org.apache.hadoop.security.token.delegation.DelegationKey[] getAllKeys()

    • logUpdateMasterKey

      protected void logUpdateMasterKey(org.apache.hadoop.security.token.delegation.DelegationKey key) throws IOException
      Throws:
      IOException

    • logExpireToken

      protected void logExpireToken(TokenIdent ident) throws IOException
      Throws:
      IOException

    • storeNewMasterKey

      protected void storeNewMasterKey(org.apache.hadoop.security.token.delegation.DelegationKey key) throws IOException
      Throws:
      IOException

    • removeStoredMasterKey

      protected void removeStoredMasterKey(org.apache.hadoop.security.token.delegation.DelegationKey key)

    • storeNewToken

      protected void storeNewToken(TokenIdent ident, long renewDate) throws IOException
      Throws:
      IOException

    • removeStoredToken

      protected void removeStoredToken(TokenIdent ident) throws IOException
      Throws:
      IOException

    • updateStoredToken

      protected void updateStoredToken(TokenIdent ident, long renewDate) throws IOException
      Throws:
      IOException

    • getCurrentKeyId

      protected int getCurrentKeyId()
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Returns:
      currentId.

    • incrementCurrentKeyId

      protected int incrementCurrentKeyId()
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Returns:
      currentId.

    • setCurrentKeyId

      protected void setCurrentKeyId(int keyId)
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      keyId - keyId.

    • getDelegationTokenSeqNum

      protected int getDelegationTokenSeqNum()
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Returns:
      delegationTokenSequenceNumber.

    • incrementDelegationTokenSeqNum

      protected int incrementDelegationTokenSeqNum()
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Returns:
      delegationTokenSequenceNumber.

    • setDelegationTokenSeqNum

      protected void setDelegationTokenSeqNum(int seqNum)
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      seqNum - seqNum.

    • getDelegationKey

      protected org.apache.hadoop.security.token.delegation.DelegationKey getDelegationKey(int keyId)
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      keyId - keyId.
      Returns:
      DelegationKey.

    • storeDelegationKey

      protected void storeDelegationKey(org.apache.hadoop.security.token.delegation.DelegationKey key) throws IOException
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      key - DelegationKey.
      Throws:
      IOException - raised on errors performing I/O.

    • updateDelegationKey

      protected void updateDelegationKey(org.apache.hadoop.security.token.delegation.DelegationKey key) throws IOException
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      key - DelegationKey.
      Throws:
      IOException - raised on errors performing I/O.

    • getTokenInfo

      protected org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation getTokenInfo(TokenIdent ident)
      For subclasses externalizing the storage, for example Zookeeper based implementations
      Parameters:
      ident - ident.
      Returns:
      DelegationTokenInformation.

    • storeToken

      protected void storeToken(TokenIdent ident, org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation tokenInfo) throws IOException
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      ident - ident.
      tokenInfo - tokenInfo.
      Throws:
      IOException - raised on errors performing I/O.

    • updateToken

      protected void updateToken(TokenIdent ident, org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation tokenInfo) throws IOException
      For subclasses externalizing the storage, for example Zookeeper based implementations.
      Parameters:
      ident - ident.
      tokenInfo - tokenInfo.
      Throws:
      IOException - raised on errors performing I/O.

    • addPersistedDelegationToken

      public void addPersistedDelegationToken(TokenIdent identifier, long renewDate) throws IOException
      This method is intended to be used for recovering persisted delegation tokens. Tokens that have an unknown DelegationKey are marked as expired and automatically cleaned up. This method must be called before this secret manager is activated (before startThreads() is called)
      Parameters:
      identifier - identifier read from persistent storage
      renewDate - token renew time
      Throws:
      IOException - raised on errors performing I/O.

    • rollMasterKey

      protected void rollMasterKey() throws IOException
      Update the current master key for generating delegation tokens It should be called only by tokenRemoverThread.
      Throws:
      IOException - raised on errors performing I/O.

    • createPassword

      protected byte[] createPassword(TokenIdent identifier)
      Description copied from class: SecretManager
      Create the password for the given identifier. identifier may be modified inside this method.
      Specified by:
      createPassword in class SecretManager<TokenIdent extends AbstractDelegationTokenIdentifier>
      Parameters:
      identifier - the identifier to use
      Returns:
      the new password

    • checkToken

      protected org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation checkToken(TokenIdent identifier) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Find the DelegationTokenInformation for the given token id, and verify that if the token is expired. Note that this method should be called with acquiring the secret manager's monitor.
      Parameters:
      identifier - identifier.
      Returns:
      DelegationTokenInformation.
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken - invalid token exception.

    • retrievePassword

      public byte[] retrievePassword(TokenIdent identifier) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Description copied from class: SecretManager
      Retrieve the password for the given token identifier. Should check the date or registry to make sure the token hasn't expired or been revoked. Returns the relevant password.
      Specified by:
      retrievePassword in class SecretManager<TokenIdent extends AbstractDelegationTokenIdentifier>
      Parameters:
      identifier - the identifier to validate
      Returns:
      the password to use
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken - the token was invalid

    • getTrackingIdIfEnabled

      protected String getTrackingIdIfEnabled(TokenIdent ident)

    • getTokenTrackingId

      public String getTokenTrackingId(TokenIdent identifier)

    • verifyToken

      public void verifyToken(TokenIdent identifier, byte[] password) throws org.apache.hadoop.security.token.SecretManager.InvalidToken
      Verifies that the given identifier and password are valid and match.
      Parameters:
      identifier - Token identifier.
      password - Password in the token.
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken - InvalidToken.

    • renewToken

      public long renewToken(Token<TokenIdent> token, String renewer) throws org.apache.hadoop.security.token.SecretManager.InvalidToken, IOException
      Renew a delegation token.
      Parameters:
      token - the token to renew
      renewer - the full principal name of the user doing the renewal
      Returns:
      the new expiration time
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken - if the token is invalid
      AccessControlException - if the user can't renew token
      IOException

    • cancelToken

      public TokenIdent cancelToken(Token<TokenIdent> token, String canceller) throws IOException
      Cancel a token by removing it from cache.
      Parameters:
      token - token.
      canceller - canceller.
      Returns:
      Identifier of the canceled token
      Throws:
      org.apache.hadoop.security.token.SecretManager.InvalidToken - for invalid token
      AccessControlException - if the user isn't allowed to cancel
      IOException

    • createSecretKey

      public static SecretKey createSecretKey(byte[] key)
      Convert the byte[] to a secret key
      Parameters:
      key - the byte[] to create the secret key from
      Returns:
      the secret key

    • getCandidateTokensForCleanup

      protected Map<TokenIdent,org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenInformation> getCandidateTokensForCleanup()

    • logExpireTokens

      protected void logExpireTokens(Collection<TokenIdent> expiredTokens) throws IOException
      Throws:
      IOException

    • removeExpiredStoredToken

      protected void removeExpiredStoredToken(TokenIdent ident) throws IOException
      Throws:
      IOException

    • stopThreads

      public void stopThreads()

    • isRunning

      public boolean isRunning()
      is secretMgr running
      Returns:
      true if secret mgr is running

    • decodeTokenIdentifier

      public TokenIdent decodeTokenIdentifier(Token<TokenIdent> token) throws IOException
      Decode the token identifier. The subclass can customize the way to decode the token identifier.
      Parameters:
      token - the token where to extract the identifier
      Returns:
      the delegation token identifier
      Throws:
      IOException - raised on errors performing I/O.

    • getTopTokenRealOwners

      public List<org.apache.hadoop.metrics2.util.Metrics2Util.NameValuePair> getTopTokenRealOwners(int n)
      Return top token real owners list as well as the tokens count.
      Parameters:
      n - top number of users
      Returns:
      map of owners to counts

    • addTokenForOwnerStats

      protected void addTokenForOwnerStats(TokenIdent id)
      Add token stats to the owner to token count mapping.
      Parameters:
      id - token id.

    • syncTokenOwnerStats

      protected void syncTokenOwnerStats()
      This method syncs token information from currentTokens to tokenOwnerStats. It is used when the currentTokens is initialized or refreshed. This is called from a single thread thus no synchronization is needed.

    • getMetrics

      protected org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.DelegationTokenSecretManagerMetrics getMetrics()