Hash Functions¶
This page lists all hash functions available in Spark SQL.
crc32¶
crc32(expr) - Returns a cyclic redundancy check value of the expr as a bigint.
Arguments:
- expr - The expression to compute the cyclic redundancy check value of. An expression that evaluates to a binary.
Examples:
> SELECT crc32('Spark');
1557323817
Since: 1.5.0
hash¶
hash(expr1, expr2, ...) - Returns a hash value of the arguments.
Examples:
> SELECT hash('Spark', array(123), 2);
-1321691492
Since: 2.0.0
md5¶
md5(expr) - Returns an MD5 128-bit checksum as a hex string of expr.
Arguments:
- expr - The expression to compute the MD5 checksum of. An expression that evaluates to a binary.
Examples:
> SELECT md5('Spark');
8cde774d6f7333752ed72cacddb05126
Since: 1.5.0
sha¶
sha(expr) - Returns a sha1 hash value as a hex string of the expr.
Arguments:
- expr - The expression to compute the SHA-1 hash of. An expression that evaluates to a binary.
Examples:
> SELECT sha('Spark');
85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c
Since: 1.5.0
sha1¶
sha1(expr) - Returns a sha1 hash value as a hex string of the expr.
Arguments:
- expr - The expression to compute the SHA-1 hash of. An expression that evaluates to a binary.
Examples:
> SELECT sha1('Spark');
85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c
Since: 1.5.0
sha2¶
sha2(expr, bitLength) - Returns a checksum of SHA-2 family as a hex string of expr.
SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256.
Arguments:
- expr - The expression to compute the SHA-2 checksum of. An expression that evaluates to a binary.
- bitLength - The bit length of the SHA-2 result (224, 256, 384, or 512). An expression that evaluates to an integer.
Examples:
> SELECT sha2('Spark', 256);
529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b
Since: 1.5.0
xxhash64¶
xxhash64(expr1, expr2, ...) - Returns a 64-bit hash value of the arguments. Hash seed is 42.
Examples:
> SELECT xxhash64('Spark', array(123), 2);
5602566077635097486
Since: 3.0.0