Hash Functions¶

This page lists all hash functions available in Spark SQL.

---

crc32¶

crc32(expr) - Returns a cyclic redundancy check value of the expr as a bigint.

Examples:

> SELECT crc32('Spark');
 1557323817

Since: 1.5.0

hash¶

hash(expr1, expr2, ...) - Returns a hash value of the arguments.

Examples:

> SELECT hash('Spark', array(123), 2);
 -1321691492

Since: 2.0.0

md5¶

md5(expr) - Returns an MD5 128-bit checksum as a hex string of expr.

Examples:

> SELECT md5('Spark');
 8cde774d6f7333752ed72cacddb05126

Since: 1.5.0

sha¶

sha(expr) - Returns a sha1 hash value as a hex string of the expr.

Examples:

> SELECT sha('Spark');
 85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c

Since: 1.5.0

sha1¶

sha1(expr) - Returns a sha1 hash value as a hex string of the expr.

Examples:

> SELECT sha1('Spark');
 85f5955f4b27a9a4c2aab6ffe5d7189fc298b92c

Since: 1.5.0

sha2¶

sha2(expr, bitLength) - Returns a checksum of SHA-2 family as a hex string of expr. SHA-224, SHA-256, SHA-384, and SHA-512 are supported. Bit length of 0 is equivalent to 256.

Examples:

> SELECT sha2('Spark', 256);
 529bc3b07127ecb7e53a4dcf1991d9152c24537d919178022b2c42657f79a26b

Since: 1.5.0

xxhash64¶

xxhash64(expr1, expr2, ...) - Returns a 64-bit hash value of the arguments. Hash seed is 42.

Examples:

> SELECT xxhash64('Spark', array(123), 2);
 5602566077635097486

Since: 3.0.0