pyspark.sql.functions.kll_sketch_get_quantile_float#

pyspark.sql.functions.kll_sketch_get_quantile_float(sketch, rank)[source]#

Extracts a quantile value from a KLL float sketch given an input rank value. The rank can be a single value or an array.

Added in version 4.1.0.

Parameters:

sketchColumn or column name: The KLL float sketch binary representation. A column that evaluates to a binary.
rankColumn or column name: The rank value(s) to extract (between 0.0 and 1.0). A column that evaluates to a double or array. Must be a constant.

Returns:

Column: The quantile value(s). Returns a column that evaluates to a float, or an array of floats if the rank argument is an array.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([1.0,2.0,3.0,4.0,5.0], "FLOAT")
>>> sketch_df = df.agg(sf.kll_sketch_agg_float("value").alias("sketch"))
>>> sketch_df.select(sf.kll_sketch_get_quantile_float("sketch", sf.lit(0.5))).show()
+------------------------------------------+
|kll_sketch_get_quantile_float(sketch, 0.5)|
+------------------------------------------+
|                                       3.0|
+------------------------------------------+