Package org.apache.sysds.runtime.transform.encode

Class ColumnEncoderBin

java.lang.Object

org.apache.sysds.runtime.transform.encode.ColumnEncoder

org.apache.sysds.runtime.transform.encode.ColumnEncoderBin

All Implemented Interfaces:

Externalizable, Serializable, Comparable<ColumnEncoder>, Encoder

public class ColumnEncoderBin
extends ColumnEncoder

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ColumnEncoderBin.BinMethod

Nested classes/interfaces inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

ColumnEncoder.EncoderType

Field Summary

Fields

Modifier and Type	Field	Description
static String	MAX_PREFIX
static String	MIN_PREFIX
static int	MINIMUM_SAMPLE_SIZE
static String	NBINS_PREFIX
static double	SAMPLE_FRACTION

Fields inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

APPLY_ROW_BLOCKS_PER_COLUMN, BUILD_ROW_BLOCKS_PER_COLUMN

Constructor Summary

Constructors

Constructor	Description
ColumnEncoderBin()
ColumnEncoderBin(int colID, int numBin, double[] binMins, double[] binMaxs)
ColumnEncoderBin(int colID, int numBin, ColumnEncoderBin.BinMethod binMethod)

Method Summary

Modifier and Type	Method	Description
void	allocateMetaData(FrameBlock meta)	Pre-allocate a FrameBlock for metadata collection.
void	build(CacheBlock<?> in)	Build the transform meta data for the given block input.
void	build(CacheBlock<?> in, double[] equiHeightMaxs)
void	buildPartial(FrameBlock in)	Partial build of internal data structures (e.g., in distributed spark operations).
void	computeBins(double min, double max)
double[]	getBinMaxs()
ColumnEncoderBin.BinMethod	getBinMethod()
double[]	getBinMins()
Callable<Object>	getBuildTask(CacheBlock<?> in)
double	getColMaxs()
double	getColMins()
FrameBlock	getMetaData(FrameBlock meta)	Construct a frame block out of the transform meta data.
int	getNumBin()
Callable<Object>	getPartialBuildTask(CacheBlock<?> in, int startRow, int blockSize, HashMap<Integer,Object> ret)
Callable<Object>	getPartialMergeBuildTask(HashMap<Integer,?> ret)
void	initMetaData(FrameBlock meta)	Sets up the required meta data for a subsequent call to apply.
void	mergeAt(ColumnEncoder other)	Merges another encoder, of a compatible type, in after a certain position.
void	prepareBuildPartial()	Allocates internal data structures for partial build.
void	readExternal(ObjectInput in)	Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.
void	setBinMethod(String method)
String	toString()
void	writeExternal(ObjectOutput out)	Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Methods inherited from class org.apache.sysds.runtime.transform.encode.ColumnEncoder

apply, apply, build, compareTo, getApplyTasks, getBuildTasks, getColID, getColMapping, getDomainSize, getEstMetaSize, getEstNumDistincts, getSparseRowsWZeros, initEmbeddings, isApplicable, isApplicable, setColID, setEstMetaSize, setEstNumDistincts, shiftCol, updateIndexRanges

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MIN_PREFIX

public static final String MIN_PREFIX

See Also:

Constant Field Values

MAX_PREFIX

public static final String MAX_PREFIX

See Also:

Constant Field Values

NBINS_PREFIX

public static final String NBINS_PREFIX

See Also:

Constant Field Values

SAMPLE_FRACTION

public static final double SAMPLE_FRACTION

See Also:

Constant Field Values

MINIMUM_SAMPLE_SIZE

public static final int MINIMUM_SAMPLE_SIZE

See Also:

Constant Field Values

Constructor Detail

ColumnEncoderBin

public ColumnEncoderBin()

ColumnEncoderBin

public ColumnEncoderBin(int colID,
                        int numBin,
                        ColumnEncoderBin.BinMethod binMethod)

ColumnEncoderBin

public ColumnEncoderBin(int colID,
                        int numBin,
                        double[] binMins,
                        double[] binMaxs)

Method Detail

getNumBin

public int getNumBin()

getColMins

public double getColMins()

getColMaxs

public double getColMaxs()

getBinMins

public double[] getBinMins()

getBinMaxs

public double[] getBinMaxs()

getBinMethod

public ColumnEncoderBin.BinMethod getBinMethod()

setBinMethod

public void setBinMethod(String method)

build

public void build(CacheBlock<?> in)

Description copied from interface: Encoder

Build the transform meta data for the given block input. This call modifies and keeps meta data as encoder state.

Parameters:

in - input frame block

build

public void build(CacheBlock<?> in,
                  double[] equiHeightMaxs)

Overrides:

build in class ColumnEncoder

getBuildTask

public Callable<Object> getBuildTask(CacheBlock<?> in)

Overrides:

getBuildTask in class ColumnEncoder

getPartialBuildTask

public Callable<Object> getPartialBuildTask(CacheBlock<?> in,
                                            int startRow,
                                            int blockSize,
                                            HashMap<Integer,Object> ret)

Overrides:

getPartialBuildTask in class ColumnEncoder

getPartialMergeBuildTask

public Callable<Object> getPartialMergeBuildTask(HashMap<Integer,?> ret)

Overrides:

getPartialMergeBuildTask in class ColumnEncoder

computeBins

public void computeBins(double min,
                        double max)

prepareBuildPartial

public void prepareBuildPartial()

Description copied from class: ColumnEncoder

Allocates internal data structures for partial build.

Specified by:

prepareBuildPartial in interface Encoder

Overrides:

prepareBuildPartial in class ColumnEncoder

buildPartial

public void buildPartial(FrameBlock in)

Description copied from class: ColumnEncoder

Partial build of internal data structures (e.g., in distributed spark operations).

Specified by:

buildPartial in interface Encoder

Overrides:

buildPartial in class ColumnEncoder

Parameters:

in - input frame block

mergeAt

public void mergeAt(ColumnEncoder other)

Description copied from class: ColumnEncoder

Merges another encoder, of a compatible type, in after a certain position. Resizes as necessary. ColumnEncoders are compatible with themselves and EncoderComposite is compatible with every other ColumnEncoders. MultiColumnEncoders are compatible with every encoder

Overrides:

mergeAt in class ColumnEncoder

Parameters:

other - the encoder that should be merged in

allocateMetaData

public void allocateMetaData(FrameBlock meta)

Description copied from interface: Encoder

Pre-allocate a FrameBlock for metadata collection.

Parameters:

meta - frame block

getMetaData

public FrameBlock getMetaData(FrameBlock meta)

Description copied from interface: Encoder

Construct a frame block out of the transform meta data.

Parameters:

meta - output frame block

Returns:

output frame block?

initMetaData

public void initMetaData(FrameBlock meta)

Description copied from interface: Encoder

Sets up the required meta data for a subsequent call to apply.

Parameters:

meta - frame block

writeExternal

public void writeExternal(ObjectOutput out)
                   throws IOException

Description copied from class: ColumnEncoder

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd serialization.

Specified by:

writeExternal in interface Externalizable

Overrides:

writeExternal in class ColumnEncoder

Parameters:

out - object output

Throws:

IOException - if IOException occurs

readExternal

public void readExternal(ObjectInput in)
                  throws IOException

Description copied from class: ColumnEncoder

Redirects the default java serialization via externalizable to our default hadoop writable serialization for efficient broadcast/rdd deserialization.

Specified by:

readExternal in interface Externalizable

Overrides:

readExternal in class ColumnEncoder

Parameters:

in - object input

Throws:

IOException - if IOException occur

toString

public String toString()

Overrides:

toString in class Object