Package org.apache.sysds.runtime.compress.colgroup.mapping

Class AMapToData

java.lang.Object

org.apache.sysds.runtime.compress.colgroup.mapping.AMapToData

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

MapToBit, MapToByte, MapToChar, MapToCharPByte, MapToInt, MapToZero

public abstract class AMapToData
extends Object
implements Serializable

This Class's job is to link into the dictionary entries for column groups. Column groups - DDC use this to map to map directly to the dictionary - SDC use this in collaboration with the offsets to only point to dictionary entries for non default values.

See Also:

Serialized Form

Method Summary

Modifier and Type	Method	Description
abstract AMapToData	append(AMapToData t)
abstract AMapToData	appendN(IMapToDataGroup[] d)
void	copy(AMapToData d)	Copy the values in this map into another mapping object.
abstract void	copyBit(BitSet d)
abstract void	copyInt(int[] d)	Copy the values of the given array into this.
abstract int	countRuns()	Count the number of runs inside the map.
int	countRuns(AOffset off)	Count the number of runs inside the map, but sparse with offsets.
void	decompressToRange(double[] c, int rl, int ru, int offR, double[] values)
void	decompressToRangeNoOff(double[] c, int rl, int ru, double[] values)
void	decompressToRangeOff(double[] c, int rl, int ru, int offR, double[] values)
boolean	equals(Object e)
abstract boolean	equals(AMapToData e)	Indicate if the given encoding is equivalent to this encoding
abstract void	fill(int v)	Fill the map with a given value.
int[]	getCounts()	Get the number of counts of each unique value contained in this map.
abstract int[]	getCounts(int[] counts)	Get the number of counts of each unique value contained in this map.
abstract long	getExactSizeOnDisk()	Get the size of this Mapping object on disk.
abstract int	getIndex(int n)	Get the given index back as a integer
abstract long	getInMemorySize()	Get the in memory size of this Mapping object.
int	getMax()
abstract int	getMaxPossible()	Get the maximum possible value to encode in this encoding.
abstract MapToFactory.MAP_TYPE	getType()
int	getUnique()	Get the number of unique values inside this map.
abstract int	getUpperBoundValue()	Get the maximum value that is possible to allocate inside this map.
void	lmSparseMatrixRow(SparseBlock sb, int r, DenseBlock db, IColIndex colIndexes, IDictionary dict)
void	preAggregateDDC_DDC(AMapToData tm, IDictionary td, Dictionary ret, int nCol)	PreAggregate into dictionary with two sides of DDC.
void	preAggregateDDC_RLE(int[] ptr, char[] data, IDictionary td, Dictionary ret, int nCol)
void	preAggregateDDC_SDCZ(AMapToData tm, IDictionary td, AOffset tof, Dictionary ret, int nCol)	PreAggregate into SDCZero dictionary from DDC dictionary.
void	preAggregateDDC_SDCZMultiCol(AMapToData tm, IDictionary td, AOffset tof, double[] v, int nCol)
void	preAggregateDDC_SDCZSingleCol(AMapToData tm, double[] td, AOffset tof, double[] v)
void	preAggregateDense(DenseBlock m, double[] preAV, int rl, int ru, int cl, int cu, AOffset indexes)	PreAggregate a Dense Matrix at index offsets.
void	preAggregateDense(MatrixBlock m, double[] preAV, int rl, int ru, int cl, int cu)	Pre aggregate a dense matrix m into pre, subject to only including a row segment and column segment.
void	preAggregateRLE_DDC(int[] ptr, char[] data, IDictionary td, Dictionary ret, int nCol)
void	preAggregateSDCZ_DDC(AMapToData tm, IDictionary td, AOffset of, Dictionary ret, int nCol)	PreAggregate into DDC dictionary from SDCZero dictionary.
void	preAggregateSDCZ_SDCZ(AMapToData tm, IDictionary td, AOffset tof, AOffset of, Dictionary ret, int nCol)
void	preAggregateSparse(SparseBlock sb, double[] preAV, int rl, int ru)	PreAggregate the sparseblock in the range of rows given.
void	preAggregateSparse(SparseBlock sb, double[] preAV, int rl, int ru, AOffset indexes)	PreAggregate the SparseBlock in the range of rows given.
abstract void	replace(int v, int r)	Replace v with r for all entries, NOTE! It is assumed that you call this correctly: - with two distinct values that is representable inside the given AMapToData.
abstract AMapToData	resize(int unique)	Reallocate the map, to a smaller instance if applicable.
abstract void	set(int n, int v)	Set the index to the value.
void	set(int n, Integer v)	Shortcut method to support Integer objects, not really efficient but for the purpose of reusing code.
abstract int	setAndGet(int n, int v)	Set the index to the value and get the contained value after.
void	setUnique(int nUnique)	Set number of unique values.
abstract int	size()	The size of the Mapping object, signaling how many value cells are stored in this mapping object.
abstract AMapToData	slice(int l, int u)	Slice out the range from lower to upper from this map toData.
String	toString()
void	verify()	Debugging verification that this mapping is correctly made.
abstract void	write(DataOutput out)	Serialize this object to the DataOutput given.

Methods inherited from class java.lang.Object

getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

getUnique

public final int getUnique()

Get the number of unique values inside this map.

Returns:

the unique count.

setUnique

public final void setUnique(int nUnique)

Set number of unique values. NOTE! The value should be representable inside the map. This requirement is not checked.

Parameters:

nUnique - the value to set.

getIndex

public abstract int getIndex(int n)

Get the given index back as a integer

Parameters:

n - the index to get

Returns:

the value represented in that cell as integer

set

public void set(int n,
                Integer v)

Shortcut method to support Integer objects, not really efficient but for the purpose of reusing code.

Parameters:

n - The index to set.

v - The value to set.

set

public abstract void set(int n,
                         int v)

Set the index to the value. NOTE! The value should be representable inside the map. This requirement is not checked.

Parameters:

n - index to set.

v - the value to set it to.

setAndGet

public abstract int setAndGet(int n,
                              int v)

Set the index to the value and get the contained value after.

Parameters:

n - index to set.

v - the value to set it to.

Returns:

v as encoded, note this value can be different that the one put in if the map is not able to represent the value

fill

public abstract void fill(int v)

Fill the map with a given value. NOTE! The value should be representable inside the map. This requirement is not checked.

Parameters:

v - the value to fill

getUpperBoundValue

public abstract int getUpperBoundValue()

Get the maximum value that is possible to allocate inside this map.

Returns:

The maximum value.

getInMemorySize

public abstract long getInMemorySize()

Get the in memory size of this Mapping object.

Returns:

The size in Bytes.

getExactSizeOnDisk

public abstract long getExactSizeOnDisk()

Get the size of this Mapping object on disk.

Returns:

The on disk size in Bytes.

size

public abstract int size()

The size of the Mapping object, signaling how many value cells are stored in this mapping object.

Returns:

The length of the mapping object.

write

public abstract void write(DataOutput out)
                    throws IOException

Serialize this object to the DataOutput given.

Parameters:

out - The object to serialize this object into.

Throws:

IOException - An IO exception if the Serialization fails.

replace

public abstract void replace(int v,
                             int r)

Replace v with r for all entries, NOTE! It is assumed that you call this correctly: - with two distinct values that is representable inside the given AMapToData.

Parameters:

v - The value to replace

r - The value to put instead

getType

public abstract MapToFactory.MAP_TYPE getType()

preAggregateDense

public final void preAggregateDense(MatrixBlock m,
                                    double[] preAV,
                                    int rl,
                                    int ru,
                                    int cl,
                                    int cu)

Pre aggregate a dense matrix m into pre, subject to only including a row segment and column segment.

Parameters:

m - The dense matrix values to preaggregate

preAV - The preAggregate double array populate with the summed values of m

rl - The row start in m

ru - The row end in m

cl - The column start in m

cu - The column end in m

preAggregateDense

public final void preAggregateDense(DenseBlock m,
                                    double[] preAV,
                                    int rl,
                                    int ru,
                                    int cl,
                                    int cu,
                                    AOffset indexes)

PreAggregate a Dense Matrix at index offsets.

Parameters:

m - The DenseBlock to preAggregate

preAV - The target double array to put the preAggregate into

rl - The row to start at

ru - The row to end at (not inclusive)

cl - The column in m to start from

cu - The column in m to end at (not inclusive)

indexes - The Offset Indexes to iterate through

preAggregateSparse

public final void preAggregateSparse(SparseBlock sb,
                                     double[] preAV,
                                     int rl,
                                     int ru,
                                     AOffset indexes)

PreAggregate the SparseBlock in the range of rows given.

Parameters:

sb - The SparseBlock to preAggregate

preAV - The target double array to put the preAggregate into

rl - The row to start at

ru - The row to end at (not inclusive)

indexes - The Offset Indexes to iterate through

preAggregateSparse

public final void preAggregateSparse(SparseBlock sb,
                                     double[] preAV,
                                     int rl,
                                     int ru)

PreAggregate the sparseblock in the range of rows given.

Parameters:

sb - Sparse block to preAggregate from

preAV - Pre aggregate target

rl - row index in sb

ru - upper row index in sp (not inclusive)

getCounts

public final int[] getCounts()

Get the number of counts of each unique value contained in this map. Note that in the case the mapping is shorter than number of rows the counts sum to the number of mapped values not the number of rows.

Returns:

The counts

getCounts

public abstract int[] getCounts(int[] counts)

Get the number of counts of each unique value contained in this map. Note that in the case the mapping is shorter than number of rows the counts sum to the number of mapped values not the number of rows.

Parameters:

counts - The object to return.

Returns:

The counts

preAggregateDDC_DDC

public final void preAggregateDDC_DDC(AMapToData tm,
                                      IDictionary td,
                                      Dictionary ret,
                                      int nCol)

PreAggregate into dictionary with two sides of DDC.

Parameters:

tm - Map of other side

td - Dictionary to take values from (other side dictionary)

ret - The output dictionary to aggregate into

nCol - The number of columns

preAggregateDDC_SDCZ

public final void preAggregateDDC_SDCZ(AMapToData tm,
                                       IDictionary td,
                                       AOffset tof,
                                       Dictionary ret,
                                       int nCol)

PreAggregate into SDCZero dictionary from DDC dictionary.

Parameters:

tm - Map of other side

td - Dictionary to take values from (other side dictionary)

tof - The offset index structure of the SDC side

ret - The output dictionary to aggregate into

nCol - The number of columns in output and td dictionary

preAggregateDDC_SDCZSingleCol

public void preAggregateDDC_SDCZSingleCol(AMapToData tm,
                                          double[] td,
                                          AOffset tof,
                                          double[] v)

preAggregateDDC_SDCZMultiCol

public void preAggregateDDC_SDCZMultiCol(AMapToData tm,
                                         IDictionary td,
                                         AOffset tof,
                                         double[] v,
                                         int nCol)

preAggregateSDCZ_DDC

public final void preAggregateSDCZ_DDC(AMapToData tm,
                                       IDictionary td,
                                       AOffset of,
                                       Dictionary ret,
                                       int nCol)

PreAggregate into DDC dictionary from SDCZero dictionary.

Parameters:

tm - Map of other side

td - Dictionary to take values from (other side dictionary)

of - Offsets of the SDC to look into DDC

ret - The output dictionary to aggregate into

nCol - The number of columns in output and td dictionary

preAggregateSDCZ_SDCZ

public final void preAggregateSDCZ_SDCZ(AMapToData tm,
                                        IDictionary td,
                                        AOffset tof,
                                        AOffset of,
                                        Dictionary ret,
                                        int nCol)

preAggregateRLE_DDC

public void preAggregateRLE_DDC(int[] ptr,
                                char[] data,
                                IDictionary td,
                                Dictionary ret,
                                int nCol)

preAggregateDDC_RLE

public void preAggregateDDC_RLE(int[] ptr,
                                char[] data,
                                IDictionary td,
                                Dictionary ret,
                                int nCol)

copy

public void copy(AMapToData d)

Copy the values in this map into another mapping object. NOTE! All contained vales should be representable inside the map given. This requirement is not checked.

Parameters:

d - Map to copy all values into.

copyInt

public abstract void copyInt(int[] d)

Copy the values of the given array into this. Note that this operation stops at the length of this AMapToData Therefore the given d length can not be longer than this size.

Parameters:

d - The array to copy

copyBit

public abstract void copyBit(BitSet d)

getMax

public int getMax()

getMaxPossible

public abstract int getMaxPossible()

Get the maximum possible value to encode in this encoding. For instance in a bit you can encode 2 values

Returns:

The maximum number of distinct values to encode

resize

public abstract AMapToData resize(int unique)

Reallocate the map, to a smaller instance if applicable. Note it does not change the length of the array, just the datatype. Note that it returns the input if the input is the smallest representation that fits, otherwise it will return something that is smaller.

Parameters:

unique - The number of tuples that should be supported in the resulting map

Returns:

The returned hopefully reduced map.

countRuns

public abstract int countRuns()

Count the number of runs inside the map.

Returns:

The number of runs

countRuns

public int countRuns(AOffset off)

Count the number of runs inside the map, but sparse with offsets.

Parameters:

off - The sparse offsets to consider counting the runs from.

Returns:

count of runs.

slice

public abstract AMapToData slice(int l,
                                 int u)

Slice out the range from lower to upper from this map toData.

Parameters:

l - Low value to slice from

u - high value to slice to (not inclusive)

Returns:

A new map containing only the values from the range.

append

public abstract AMapToData append(AMapToData t)

appendN

public abstract AMapToData appendN(IMapToDataGroup[] d)

equals

public boolean equals(Object e)

Overrides:

equals in class Object

equals

public abstract boolean equals(AMapToData e)

Indicate if the given encoding is equivalent to this encoding

Parameters:

e - The other encoding to be compared with this

Returns:

If the encoding is equivalent

verify

public void verify()

Debugging verification that this mapping is correctly made.

lmSparseMatrixRow

public void lmSparseMatrixRow(SparseBlock sb,
                              int r,
                              DenseBlock db,
                              IColIndex colIndexes,
                              IDictionary dict)

decompressToRange

public void decompressToRange(double[] c,
                              int rl,
                              int ru,
                              int offR,
                              double[] values)

decompressToRangeOff

public void decompressToRangeOff(double[] c,
                                 int rl,
                                 int ru,
                                 int offR,
                                 double[] values)

decompressToRangeNoOff

public void decompressToRangeNoOff(double[] c,
                                   int rl,
                                   int ru,
                                   double[] values)

toString

public String toString()

Overrides:

toString in class Object