Package org.apache.sysds.runtime.transform.tokenize.applier

Class TokenizerApplierHash

java.lang.Object

org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplierHash

All Implemented Interfaces:

Serializable

public class TokenizerApplierHash
extends TokenizerApplier

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
int	num_features

Fields inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

PADDING_STRING

Constructor Summary

Constructors

Constructor	Description
TokenizerApplierHash(int numIdCols, int maxTokens, boolean wideFormat, boolean applyPadding, org.apache.wink.json4j.JSONObject params)

Method Summary

Modifier and Type	Method	Description
void	allocateInternalMeta(int numDocuments)
int	applyInternalRepresentation(DocumentRepresentation[] internalRepresentation, FrameBlock out, int inputRowStart, int blk)
void	build(DocumentRepresentation[] internalRepresentation, int inputRowStart, int blk)
int	getNumRows(DocumentRepresentation[] internalRepresentation)
Types.ValueType[]	getOutSchema()

Methods inherited from class org.apache.sysds.runtime.transform.tokenize.applier.TokenizerApplier

applyInternalRepresentation, getApplyTasks, getBuildTasks, getMaxNumRows, getMaxTokens, getNumCols, getOutputRow, getOutputRow, hasPadding, isWideFormat

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

num_features

public int num_features

Constructor Detail

TokenizerApplierHash

public TokenizerApplierHash(int numIdCols,
                            int maxTokens,
                            boolean wideFormat,
                            boolean applyPadding,
                            org.apache.wink.json4j.JSONObject params)
                     throws org.apache.wink.json4j.JSONException

Throws:

org.apache.wink.json4j.JSONException

Method Detail

getNumRows

public int getNumRows(DocumentRepresentation[] internalRepresentation)

Specified by:

getNumRows in class TokenizerApplier

allocateInternalMeta

public void allocateInternalMeta(int numDocuments)

Overrides:

allocateInternalMeta in class TokenizerApplier

build

public void build(DocumentRepresentation[] internalRepresentation,
                  int inputRowStart,
                  int blk)

Overrides:

build in class TokenizerApplier

applyInternalRepresentation

public int applyInternalRepresentation(DocumentRepresentation[] internalRepresentation,
                                       FrameBlock out,
                                       int inputRowStart,
                                       int blk)

getOutSchema

public Types.ValueType[] getOutSchema()

Specified by:

getOutSchema in class TokenizerApplier