java.lang.Object
- org.apache.sysds.hops.Hop

All Implemented Interfaces:

ParseInfo

Direct Known Subclasses:

DataOp, IndexingOp, LeftIndexingOp, LiteralOp, MultiThreadedHop, NaryOp
```
public abstract class Hop
extends Object
implements ParseInfo
```
Hop is a High level operator, that is the first intermediate representation compiled from the definitions supplied in DML.

Field Summary

Fields
Modifier and Type Field Description

int _beginColumn

int _beginLine

int _endColumn

int _endLine

String _filename

String _text

static long CPThreshold

Constructor Summary

Constructors
Constructor Description

Hop(String l, Types.DataType dt, Types.ValueType vt)

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method	Description
`void`	`activatePrefetch()`	Activate prefetch of HOP.
`void`	`addAllInputs(List<Hop> list)`
`void`	`addInput(Hop h)`
`abstract boolean`	`allowsAllExecTypes()`
`void`	`checkAndSetForcedPlatform()`
`void`	`checkAndSetInvalidCPDimsAndSize()`
`abstract void`	`checkArity()`	Check whether this Hop has a correct number of inputs.
`void`	`clearMemEstimate()`
`abstract Object`	`clone()`
`boolean`	`colsKnown()`
`abstract boolean`	`compare(Hop that)`
`long`	`compressedSize()`
`double`	`computeBoundsInformation(Hop input)`
`static double`	`computeBoundsInformation(Hop input, LocalVariableMap vars)`
`static double`	`computeBoundsInformation(Hop input, LocalVariableMap vars, Map<Long,Double> memo)`
`void`	`computeMemEstimate(MemoTable memo)`	Computes the estimate of memory required to store the input/output of this hop in memory.
`static long`	`computeSizeInformation(Hop input)`
`long`	`computeSizeInformation(Hop input, LocalVariableMap vars)`
`long`	`computeSizeInformation(Hop input, LocalVariableMap vars, Map<Long,Long> memo)`
`void`	`constructAndSetLopsDataFlowProperties()`
`abstract Lop`	`constructLops()`
`static Lop`	`createOffsetLop(Hop hop, boolean repCols)`
`void`	`deactivatePrefetch()`
`boolean`	`dimsKnown()`
`boolean`	`dimsKnown(boolean includeNnz)`
`boolean`	`dimsKnownAny()`
`int`	`getBeginColumn()`
`int`	`getBeginLine()`
`int`	`getBlocksize()`
`long`	`getCompressedSize()`
`DataCharacteristics`	`getDataCharacteristics()`
`Types.DataType`	`getDataType()`
`long`	`getDim(int i)`
`long`	`getDim1()`	Get the number of rows in the Hop.
`long`	`getDim2()`	Get the number of columns in the Hop.
`int`	`getEndColumn()`
`int`	`getEndLine()`
`Types.ExecType`	`getExecType()`
`FEDInstruction.FederatedOutput`	`getFederatedOutput()`
`String`	`getFilename()`
`Types.ExecType`	`getForcedExecType()`
`long`	`getHopID()`
`List<Hop>`	`getInput()`
`Hop`	`getInput(int ix)`
`double`	`getInputMemEstimate()`	Get the memory estimate of inputs as the sum of input estimates in bytes.
`double`	`getInputMemEstimate(double injectedDefault)`	Get the memory estimate of inputs as the sum of input estimates in bytes.
`double`	`getInputOutputSize(Collection<String> exclVars)`
`double`	`getIntermediateMemEstimate()`
`long`	`getLength()`
`Lop`	`getLops()`
`double`	`getMemEstimate()`	NOTES: * Purpose: Whenever the output dimensions / sparsity of a hop are unknown, this hop should store its worst-case output statistics (if known) in that table.
`String`	`getName()`
`long`	`getNnz()`
`abstract String`	`getOpString()`
`double`	`getOutputMemEstimate()`	Output memory estimate in bytes.
`double`	`getOutputMemEstimate(double injectedDefault)`	Output memory estimate in bytes with negative memory estimates replaced by the injected default.
`List<Hop>`	`getParent()`
`double`	`getSparsity()`
`double`	`getSpBroadcastSize()`
`String`	`getText()`
`MatrixObject.UpdateType`	`getUpdateType()`
`Types.ValueType`	`getValueType()`
`boolean`	`hasCompressedInput()`
`boolean`	`hasFederatedOutput()`
`boolean`	`hasLocalOutput()`
`boolean`	`hasMatrixInputWithDifferentBlocksizes()`
`boolean`	`hasValidCPDimsAndSize()`
`boolean`	`isCompressedOutput()`
`boolean`	`isFederated()`	Checks if ExecType is federated.
`boolean`	`isFederatedDataOp()`	Checks if the hop is a DataOp with federated data.
`abstract boolean`	`isGPUEnabled()`	In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop.
`boolean`	`isMatrix()`
`boolean`	`isMemEstimated()`
`boolean`	`isOutputEmptyBlocks()`
`boolean`	`isRequiredDecompression()`
`boolean`	`isScalar()`
`boolean`	`isTransposeSafe()`	Defines if this operation is transpose-safe, which means that the result of op(input) is equivalent to op(t(input)).
`boolean`	`isVisited()`
`boolean`	`prefetchActivated()`	Checks if prefetch is activated for this hop.
`String`	`printErrorLocation()`
`void`	`refreshColsParameterInformation(Hop input, LocalVariableMap vars)`
`void`	`refreshColsParameterInformation(Hop input, LocalVariableMap vars, Map<Long,Long> memo)`
`void`	`refreshMemEstimates(MemoTable memo)`	Recursively computes memory estimates for all the Hops in the DAG rooted at the current hop pointed by `this`.
`void`	`refreshRowsParameterInformation(Hop input, LocalVariableMap vars)`
`void`	`refreshRowsParameterInformation(Hop input, LocalVariableMap vars, Map<Long,Long> memo)`
`abstract void`	`refreshSizeInformation()`	Update the output size information for this hop.
`boolean`	`requiresCheckpoint()`
`boolean`	`requiresCompression()`
`boolean`	`requiresLineageCaching()`
`boolean`	`requiresReblock()`
`boolean`	`requiresRecompile()`	Indicates if dynamic recompilation is required for this hop.
`void`	`resetExecType()`
`static void`	`resetRecompilationFlag(ArrayList<Hop> hops, Types.ExecType et, Recompiler.ResetType reset)`
`static void`	`resetRecompilationFlag(Hop hops, Types.ExecType et, Recompiler.ResetType reset)`
`Hop`	`resetVisitStatus()`
`static void`	`resetVisitStatus(List<Hop> hops)`
`static void`	`resetVisitStatus(List<Hop> hops, boolean force)`
`void`	`resetVisitStatusForced(HashSet<Long> memo)`
`boolean`	`rowsKnown()`
`void`	`setBeginColumn(int passed)`
`void`	`setBeginLine(int passed)`
`void`	`setBlocksize(int blen)`
`void`	`setCompressedOutput(boolean value)`
`void`	`setCompressedSize(long size)`
`void`	`setDataType(Types.DataType dt)`
`void`	`setDim(int i, long dim)`
`void`	`setDim1(long dim1)`
`void`	`setDim2(long dim2)`
`void`	`setEndColumn(int passed)`
`void`	`setEndLine(int passed)`
`void`	`setExecType(Types.ExecType execType)`
`void`	`setFederatedOutput(FEDInstruction.FederatedOutput federatedOutput)`
`void`	`setFilename(String passed)`
`void`	`setForcedExecType(Types.ExecType etype)`
`void`	`setLops(Lop lops)`
`void`	`setMemEstimate(double mem)`	Sets memory estimate in bytes
`void`	`setName(String _name)`
`void`	`setNnz(long nnz)`
`void`	`setOutputEmptyBlocks(boolean flag)`
`void`	`setParseInfo(ParseInfo parseInfo)`	Set parse information.
`void`	`setRequiresCheckpoint(boolean flag)`
`void`	`setRequiresCompression()`
`void`	`setRequiresCompression(AWTreeNode node)`
`void`	`setRequiresDeCompression()`
`void`	`setRequiresLineageCaching(boolean flag)`
`void`	`setRequiresReblock(boolean flag)`
`void`	`setRequiresRecompile()`	Marks the hop for dynamic recompilation.
`void`	`setText(String text)`
`void`	`setUpdateType(MatrixObject.UpdateType update)`
`void`	`setValueType(Types.ValueType vt)`
`void`	`setVisited()`
`void`	`setVisited(boolean flag)`
`boolean`	`someInputFederated()`
`String`	`toString()`
`void`	`updateLopFedOut(Lop lop)`
`void`	`updateLopFedOut(Lop lop, Types.ExecType execType, FEDInstruction.FederatedOutput fedOut)`

Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CPThreshold
```
public static final long CPThreshold
```
See Also:

Constant Field Values

_beginLine
```
public int _beginLine
```

_beginColumn
```
public int _beginColumn
```

_endLine
```
public int _endLine
```

_endColumn
```
public int _endColumn
```

_filename
```
public String _filename
```

_text
```
public String _text
```

Constructor Detail

Hop

public Hop(String l,
           Types.DataType dt,
           Types.ValueType vt)

Method Detail

getHopID
```
public long getHopID()
```

checkArity
```
public abstract void checkArity()
```
Check whether this Hop has a correct number of inputs. (Some Hops can have a variable number of inputs, such as DataOp, DataGenOp, ParameterizedBuiltinOp, ReorgOp, TernaryOp, QuaternaryOp, MultipleOp, DnnOp, and SpoofFusedOp.) Parameterized Hops (such as DataOp) can check that the number of parameters matches the number of inputs.

getExecType
```
public Types.ExecType getExecType()
```

setExecType

public void setExecType(Types.ExecType execType)

setFederatedOutput

public void setFederatedOutput(FEDInstruction.FederatedOutput federatedOutput)

activatePrefetch
```
public void activatePrefetch()
```
Activate prefetch of HOP.

deactivatePrefetch
```
public void deactivatePrefetch()
```

prefetchActivated
```
public boolean prefetchActivated()
```
Checks if prefetch is activated for this hop.

Returns:

true if prefetch is activated

resetExecType
```
public void resetExecType()
```

getForcedExecType

public Types.ExecType getForcedExecType()

setForcedExecType

public void setForcedExecType(Types.ExecType etype)

allowsAllExecTypes

public abstract boolean allowsAllExecTypes()

isTransposeSafe
```
public boolean isTransposeSafe()
```
Defines if this operation is transpose-safe, which means that the result of op(input) is equivalent to op(t(input)). Usually, this applies to aggregate operations with fixed output dimension. Finally, this information is very useful in order to safely optimize the plan for sparse vectors, which otherwise would be (currently) always represented dense.

Returns:

always returns false

checkAndSetForcedPlatform

public void checkAndSetForcedPlatform()

checkAndSetInvalidCPDimsAndSize

public void checkAndSetInvalidCPDimsAndSize()

hasValidCPDimsAndSize

public boolean hasValidCPDimsAndSize()

hasMatrixInputWithDifferentBlocksizes

public boolean hasMatrixInputWithDifferentBlocksizes()

setRequiresReblock

public void setRequiresReblock(boolean flag)

requiresReblock
```
public boolean requiresReblock()
```

setRequiresCheckpoint

public void setRequiresCheckpoint(boolean flag)

requiresCheckpoint
```
public boolean requiresCheckpoint()
```

setRequiresCompression
```
public void setRequiresCompression()
```

setRequiresCompression

public void setRequiresCompression(AWTreeNode node)

setRequiresDeCompression

public void setRequiresDeCompression()

isRequiredDecompression

public boolean isRequiredDecompression()

requiresCompression
```
public boolean requiresCompression()
```

setCompressedOutput

public void setCompressedOutput(boolean value)

setCompressedSize

public void setCompressedSize(long size)

getCompressedSize
```
public long getCompressedSize()
```

isCompressedOutput
```
public boolean isCompressedOutput()
```

hasCompressedInput
```
public boolean hasCompressedInput()
```

compressedSize
```
public long compressedSize()
```

setRequiresLineageCaching

public void setRequiresLineageCaching(boolean flag)

requiresLineageCaching

public boolean requiresLineageCaching()

updateLopFedOut

public void updateLopFedOut(Lop lop)

updateLopFedOut

public void updateLopFedOut(Lop lop,
                            Types.ExecType execType,
                            FEDInstruction.FederatedOutput fedOut)

constructAndSetLopsDataFlowProperties

public void constructAndSetLopsDataFlowProperties()

createOffsetLop

public static Lop createOffsetLop(Hop hop,
                                  boolean repCols)

setOutputEmptyBlocks

public void setOutputEmptyBlocks(boolean flag)

isOutputEmptyBlocks
```
public boolean isOutputEmptyBlocks()
```

getInputOutputSize

public double getInputOutputSize(Collection<String> exclVars)

getMemEstimate
```
public double getMemEstimate()
```
NOTES: * Purpose: Whenever the output dimensions / sparsity of a hop are unknown, this hop should store its worst-case output statistics (if known) in that table. * Invocation: Intended to be called for ALL root nodes of one Hops DAG with the same (initially empty) memo table.

Returns:

memory estimate in bytes

setMemEstimate
```
public void setMemEstimate(double mem)
```
Sets memory estimate in bytes

Parameters:

mem - memory estimate

clearMemEstimate
```
public void clearMemEstimate()
```

isMemEstimated
```
public boolean isMemEstimated()
```

getInputMemEstimate
```
public double getInputMemEstimate()
```
Get the memory estimate of inputs as the sum of input estimates in bytes.

Returns:

input memory estimate in bytes

getInputMemEstimate
```
public double getInputMemEstimate(double injectedDefault)
```
Get the memory estimate of inputs as the sum of input estimates in bytes.

Parameters:

injectedDefault - default memory estimate (bytes) used when the memory estimate of the input is negative

Returns:

input memory estimate in bytes

getOutputMemEstimate
```
public double getOutputMemEstimate()
```
Output memory estimate in bytes.

Returns:

output memory estimate in bytes

getOutputMemEstimate
```
public double getOutputMemEstimate(double injectedDefault)
```
Output memory estimate in bytes with negative memory estimates replaced by the injected default. The injected default represents the memory estimate per output cell, hence it is multiplied by the estimated dimensions of the output of the hop.

Parameters:

injectedDefault - memory estimate to be returned in case the memory estimate defaults to a negative number

Returns:

output memory estimate in bytes

getIntermediateMemEstimate

public double getIntermediateMemEstimate()

getSpBroadcastSize
```
public double getSpBroadcastSize()
```

computeMemEstimate
```
public void computeMemEstimate(MemoTable memo)
```
Computes the estimate of memory required to store the input/output of this hop in memory. This is the default implementation (orchestration of hop-specific implementation) that should suffice for most hops. If a hop requires more control, this method should be overwritten with awareness of (1) output estimates, and (2) propagation of worst-case matrix characteristics (dimensions, sparsity). TODO remove memo table and, on constructor refresh, inference in refresh, single compute mem, maybe general computeMemEstimate, flags to indicate if estimate or not.

Parameters:

memo - memory table

refreshMemEstimates
```
public void refreshMemEstimates(MemoTable memo)
```
Recursively computes memory estimates for all the Hops in the DAG rooted at the current hop pointed by this.

Parameters:

memo - memory table

isFederated
```
public boolean isFederated()
```
Checks if ExecType is federated.

Returns:

true if ExecType is federated

someInputFederated
```
public boolean someInputFederated()
```

isFederatedDataOp
```
public boolean isFederatedDataOp()
```
Checks if the hop is a DataOp with federated data.

Returns:

true if hop is a federated DataOp

getParent
```
public List<Hop> getParent()
```

getInput
```
public List<Hop> getInput()
```

getInput
```
public Hop getInput(int ix)
```

addInput
```
public void addInput(Hop h)
```

addAllInputs

public void addAllInputs(List<Hop> list)

getBlocksize
```
public int getBlocksize()
```

setBlocksize

public void setBlocksize(int blen)

setNnz
```
public void setNnz(long nnz)
```

getNnz
```
public long getNnz()
```

getFederatedOutput

public FEDInstruction.FederatedOutput getFederatedOutput()

hasFederatedOutput
```
public boolean hasFederatedOutput()
```

hasLocalOutput
```
public boolean hasLocalOutput()
```

setUpdateType

public void setUpdateType(MatrixObject.UpdateType update)

getUpdateType

public MatrixObject.UpdateType getUpdateType()

constructLops
```
public abstract Lop constructLops()
```

getOpString
```
public abstract String getOpString()
```

toString
```
public final String toString()
```
Overrides:

toString in class Object

isGPUEnabled
```
public abstract boolean isGPUEnabled()
```
In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop. Please see findExecTypeByMemEstimate for more detail. This method is necessary because not all operator are supported efficiently on GPU (for example: operations on frames and scalar as well as operations such as table).

Returns:

true if the Hop is eligible for GPU Exectype.

dimsKnown
```
public boolean dimsKnown()
```

dimsKnown

public boolean dimsKnown(boolean includeNnz)

dimsKnownAny
```
public boolean dimsKnownAny()
```

rowsKnown
```
public boolean rowsKnown()
```

colsKnown
```
public boolean colsKnown()
```

resetVisitStatus

public static void resetVisitStatus(List<Hop> hops)

resetVisitStatus

public static void resetVisitStatus(List<Hop> hops,
                                    boolean force)

resetVisitStatus
```
public Hop resetVisitStatus()
```

resetVisitStatusForced

public void resetVisitStatusForced(HashSet<Long> memo)

resetRecompilationFlag

public static void resetRecompilationFlag(ArrayList<Hop> hops,
                                          Types.ExecType et,
                                          Recompiler.ResetType reset)

resetRecompilationFlag

public static void resetRecompilationFlag(Hop hops,
                                          Types.ExecType et,
                                          Recompiler.ResetType reset)

getDim1
```
public long getDim1()
```
Get the number of rows in the Hop.

Returns:

a long.

setDim1
```
public void setDim1(long dim1)
```

getDim2
```
public long getDim2()
```
Get the number of columns in the Hop.

Returns:

a long.

setDim2
```
public void setDim2(long dim2)
```

getDim
```
public long getDim(int i)
```

setDim

public void setDim(int i,
                   long dim)

getLength
```
public long getLength()
```

getSparsity
```
public double getSparsity()
```

getDataCharacteristics

public DataCharacteristics getDataCharacteristics()

getLops
```
public Lop getLops()
```

setLops
```
public void setLops(Lop lops)
```

isVisited
```
public boolean isVisited()
```

getDataType
```
public Types.DataType getDataType()
```

setDataType

public void setDataType(Types.DataType dt)

isScalar
```
public boolean isScalar()
```

isMatrix
```
public boolean isMatrix()
```

setVisited
```
public void setVisited()
```

setVisited

public void setVisited(boolean flag)

setName
```
public void setName(String _name)
```

getName
```
public String getName()
```

getValueType
```
public Types.ValueType getValueType()
```

setValueType

public void setValueType(Types.ValueType vt)

requiresRecompile
```
public boolean requiresRecompile()
```
Indicates if dynamic recompilation is required for this hop.

Returns:

true if dynamic recompilation required

setRequiresRecompile
```
public void setRequiresRecompile()
```
Marks the hop for dynamic recompilation.

refreshSizeInformation
```
public abstract void refreshSizeInformation()
```
Update the output size information for this hop.

computeSizeInformation

public static long computeSizeInformation(Hop input)

refreshRowsParameterInformation

public void refreshRowsParameterInformation(Hop input,
                                            LocalVariableMap vars)

refreshRowsParameterInformation

public void refreshRowsParameterInformation(Hop input,
                                            LocalVariableMap vars,
                                            Map<Long,Long> memo)

refreshColsParameterInformation

public void refreshColsParameterInformation(Hop input,
                                            LocalVariableMap vars)

refreshColsParameterInformation

public void refreshColsParameterInformation(Hop input,
                                            LocalVariableMap vars,
                                            Map<Long,Long> memo)

computeSizeInformation

public long computeSizeInformation(Hop input,
                                   LocalVariableMap vars)

computeSizeInformation

public long computeSizeInformation(Hop input,
                                   LocalVariableMap vars,
                                   Map<Long,Long> memo)

computeBoundsInformation

public double computeBoundsInformation(Hop input)

computeBoundsInformation

public static double computeBoundsInformation(Hop input,
                                              LocalVariableMap vars)

computeBoundsInformation

public static double computeBoundsInformation(Hop input,
                                              LocalVariableMap vars,
                                              Map<Long,Double> memo)

clone

public abstract Object clone()
                      throws CloneNotSupportedException

Throws:: CloneNotSupportedException

compare

public abstract boolean compare(Hop that)

setBeginLine
```
public void setBeginLine(int passed)
```
Specified by:

setBeginLine in interface ParseInfo

setBeginColumn
```
public void setBeginColumn(int passed)
```
Specified by:

setBeginColumn in interface ParseInfo

setEndLine
```
public void setEndLine(int passed)
```
Specified by:

setEndLine in interface ParseInfo

setEndColumn
```
public void setEndColumn(int passed)
```
Specified by:

setEndColumn in interface ParseInfo

setFilename
```
public void setFilename(String passed)
```
Specified by:

setFilename in interface ParseInfo

setText
```
public void setText(String text)
```
Specified by:

setText in interface ParseInfo

getBeginLine
```
public int getBeginLine()
```
Specified by:

getBeginLine in interface ParseInfo

getBeginColumn
```
public int getBeginColumn()
```
Specified by:

getBeginColumn in interface ParseInfo

getEndLine
```
public int getEndLine()
```
Specified by:

getEndLine in interface ParseInfo

getEndColumn
```
public int getEndColumn()
```
Specified by:

getEndColumn in interface ParseInfo

getFilename
```
public String getFilename()
```
Specified by:

getFilename in interface ParseInfo

getText
```
public String getText()
```
Specified by:

getText in interface ParseInfo

printErrorLocation
```
public String printErrorLocation()
```

setParseInfo
```
public void setParseInfo(ParseInfo parseInfo)
```
Set parse information.

Parameters:

parseInfo - parse information, such as beginning line position, beginning column position, ending line position, ending column position, text, and filename

Modifier and Type	Field	Description
`int`	`_beginColumn`
`int`	`_beginLine`
`int`	`_endColumn`
`int`	`_endLine`
`String`	`_filename`
`String`	`_text`
`static long`	`CPThreshold`

Class Hop

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

CPThreshold

_beginLine

_beginColumn

_endLine

_endColumn

_filename

_text

Constructor Detail

Hop

Method Detail

getHopID

checkArity

getExecType

setExecType

setFederatedOutput

activatePrefetch

deactivatePrefetch

prefetchActivated

resetExecType

getForcedExecType

setForcedExecType

allowsAllExecTypes

isTransposeSafe

checkAndSetForcedPlatform

checkAndSetInvalidCPDimsAndSize

hasValidCPDimsAndSize

hasMatrixInputWithDifferentBlocksizes

setRequiresReblock

requiresReblock

setRequiresCheckpoint

requiresCheckpoint

setRequiresCompression

setRequiresCompression

setRequiresDeCompression

isRequiredDecompression

requiresCompression

setCompressedOutput

setCompressedSize

getCompressedSize

isCompressedOutput

hasCompressedInput

compressedSize

setRequiresLineageCaching

requiresLineageCaching

updateLopFedOut

updateLopFedOut

constructAndSetLopsDataFlowProperties

createOffsetLop

setOutputEmptyBlocks

isOutputEmptyBlocks

getInputOutputSize

getMemEstimate

setMemEstimate

clearMemEstimate

isMemEstimated

getInputMemEstimate

getInputMemEstimate

getOutputMemEstimate

getOutputMemEstimate

getIntermediateMemEstimate

getSpBroadcastSize

computeMemEstimate

refreshMemEstimates

isFederated

someInputFederated

isFederatedDataOp

getParent

getInput

getInput

addInput

addAllInputs

getBlocksize

setBlocksize

setNnz