Package org.apache.sysds.hops

Class ParameterizedBuiltinOp

java.lang.Object

org.apache.sysds.hops.Hop

org.apache.sysds.hops.MultiThreadedHop

org.apache.sysds.hops.ParameterizedBuiltinOp

All Implemented Interfaces:

ParseInfo

public class ParameterizedBuiltinOp
extends MultiThreadedHop

Defines the HOP for calling an internal function (with custom parameters) from a DML script.

Field Summary

Fields

Modifier and Type	Field	Description
static boolean	FORCE_DIST_RM_EMPTY

Fields inherited from class org.apache.sysds.hops.Hop

_beginColumn, _beginLine, _endColumn, _endLine, _filename, _text, CPThreshold

Constructor Summary

Constructors

Constructor	Description
ParameterizedBuiltinOp(String l, Types.DataType dt, Types.ValueType vt, Types.ParamBuiltinOp op, LinkedHashMap<String,Hop> inputParameters)	Creates a new HOP for a function call

Method Summary

Modifier and Type	Method	Description
boolean	allowsAllExecTypes()
void	checkArity()	Check whether this Hop has a correct number of inputs.
Object	clone()
boolean	compare(Hop that)
void	computeMemEstimate(MemoTable memo)	Computes the estimate of memory required to store the input/output of this hop in memory.
Lop	constructLops()
Types.ParamBuiltinOp	getOp()
String	getOpString()
Hop	getParameterHop(String name)
HashMap<String,Integer>	getParamIndexMap()
List<FunctionOp>	getParamservPseudoFunctionCalls()
Hop	getTargetHop()
boolean	isCountFunction()
boolean	isGPUEnabled()	In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop.
boolean	isKnownNGroups()
boolean	isMultiThreadedOpType()
boolean	isTargetDiagInput()
boolean	isTransposeSafe()	Defines if this operation is transpose-safe, which means that the result of op(input) is equivalent to op(t(input)).
void	refreshSizeInformation()	Update the output size information for this hop.
void	setOutputPermutationMatrix(boolean flag)

Methods inherited from class org.apache.sysds.hops.MultiThreadedHop

getMaxNumThreads, setMaxNumThreads

Methods inherited from class org.apache.sysds.hops.Hop

activatePrefetch, addAllInputs, addInput, checkAndSetForcedPlatform, checkAndSetInvalidCPDimsAndSize, clearMemEstimate, colsKnown, compressedSize, computeBoundsInformation, computeBoundsInformation, computeBoundsInformation, computeSizeInformation, computeSizeInformation, computeSizeInformation, constructAndSetLopsDataFlowProperties, createOffsetLop, deactivatePrefetch, dimsKnown, dimsKnown, dimsKnownAny, getBeginColumn, getBeginLine, getBlocksize, getCompressedSize, getDataCharacteristics, getDataType, getDim, getDim1, getDim2, getEndColumn, getEndLine, getExecType, getFederatedOutput, getFilename, getForcedExecType, getHopID, getInput, getInput, getInputMemEstimate, getInputMemEstimate, getInputOutputSize, getIntermediateMemEstimate, getLength, getLops, getMemEstimate, getName, getNnz, getOutputMemEstimate, getOutputMemEstimate, getParent, getSparsity, getSpBroadcastSize, getText, getUpdateType, getValueType, hasCompressedInput, hasFederatedOutput, hasLocalOutput, hasMatrixInputWithDifferentBlocksizes, hasValidCPDimsAndSize, isCompressedOutput, isFederated, isFederatedDataOp, isMatrix, isMemEstimated, isOutputEmptyBlocks, isRequiredDecompression, isScalar, isVisited, prefetchActivated, printErrorLocation, refreshColsParameterInformation, refreshColsParameterInformation, refreshMemEstimates, refreshRowsParameterInformation, refreshRowsParameterInformation, requiresCheckpoint, requiresCompression, requiresLineageCaching, requiresReblock, requiresRecompile, resetExecType, resetRecompilationFlag, resetRecompilationFlag, resetVisitStatus, resetVisitStatus, resetVisitStatus, resetVisitStatusForced, rowsKnown, setBeginColumn, setBeginLine, setBlocksize, setCompressedOutput, setCompressedSize, setDataType, setDim, setDim1, setDim2, setEndColumn, setEndLine, setExecType, setFederatedOutput, setFilename, setForcedExecType, setLops, setMemEstimate, setName, setNnz, setOutputEmptyBlocks, setParseInfo, setRequiresCheckpoint, setRequiresCompression, setRequiresCompression, setRequiresDeCompression, setRequiresLineageCaching, setRequiresReblock, setRequiresRecompile, setText, setUpdateType, setValueType, setVisited, setVisited, someInputFederated, toString, updateLopFedOut, updateLopFedOut

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

FORCE_DIST_RM_EMPTY

public static boolean FORCE_DIST_RM_EMPTY

Constructor Detail

ParameterizedBuiltinOp

public ParameterizedBuiltinOp(String l,
                              Types.DataType dt,
                              Types.ValueType vt,
                              Types.ParamBuiltinOp op,
                              LinkedHashMap<String,Hop> inputParameters)

Creates a new HOP for a function call

Parameters:

l - ?

dt - data type

vt - value type

op - the ParamBuiltinOp

inputParameters - map of input parameters

Method Detail

checkArity

public void checkArity()

Description copied from class: Hop

Check whether this Hop has a correct number of inputs. (Some Hops can have a variable number of inputs, such as DataOp, DataGenOp, ParameterizedBuiltinOp, ReorgOp, TernaryOp, QuaternaryOp, MultipleOp, DnnOp, and SpoofFusedOp.) Parameterized Hops (such as DataOp) can check that the number of parameters matches the number of inputs.

Specified by:

checkArity in class Hop

getParamIndexMap

public HashMap<String,Integer> getParamIndexMap()

getOpString

public String getOpString()

Specified by:

getOpString in class Hop

getOp

public Types.ParamBuiltinOp getOp()

setOutputPermutationMatrix

public void setOutputPermutationMatrix(boolean flag)

getTargetHop

public Hop getTargetHop()

getParameterHop

public Hop getParameterHop(String name)

isGPUEnabled

public boolean isGPUEnabled()

Description copied from class: Hop

In memory-based optimizer mode (see OptimizerUtils.isMemoryBasedOptLevel()), the exectype is determined by checking this method as well as memory budget of this Hop. Please see findExecTypeByMemEstimate for more detail. This method is necessary because not all operator are supported efficiently on GPU (for example: operations on frames and scalar as well as operations such as table).

Specified by:

isGPUEnabled in class Hop

Returns:

true if the Hop is eligible for GPU Exectype.

isMultiThreadedOpType

public boolean isMultiThreadedOpType()

Specified by:

isMultiThreadedOpType in class MultiThreadedHop

constructLops

public Lop constructLops()

Specified by:

constructLops in class Hop

computeMemEstimate

public void computeMemEstimate(MemoTable memo)

Description copied from class: Hop

Computes the estimate of memory required to store the input/output of this hop in memory. This is the default implementation (orchestration of hop-specific implementation) that should suffice for most hops. If a hop requires more control, this method should be overwritten with awareness of (1) output estimates, and (2) propagation of worst-case matrix characteristics (dimensions, sparsity). TODO remove memo table and, on constructor refresh, inference in refresh, single compute mem, maybe general computeMemEstimate, flags to indicate if estimate or not.

Overrides:

computeMemEstimate in class Hop

Parameters:

memo - memory table

allowsAllExecTypes

public boolean allowsAllExecTypes()

Specified by:

allowsAllExecTypes in class Hop

refreshSizeInformation

public void refreshSizeInformation()

Description copied from class: Hop

Update the output size information for this hop.

Specified by:

refreshSizeInformation in class Hop

clone

public Object clone()
             throws CloneNotSupportedException

Specified by:

clone in class Hop

Throws:

CloneNotSupportedException

compare

public boolean compare(Hop that)

Specified by:

compare in class Hop

isTransposeSafe

public boolean isTransposeSafe()

Description copied from class: Hop

Defines if this operation is transpose-safe, which means that the result of op(input) is equivalent to op(t(input)). Usually, this applies to aggregate operations with fixed output dimension. Finally, this information is very useful in order to safely optimize the plan for sparse vectors, which otherwise would be (currently) always represented dense.

Overrides:

isTransposeSafe in class Hop

Returns:

always returns false

isCountFunction

public boolean isCountFunction()

isKnownNGroups

public boolean isKnownNGroups()

isTargetDiagInput

public boolean isTargetDiagInput()

getParamservPseudoFunctionCalls

public List<FunctionOp> getParamservPseudoFunctionCalls()