Package org.apache.sysds.api.mlcontext
Class ScriptExecutor
- java.lang.Object
-
- org.apache.sysds.api.mlcontext.ScriptExecutor
-
public class ScriptExecutor extends Object
ScriptExecutor executes a DML or PYDML Script object using SystemDS. This is accomplished by calling theexecute(org.apache.sysds.api.mlcontext.Script)
method.Script execution via the MLContext API typically consists of the following steps:
- Language Steps
- Parse script into program
- Live variable analysis
- Validate program
- HOP (High-Level Operator) Steps
- Construct HOP DAGs
- Static rewrites
- Intra-/Inter-procedural analysis
- Dynamic rewrites
- Compute memory estimates
- Rewrite persistent reads and writes (MLContext-specific)
- LOP (Low-Level Operator) Steps
- Contruct LOP DAGs
- Generate runtime program
- Execute runtime program
- Dynamic recompilation
Modifications to these steps can be accomplished by subclassing ScriptExecutor. For more information, please see the
execute(org.apache.sysds.api.mlcontext.Script)
method. - Language Steps
-
-
Constructor Summary
Constructors Constructor Description ScriptExecutor()
ScriptExecutor constructor.ScriptExecutor(DMLConfig config)
ScriptExecutor constructor, where the configuration properties are passed in.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
compile(Script script)
void
compile(Script script, boolean performHOPRewrites)
Compile a DML or PYDML script.MLResults
execute(Script script)
Execute a DML or PYDML script.DMLConfig
getConfig()
Obtain the SystemDS configuration properties.DMLProgram
getDmlProgram()
Obtain the programDMLTranslator
getDmlTranslator()
Obtain the translatorExecutionContext
getExecutionContext()
Obtain the execution contextMLContext.ExecutionType
getExecutionType()
Obtain the current execution environment.Program
getRuntimeProgram()
Obtain the runtime programScript
getScript()
Obtain the Script object associated with this ScriptExecutorboolean
isMaintainSymbolTable()
Obtain whether or not all values should be maintained in the symbol table after execution.void
setConfig(DMLConfig config)
Set the SystemDS configuration properties.void
setExecutionType(MLContext.ExecutionType executionType)
Set the execution environment.void
setExplain(boolean explain)
Whether or not an explanation of the DML/PYDML program should be output to standard output.void
setExplainLevel(MLContext.ExplainLevel explainLevel)
Set the level of program explanation that should be displayed if explain is set to true.void
setForceGPU(boolean enabled)
Whether or not to force GPU usage.void
setGPU(boolean enabled)
Whether or not to enable GPU usage.void
setInit(boolean init)
Whether or not to initialize the scratch_space, bufferpool, etc.void
setMaintainSymbolTable(boolean maintainSymbolTable)
Set whether or not all values should be maintained in the symbol table after execution.void
setStatistics(boolean statistics)
Whether or not statistics about the DML/PYDML program should be output to standard output.void
setStatisticsMaxHeavyHitters(int maxHeavyHitters)
Set the maximum number of heavy hitters to display with statistics.
-
-
-
Constructor Detail
-
ScriptExecutor
public ScriptExecutor()
ScriptExecutor constructor.
-
ScriptExecutor
public ScriptExecutor(DMLConfig config)
ScriptExecutor constructor, where the configuration properties are passed in.- Parameters:
config
- the configuration properties to use by the ScriptExecutor
-
-
Method Detail
-
compile
public void compile(Script script)
-
compile
public void compile(Script script, boolean performHOPRewrites)
Compile a DML or PYDML script. This will help analysis of DML programs that have dynamic recompilation flag set to false without actually executing it. This is broken down into the following primary methods:- Parameters:
script
- the DML or PYDML script to compileperformHOPRewrites
- should perform static rewrites, perform intra-/inter-procedural analysis to propagate size information into functions and apply dynamic rewrites
-
execute
public MLResults execute(Script script)
Execute a DML or PYDML script. This is broken down into the following primary methods:- Parameters:
script
- the DML or PYDML script to execute- Returns:
- the results as a MLResults object
-
setConfig
public void setConfig(DMLConfig config)
Set the SystemDS configuration properties.- Parameters:
config
- The configuration properties
-
getDmlProgram
public DMLProgram getDmlProgram()
Obtain the program- Returns:
- the program
-
getDmlTranslator
public DMLTranslator getDmlTranslator()
Obtain the translator- Returns:
- the translator
-
getRuntimeProgram
public Program getRuntimeProgram()
Obtain the runtime program- Returns:
- the runtime program
-
getExecutionContext
public ExecutionContext getExecutionContext()
Obtain the execution context- Returns:
- the execution context
-
getScript
public Script getScript()
Obtain the Script object associated with this ScriptExecutor- Returns:
- the Script object associated with this ScriptExecutor
-
setExplain
public void setExplain(boolean explain)
Whether or not an explanation of the DML/PYDML program should be output to standard output.- Parameters:
explain
-true
if explanation should be output,false
otherwise
-
setStatistics
public void setStatistics(boolean statistics)
Whether or not statistics about the DML/PYDML program should be output to standard output.- Parameters:
statistics
-true
if statistics should be output,false
otherwise
-
setStatisticsMaxHeavyHitters
public void setStatisticsMaxHeavyHitters(int maxHeavyHitters)
Set the maximum number of heavy hitters to display with statistics.- Parameters:
maxHeavyHitters
- the maximum number of heavy hitters
-
isMaintainSymbolTable
public boolean isMaintainSymbolTable()
Obtain whether or not all values should be maintained in the symbol table after execution.- Returns:
true
if all values should be maintained in the symbol table,false
otherwise
-
setMaintainSymbolTable
public void setMaintainSymbolTable(boolean maintainSymbolTable)
Set whether or not all values should be maintained in the symbol table after execution.- Parameters:
maintainSymbolTable
-true
if all values should be maintained in the symbol table,false
otherwise
-
setInit
public void setInit(boolean init)
Whether or not to initialize the scratch_space, bufferpool, etc. Note that any redundant initialize (e.g., multiple scripts from one MLContext) clears existing files from the scratch space and buffer pool.- Parameters:
init
-true
if should initialize,false
otherwise
-
setExplainLevel
public void setExplainLevel(MLContext.ExplainLevel explainLevel)
Set the level of program explanation that should be displayed if explain is set to true.- Parameters:
explainLevel
- the level of program explanation
-
setGPU
public void setGPU(boolean enabled)
Whether or not to enable GPU usage.- Parameters:
enabled
-true
if enabled,false
otherwise
-
setForceGPU
public void setForceGPU(boolean enabled)
Whether or not to force GPU usage.- Parameters:
enabled
-true
if enabled,false
otherwise
-
getConfig
public DMLConfig getConfig()
Obtain the SystemDS configuration properties.- Returns:
- the configuration properties
-
getExecutionType
public MLContext.ExecutionType getExecutionType()
Obtain the current execution environment.- Returns:
- the execution environment
-
setExecutionType
public void setExecutionType(MLContext.ExecutionType executionType)
Set the execution environment.- Parameters:
executionType
- the execution environment
-
-