Package org.apache.sysds.resource

Class ResourceCompiler

java.lang.Object

org.apache.sysds.resource.ResourceCompiler

public class ResourceCompiler
extends Object

This class does full or partial program recompilation based on given runtime program. It uses the methods provided by hops.recompile.Recompiler). It keeps a state of the current recompilation phase in order to decide when to do full recompilation and when not.

Field Summary

Fields

Modifier and Type	Field	Description
static long	DEFAULT_DRIVER_MEMORY
static int	DEFAULT_DRIVER_THREADS
static long	DEFAULT_EXECUTOR_MEMORY
static int	DEFAULT_EXECUTOR_THREADS
static int	DEFAULT_NUMBER_EXECUTORS

Constructor Summary

Constructors

Constructor	Description
ResourceCompiler()

Method Summary

Modifier and Type	Method	Description
static Program	compile(String filePath, Map<String,String> args)
static Program	doFullRecompilation(Program program, long driverMemory, int driverCores)	Recompiling a given program for resource optimization for single node execution
static Program	doFullRecompilation(Program program, long driverMemory, int driverCores, int numberExecutors, long executorMemory, int executorCores)	Recompiling a given program for resource optimization for Spark execution
static void	setDriverConfigurations(long nodeMemory, int nodeNumCores)
static void	setExecutorConfigurations(int numExecutors, long nodeMemory, int nodeNumCores)
static void	setSingleNodeExecution()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_DRIVER_MEMORY

public static final long DEFAULT_DRIVER_MEMORY

See Also:

Constant Field Values

DEFAULT_DRIVER_THREADS

public static final int DEFAULT_DRIVER_THREADS

See Also:

Constant Field Values

DEFAULT_EXECUTOR_MEMORY

public static final long DEFAULT_EXECUTOR_MEMORY

See Also:

Constant Field Values

DEFAULT_EXECUTOR_THREADS

public static final int DEFAULT_EXECUTOR_THREADS

See Also:

Constant Field Values

DEFAULT_NUMBER_EXECUTORS

public static final int DEFAULT_NUMBER_EXECUTORS

See Also:

Constant Field Values

Constructor Detail

ResourceCompiler

public ResourceCompiler()

Method Detail

compile

public static Program compile(String filePath,
                              Map<String,String> args)
                       throws IOException

Throws:

IOException

doFullRecompilation

public static Program doFullRecompilation(Program program,
                                          long driverMemory,
                                          int driverCores)

Recompiling a given program for resource optimization for single node execution

Parameters:

program - program to be recompiled

driverMemory - target driver memory

driverCores - target driver threads/cores

Returns:

the recompiled program as a new Program instance

doFullRecompilation

public static Program doFullRecompilation(Program program,
                                          long driverMemory,
                                          int driverCores,
                                          int numberExecutors,
                                          long executorMemory,
                                          int executorCores)

Recompiling a given program for resource optimization for Spark execution

Parameters:

program - program to be recompiled

driverMemory - target driver memory

driverCores - target driver threads/cores

numberExecutors - target number of executor nodes

executorMemory - target executor memory

executorCores - target executor threads/cores

Returns:

the recompiled program as a new Program instance

setDriverConfigurations

public static void setDriverConfigurations(long nodeMemory,
                                           int nodeNumCores)

setExecutorConfigurations

public static void setExecutorConfigurations(int numExecutors,
                                             long nodeMemory,
                                             int nodeNumCores)

setSingleNodeExecution

public static void setSingleNodeExecution()