Uses of Class
org.apache.sysds.runtime.meta.DataCharacteristics

Packages that use DataCharacteristics

Package	Description
org.apache.sysds.api.mlcontext
org.apache.sysds.hops
org.apache.sysds.hops.estim
org.apache.sysds.hops.recompile
org.apache.sysds.hops.rewrite
org.apache.sysds.lops
org.apache.sysds.resource.cost
org.apache.sysds.runtime.compress.io
org.apache.sysds.runtime.controlprogram
org.apache.sysds.runtime.controlprogram.caching
org.apache.sysds.runtime.controlprogram.context
org.apache.sysds.runtime.controlprogram.federated
org.apache.sysds.runtime.controlprogram.parfor
org.apache.sysds.runtime.data
org.apache.sysds.runtime.frame.data
org.apache.sysds.runtime.functionobjects
org.apache.sysds.runtime.instructions.cp
org.apache.sysds.runtime.instructions.spark
org.apache.sysds.runtime.instructions.spark.data
org.apache.sysds.runtime.instructions.spark.functions
org.apache.sysds.runtime.instructions.spark.utils
org.apache.sysds.runtime.io
org.apache.sysds.runtime.lineage
org.apache.sysds.runtime.matrix.data
org.apache.sysds.runtime.meta
org.apache.sysds.runtime.util

Uses of DataCharacteristics in org.apache.sysds.api.mlcontext

Constructors in org.apache.sysds.api.mlcontext with parameters of type DataCharacteristics

Constructor	Description
FrameMetadata(DataCharacteristics dataCharacteristics)	Constructor to create a FrameMetadata object based on a DataCharacteristics object.
MatrixMetadata(DataCharacteristics dataCharacteristics)	Constructor to create a MatrixMetadata object based on a DataCharacteristics object.

Uses of DataCharacteristics in org.apache.sysds.hops

Methods in org.apache.sysds.hops that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics[]	MemoTable.getAllInputStats(List<Hop> inputs)
DataCharacteristics	MemoTable.getAllInputStats(Hop input)
DataCharacteristics	Hop.getDataCharacteristics()

Methods in org.apache.sysds.hops with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static boolean	OptimizerUtils.checkSparkCollectMemoryBudget(DataCharacteristics dc, long memPinned)
static boolean	OptimizerUtils.checkSparkCollectMemoryBudget(DataCharacteristics dc, long memPinned, boolean checkBP)
static boolean	OptimizerUtils.checkSparseBlockCSRConversion(DataCharacteristics dcIn)
static long	OptimizerUtils.estimatePartitionedSizeExactSparsity(DataCharacteristics dc)	Estimates the footprint (in bytes) for a partitioned in-memory representation of a matrix with the given matrix characteristics
static long	OptimizerUtils.estimatePartitionedSizeExactSparsity(DataCharacteristics dc, boolean outputEmptyBlocks)
static long	OptimizerUtils.estimateSize(DataCharacteristics dc)
static long	OptimizerUtils.estimateSizeExactSparsity(DataCharacteristics dc)
static double	OptimizerUtils.getSparsity(DataCharacteristics dc)
static boolean	OptimizerUtils.isIndexingRangeBlockAligned(IndexRange ixrange, DataCharacteristics mc)	Indicates if the given indexing range is block aligned, i.e., it does not require global aggregation of blocks.
static boolean	OptimizerUtils.isValidCPDimensions(DataCharacteristics mc)
void	MemoTable.memoizeStatistics(long hopID, DataCharacteristics dc)

Uses of DataCharacteristics in org.apache.sysds.hops.estim

Methods in org.apache.sysds.hops.estim that return DataCharacteristics

Modifier and Type	Method	Description
static DataCharacteristics	EstimatorMatrixHistogram.MatrixHistogram.deriveOutputCharacteristics(EstimatorMatrixHistogram.MatrixHistogram h1, EstimatorMatrixHistogram.MatrixHistogram h2, double spOut, SparsityEstimator.OpCode op, long[] misc)
DataCharacteristics	EstimatorBasicAvg.estim(MMNode root)
DataCharacteristics	EstimatorBasicWorst.estim(MMNode root)
DataCharacteristics	EstimatorBitsetMM.estim(MMNode root)
DataCharacteristics	EstimatorDensityMap.estim(MMNode root)
DataCharacteristics	EstimatorLayeredGraph.estim(MMNode root)
DataCharacteristics	EstimatorMatrixHistogram.estim(MMNode root)
DataCharacteristics	EstimatorMatrixHistogram.estim(MMNode root, boolean topLevel)
DataCharacteristics	EstimatorSample.estim(MMNode root)
DataCharacteristics	EstimatorSampleRa.estim(MMNode root)
abstract DataCharacteristics	SparsityEstimator.estim(MMNode root)	Estimates the output sparsity of a DAG of matrix multiplications for the given operator graph of a single root node.
DataCharacteristics	MMNode.getDataCharacteristics()
DataCharacteristics	MMNode.setDataCharacteristics(DataCharacteristics mc)

Methods in org.apache.sysds.hops.estim with parameters of type DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	MMNode.setDataCharacteristics(DataCharacteristics mc)

Uses of DataCharacteristics in org.apache.sysds.hops.recompile

Methods in org.apache.sysds.hops.recompile that return types with arguments of type DataCharacteristics

Modifier and Type	Method	Description
Map<String,DataCharacteristics>	RecompileStatus.getTWriteStats()

Methods in org.apache.sysds.hops.recompile with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static boolean	Recompiler.checkCPCheckpoint(DataCharacteristics dc)

Uses of DataCharacteristics in org.apache.sysds.hops.rewrite

Methods in org.apache.sysds.hops.rewrite with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static long	HopRewriteUtils.getMaxInputDim(DataCharacteristics[] dc, boolean dim1)
static long	HopRewriteUtils.getSumValidInputDims(DataCharacteristics[] mc, boolean dim1)
static long	HopRewriteUtils.getSumValidInputNnz(DataCharacteristics[] mc, boolean worstcase)
static boolean	HopRewriteUtils.hasValidInputDims(DataCharacteristics[] mc, boolean dim1)
static boolean	HopRewriteUtils.hasValidInputNnz(DataCharacteristics[] mc, boolean worstcase)

Uses of DataCharacteristics in org.apache.sysds.lops

Methods in org.apache.sysds.lops with parameters of type DataCharacteristics

Modifier and Type	Method	Description
void	OutputParameters.setDimensions(DataCharacteristics dc)

Uses of DataCharacteristics in org.apache.sysds.resource.cost

Constructors in org.apache.sysds.resource.cost with parameters of type DataCharacteristics

Constructor	Description
VarStats(DataCharacteristics dc)
VarStats(DataCharacteristics dc, long sizeEstimate)

Uses of DataCharacteristics in org.apache.sysds.runtime.compress.io

Methods in org.apache.sysds.runtime.compress.io with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static void	WriterCompressed.writeRDDToHDFS(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> src, String path, int blen, DataCharacteristics mc)

Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram

Methods in org.apache.sysds.runtime.controlprogram with parameters of type DataCharacteristics

Modifier and Type	Method	Description
long	ParForProgramBlock.PartitionFormat.getNumColumns(DataCharacteristics mc)
long	ParForProgramBlock.PartitionFormat.getNumParts(DataCharacteristics mc)
long	ParForProgramBlock.PartitionFormat.getNumRows(DataCharacteristics mc)

Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.caching

Methods in org.apache.sysds.runtime.controlprogram.caching that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	CacheableData.getDataCharacteristics()
DataCharacteristics	CacheBlock.getDataCharacteristics()

Methods in org.apache.sysds.runtime.controlprogram.caching with parameters of type DataCharacteristics

Modifier and Type	Method	Description
void	MatrixObject.updateDataCharacteristics(DataCharacteristics dc)

Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.context

Methods in org.apache.sysds.runtime.controlprogram.context that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	ExecutionContext.getDataCharacteristics(String varname)

Methods in org.apache.sysds.runtime.controlprogram.context with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static FrameObject	ExecutionContext.createFrameObject(DataCharacteristics dc)
static MatrixObject	ExecutionContext.createMatrixObject(DataCharacteristics dc)
static TensorBlock	SparkExecutionContext.toTensorBlock(org.apache.spark.api.java.JavaPairRDD<TensorIndexes,TensorBlock> rdd, DataCharacteristics dc)

Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.federated

Methods in org.apache.sysds.runtime.controlprogram.federated that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	MatrixLineagePair.getDataCharacteristics()

Uses of DataCharacteristics in org.apache.sysds.runtime.controlprogram.parfor

Constructors in org.apache.sysds.runtime.controlprogram.parfor with parameters of type DataCharacteristics

Constructor	Description
DataPartitionerRemoteSparkMapper(DataCharacteristics mc, Types.FileFormat fmt, ParForProgramBlock.PDataPartitionFormat dpf, int n)
RemoteDPParForSparkWorker(String program, boolean isLocal, HashMap<String,byte[]> clsMap, String inputVar, String iterVar, boolean cpCaching, DataCharacteristics mc, boolean tSparseCol, ParForProgramBlock.PartitionFormat dpf, Types.FileFormat fmt, org.apache.spark.util.LongAccumulator atasks, org.apache.spark.util.LongAccumulator aiters)

Uses of DataCharacteristics in org.apache.sysds.runtime.data

Methods in org.apache.sysds.runtime.data that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	TensorBlock.getDataCharacteristics()

Methods in org.apache.sysds.runtime.data with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static List<IndexedTensorBlock>	LibTensorReorg.reshape(IndexedTensorBlock in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks)	MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.

Uses of DataCharacteristics in org.apache.sysds.runtime.frame.data

Methods in org.apache.sysds.runtime.frame.data that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	FrameBlock.getDataCharacteristics()

Uses of DataCharacteristics in org.apache.sysds.runtime.functionobjects

Methods in org.apache.sysds.runtime.functionobjects with parameters of type DataCharacteristics

Modifier and Type	Method	Description
boolean	DiagIndex.computeDimension(DataCharacteristics in, DataCharacteristics out)
abstract boolean	IndexFunction.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	OffsetColumnIndex.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	ReduceAll.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	ReduceCol.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	ReduceDiag.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	ReduceRow.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	RevIndex.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	SortIndex.computeDimension(DataCharacteristics in, DataCharacteristics out)
boolean	SwapIndex.computeDimension(DataCharacteristics in, DataCharacteristics out)

Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.cp

Methods in org.apache.sysds.runtime.instructions.cp that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	Encrypted.getDataCharacteristics()

Methods in org.apache.sysds.runtime.instructions.cp with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static Instruction	VariableCPInstruction.prepCreatevarInstruction(String varName, String fileName, boolean fNameOverride, Types.DataType dt, String format, DataCharacteristics mc, MatrixObject.UpdateType update)
static Instruction	VariableCPInstruction.prepCreatevarInstruction(String varName, String fileName, boolean fNameOverride, Types.DataType dt, String format, DataCharacteristics mc, MatrixObject.UpdateType update, boolean hasHeader, String delim, boolean sparse)
void	Data.updateDataCharacteristics(DataCharacteristics mc)

Constructors in org.apache.sysds.runtime.instructions.cp with parameters of type DataCharacteristics

Constructor	Description
CiphertextMatrix(int[] dims, DataCharacteristics dc, byte[] data)
Encrypted(int[] dims, DataCharacteristics dc, byte[] data, Types.DataType dt)
PlaintextMatrix(int[] dims, DataCharacteristics dc, byte[] data)

Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark

Methods in org.apache.sysds.runtime.instructions.spark with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static MatrixBlock	MatrixIndexingSPInstruction.inmemoryIndexing(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in1, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)
static boolean	MatrixIndexingSPInstruction.isMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in, DataCharacteristics mcIn, DataCharacteristics mcOut, IndexRange ixrange)	Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).
static boolean	MatrixIndexingSPInstruction.isSingleBlockLookup(DataCharacteristics mcIn, IndexRange ixrange)	Indicates if the given index range only covers a single blocks of the inputs matrix.
void	ParameterizedBuiltinSPInstruction.setOutputCharacteristicsForGroupedAgg(DataCharacteristics mc1, DataCharacteristics mcOut, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> out)

Constructors in org.apache.sysds.runtime.instructions.spark with parameters of type DataCharacteristics

Constructor	Description
MergeWithShiftedBlocks(DataCharacteristics mc1, DataCharacteristics mc2, boolean cbind)
PadBlocksFunction(DataCharacteristics mcOut)
ShiftMatrix(DataCharacteristics mc1, DataCharacteristics mc2, boolean cbind)

Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark.data

Methods in org.apache.sysds.runtime.instructions.spark.data that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	PartitionedBroadcast.getDataCharacteristics()
DataCharacteristics	RDDObject.getDataCharacteristics()

Methods in org.apache.sysds.runtime.instructions.spark.data with parameters of type DataCharacteristics

Modifier and Type	Method	Description
void	RDDObject.setDataCharacteristics(DataCharacteristics dc)

Constructors in org.apache.sysds.runtime.instructions.spark.data with parameters of type DataCharacteristics

Constructor	Description
BlockPartitioner(DataCharacteristics mc, int numParts)
PartitionedBroadcast(org.apache.spark.broadcast.Broadcast<PartitionedBlock<T>>[] broadcasts, DataCharacteristics dc)

Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark.functions

Constructors in org.apache.sysds.runtime.instructions.spark.functions with parameters of type DataCharacteristics

Constructor	Description
ExtractBlockForBinaryReblock(DataCharacteristics mcIn, DataCharacteristics mcOut)
IsBlockInList(long[] cols, DataCharacteristics mc)
IsBlockInRange(long rl, long ru, long cl, long cu, DataCharacteristics mc)
IsFrameBlockInRange(long rl, long ru, DataCharacteristics mcOut)

Uses of DataCharacteristics in org.apache.sysds.runtime.instructions.spark.utils

Methods in org.apache.sysds.runtime.instructions.spark.utils that return DataCharacteristics

Modifier and Type	Method	Description
static DataCharacteristics	SparkUtils.computeDataCharacteristics(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input)	Utility to compute dimensions and non-zeros in a given RDD of binary cells.

Methods in org.apache.sysds.runtime.instructions.spark.utils with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.binaryBlockToBinaryBlock(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mcIn, DataCharacteristics mcOut)
static org.apache.spark.api.java.JavaRDD<String>	FrameRDDConverterUtils.binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mcIn, FileFormatPropertiesCSV props, boolean strict)
static org.apache.spark.api.java.JavaRDD<String>	RDDConverterUtils.binaryBlockToCsv(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mcIn, FileFormatPropertiesCSV props, boolean strict)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	FrameRDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession, org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mc, Types.ValueType[] schema)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	FrameRDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext, org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> in, DataCharacteristics mc, Types.ValueType[] schema)	Deprecated.
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	RDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SparkSession sparkSession, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mc, boolean toVector)
static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>	RDDConverterUtils.binaryBlockToDataFrame(org.apache.spark.sql.SQLContext sqlContext, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mc, boolean toVector)	Deprecated.
static org.apache.spark.api.java.JavaRDD<String>	RDDConverterUtils.binaryBlockToLibsvm(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mcIn, FileFormatPropertiesLIBSVM props, boolean strict)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	FrameRDDConverterUtils.binaryBlockToMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input, DataCharacteristics mcIn, DataCharacteristics mcOut)
static org.apache.spark.api.java.JavaRDD<String>	FrameRDDConverterUtils.binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock> input, DataCharacteristics mcIn)
static org.apache.spark.api.java.JavaRDD<String>	RDDConverterUtils.binaryBlockToTextCell(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> in, DataCharacteristics mc)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.binaryCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixCell> input, DataCharacteristics mcOut, boolean outputEmptyBlocks)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtilsExt.coordinateMatrixToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input, DataCharacteristics mcIn, boolean outputEmptyBlocks)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtilsExt.coordinateMatrixToBinaryBlock(org.apache.spark.SparkContext sc, org.apache.spark.mllib.linalg.distributed.CoordinateMatrix input, DataCharacteristics mcIn, boolean outputEmptyBlocks)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, Types.ValueType[] schema, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaRDD<String> input, DataCharacteristics mcOut, Types.ValueType[] schema, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.csvToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaRDD<String> input, DataCharacteristics mcOut, boolean hasHeader, String delim, boolean fill, double fillValue, Set<String> naStrings)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID, Pair<String[],Types.ValueType[]> out)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.dataFrameToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> df, DataCharacteristics mc, boolean containsID, boolean isVector)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	SparkUtils.getEmptyBlockRDD(org.apache.spark.api.java.JavaSparkContext sc, DataCharacteristics mc)	Creates an RDD of empty blocks according to the given matrix characteristics.
static int	SparkUtils.getNumPreferredPartitions(DataCharacteristics dc)
static int	SparkUtils.getNumPreferredPartitions(DataCharacteristics dc, boolean outputEmptyBlocks)
static int	SparkUtils.getNumPreferredPartitions(DataCharacteristics dc, org.apache.spark.api.java.JavaPairRDD<?,?> in)
static void	RDDConverterUtils.libsvmToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, String pathIn, String pathX, String pathY, DataCharacteristics mcOutX)	Converts a libsvm text input file into two binary block matrices for features and labels, and saves these to the specified output files.
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.libsvmToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mc, String delim, String indexDelim)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.matrixBlockToAlignedMatrixBlock(org.apache.spark.api.java.JavaPairRDD<Long,MatrixBlock> input, DataCharacteristics mcIn, DataCharacteristics mcOut)
static org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,FrameBlock>	FrameRDDConverterUtils.matrixBlockToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input, DataCharacteristics mcIn)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.matrixBlockToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> input, DataCharacteristics dcIn)
static void	SparkUtils.postprocessUltraSparseOutput(MatrixObject mo, DataCharacteristics mcOut)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> in, DataCharacteristics mcOut, Types.ValueType[] schema)
static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock>	RDDConverterUtils.textCellToBinaryBlock(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> input, DataCharacteristics mcOut, boolean outputEmptyBlocks, FileFormatPropertiesMM mmProps)
static org.apache.spark.api.java.JavaPairRDD<Long,FrameBlock>	FrameRDDConverterUtils.textCellToBinaryBlockLongIndex(org.apache.spark.api.java.JavaSparkContext sc, org.apache.spark.api.java.JavaPairRDD<Long,org.apache.hadoop.io.Text> input, DataCharacteristics mc, Types.ValueType[] schema)

Constructors in org.apache.sysds.runtime.instructions.spark.utils with parameters of type DataCharacteristics

Constructor	Description
BinaryCellToBinaryBlockFunction(DataCharacteristics mc)

Uses of DataCharacteristics in org.apache.sysds.runtime.io

Methods in org.apache.sysds.runtime.io with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static List<org.apache.commons.lang3.tuple.Pair<FederatedRange,FederatedData>>	ReaderWriterFederated.read(String file, DataCharacteristics mc)	Read a federated map from disk, It is not initialized before it is used in: org.apache.sysds.runtime.instructions.fed.InitFEDInstruction

Uses of DataCharacteristics in org.apache.sysds.runtime.lineage

Methods in org.apache.sysds.runtime.lineage that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	LineageCacheEntry.getDataCharacteristics()

Uses of DataCharacteristics in org.apache.sysds.runtime.matrix.data

Methods in org.apache.sysds.runtime.matrix.data that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	MatrixBlock.getDataCharacteristics()

Methods in org.apache.sysds.runtime.matrix.data with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static long	MatrixBlock.estimateSizeInMemory(DataCharacteristics dc)
static boolean	MatrixBlock.evalSparseFormatInMemory(DataCharacteristics dc)
static List<IndexedMatrixValue>	LibMatrixReorg.reshape(IndexedMatrixValue in, DataCharacteristics mcIn, DataCharacteristics mcOut, boolean rowwise, boolean outputEmptyBlocks)	MR/SPARK reshape interface - for reshape we cannot view blocks independently, and hence, there are different CP and MR interfaces.

Uses of DataCharacteristics in org.apache.sysds.runtime.meta

Subclasses of DataCharacteristics in org.apache.sysds.runtime.meta

Modifier and Type	Class	Description
class	MatrixCharacteristics
class	TensorCharacteristics

Methods in org.apache.sysds.runtime.meta that return DataCharacteristics

Modifier and Type	Method	Description
DataCharacteristics	MetaData.getDataCharacteristics()
DataCharacteristics	MetaDataAll.getDataCharacteristics()
abstract DataCharacteristics	DataCharacteristics.set(long[] dims, int blocksize)
abstract DataCharacteristics	DataCharacteristics.set(long[] dims, int blocksize, long nnz)
abstract DataCharacteristics	DataCharacteristics.set(long nr, long nc, int blen)
abstract DataCharacteristics	DataCharacteristics.set(long nr, long nc, int blen, long nnz)
abstract DataCharacteristics	DataCharacteristics.set(DataCharacteristics that)
DataCharacteristics	MatrixCharacteristics.set(long[] dims, int blocksize)
DataCharacteristics	MatrixCharacteristics.set(long[] dims, int blocksize, long nnz)
DataCharacteristics	MatrixCharacteristics.set(long nr, long nc, int blen)
DataCharacteristics	MatrixCharacteristics.set(long nr, long nc, int blen, long nnz)
DataCharacteristics	MatrixCharacteristics.set(DataCharacteristics that)
DataCharacteristics	TensorCharacteristics.set(long[] dims, int blocksize)
DataCharacteristics	TensorCharacteristics.set(long[] dims, int blocksize, long nnz)
DataCharacteristics	TensorCharacteristics.set(long nr, long nc, int blen)
DataCharacteristics	TensorCharacteristics.set(long nr, long nc, int blen, long nnz)
DataCharacteristics	TensorCharacteristics.set(DataCharacteristics that)
DataCharacteristics	DataCharacteristics.setBlocksize(int blen)
abstract DataCharacteristics	DataCharacteristics.setCols(long clen)
DataCharacteristics	MatrixCharacteristics.setCols(long clen)
DataCharacteristics	TensorCharacteristics.setCols(long clen)
abstract DataCharacteristics	DataCharacteristics.setDim(int i, long dim)
DataCharacteristics	MatrixCharacteristics.setDim(int i, long dim)
DataCharacteristics	TensorCharacteristics.setDim(int i, long dim)
abstract DataCharacteristics	DataCharacteristics.setDimension(long nr, long nc)
DataCharacteristics	MatrixCharacteristics.setDimension(long nr, long nc)
DataCharacteristics	TensorCharacteristics.setDimension(long nr, long nc)
abstract DataCharacteristics	DataCharacteristics.setDims(long[] dims)
DataCharacteristics	MatrixCharacteristics.setDims(long[] dims)
DataCharacteristics	TensorCharacteristics.setDims(long[] dims)
DataCharacteristics	DataCharacteristics.setNoEmptyBlocks(boolean flag)
abstract DataCharacteristics	DataCharacteristics.setNonZeros(long nnz)
DataCharacteristics	MatrixCharacteristics.setNonZeros(long nnz)
DataCharacteristics	TensorCharacteristics.setNonZeros(long nnz)
abstract DataCharacteristics	DataCharacteristics.setNonZerosBound(long nnz)
DataCharacteristics	MatrixCharacteristics.setNonZerosBound(long nnz)
DataCharacteristics	TensorCharacteristics.setNonZerosBound(long nnz)
abstract DataCharacteristics	DataCharacteristics.setRows(long rlen)
DataCharacteristics	MatrixCharacteristics.setRows(long rlen)
DataCharacteristics	TensorCharacteristics.setRows(long rlen)

Methods in org.apache.sysds.runtime.meta with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static void	DataCharacteristics.aggregateBinary(DataCharacteristics dim1, DataCharacteristics dim2, AggregateBinaryOperator op, DataCharacteristics dimOut)
static void	DataCharacteristics.aggregateUnary(DataCharacteristics dim, AggregateUnaryOperator op, DataCharacteristics dimOut)
static void	DataCharacteristics.reorg(DataCharacteristics dim, ReorgOperator op, DataCharacteristics dimOut)
abstract DataCharacteristics	DataCharacteristics.set(DataCharacteristics that)
DataCharacteristics	MatrixCharacteristics.set(DataCharacteristics that)
DataCharacteristics	TensorCharacteristics.set(DataCharacteristics that)
static void	MetaDataUtils.updateAppendDataCharacteristics(DataCharacteristics mc1, DataCharacteristics mc2, DataCharacteristics mcOut, boolean cbind)

Constructors in org.apache.sysds.runtime.meta with parameters of type DataCharacteristics

Constructor	Description
MatrixCharacteristics(DataCharacteristics that)
MetaData(DataCharacteristics dc)
MetaDataFormat(DataCharacteristics dc, Types.FileFormat fmt)
TensorCharacteristics(DataCharacteristics that)

Uses of DataCharacteristics in org.apache.sysds.runtime.util

Methods in org.apache.sysds.runtime.util with parameters of type DataCharacteristics

Modifier and Type	Method	Description
static String	HDFSTool.metaDataToString(Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties)
static void	DataConverter.writeMatrixToHDFS(MatrixBlock mat, String dir, Types.FileFormat fmt, DataCharacteristics dc)
static void	DataConverter.writeMatrixToHDFS(MatrixBlock mat, String dir, Types.FileFormat fmt, DataCharacteristics dc, int replication, FileFormatProperties formatProperties)
static void	DataConverter.writeMatrixToHDFS(MatrixBlock mat, String dir, Types.FileFormat fmt, DataCharacteristics dc, int replication, FileFormatProperties formatProperties, boolean diag)
static void	HDFSTool.writeMetaDataFile(String mtdfile, Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics mc, Types.FileFormat fmt)
static void	HDFSTool.writeMetaDataFile(String mtdfile, Types.ValueType vt, Types.ValueType[] schema, Types.DataType dt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties)
static void	HDFSTool.writeMetaDataFile(String mtdfile, Types.ValueType vt, DataCharacteristics mc, Types.FileFormat fmt)
static void	HDFSTool.writeMetaDataFile(String mtdfile, Types.ValueType vt, DataCharacteristics dc, Types.FileFormat fmt, FileFormatProperties formatProperties)
static void	HDFSTool.writeMetaDataFileFrame(String mtdfile, Types.ValueType[] schema, DataCharacteristics dc, Types.FileFormat fmt)
static void	DataConverter.writeTensorToHDFS(TensorBlock tensor, String dir, Types.FileFormat fmt, DataCharacteristics dc)