Package org.apache.sysds.runtime.io

Class FrameReaderParquet

java.lang.Object

org.apache.sysds.runtime.io.FrameReader

org.apache.sysds.runtime.io.FrameReaderParquet

Direct Known Subclasses:

FrameReaderParquetParallel

public class FrameReaderParquet
extends FrameReader

Single-threaded frame parquet reader.

Constructor Summary

Constructors

Constructor	Description
FrameReaderParquet()

Method Summary

Modifier and Type	Method	Description
FrameBlock	readFrameFromHDFS(String fname, Types.ValueType[] schema, String[] names, long rlen, long clen)	Reads a Parquet file from HDFS and converts it into a FrameBlock.
FrameBlock	readFrameFromInputStream(InputStream is, Types.ValueType[] schema, String[] names, long rlen, long clen)

Methods inherited from class org.apache.sysds.runtime.io.FrameReader

getDefColNames, getDefSchema, readFrameFromHDFS, readFrameFromHDFS, readFrameFromInputStream, readFrameFromInputStream

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FrameReaderParquet

public FrameReaderParquet()

Method Detail

readFrameFromHDFS

public FrameBlock readFrameFromHDFS(String fname,
                                    Types.ValueType[] schema,
                                    String[] names,
                                    long rlen,
                                    long clen)
                             throws IOException,
                                    DMLRuntimeException

Reads a Parquet file from HDFS and converts it into a FrameBlock.

Specified by:

readFrameFromHDFS in class FrameReader

Parameters:

fname - The HDFS file path to the Parquet file.

schema - The expected data types of the columns.

names - The names of the columns.

rlen - The expected number of rows.

clen - The expected number of columns.

Returns:

A FrameBlock containing the data read from the Parquet file.

Throws:

IOException

DMLRuntimeException

readFrameFromInputStream

public FrameBlock readFrameFromInputStream(InputStream is,
                                           Types.ValueType[] schema,
                                           String[] names,
                                           long rlen,
                                           long clen)
                                    throws IOException,
                                           DMLRuntimeException

Specified by:

readFrameFromInputStream in class FrameReader

Throws:

IOException

DMLRuntimeException