SystemDS is a flexible, scalable machine learning system. SystemDS’s distinguishing characteristics are:
- Algorithm customizability via R-like and Python-like languages.
- Multiple execution modes, including Spark MLContext, Spark Batch, Standalone, and JMLC.
- Automatic optimization based on data and cluster characteristics to ensure both efficiency and scalability.
This version of SystemDS supports: Java 8+, Python 3.5+, Hadoop 3.x, and Spark 3.x, Nvidia CUDA 10.2 (CuDNN 7.x) Intel MKL (<=2019.x).
Various forms of documentation for SystemDS are available.
- a DML Language Reference for an list of operations possible inside SystemDS.
- Builtin Functions contains a collection of builtin functions providing an high level abstraction on complex machine learning algorithms.
- Algorithm Reference contains specifics on algorithms supported in systemds.
- Entity Resolution provides a collection of customizable entity resolution primitives and pipelines.
- Run SystemDS contains an Helloworld example along with an environment setup guide.
- Instructions on python can be found at Python Documentation
- The JavaDOC contains internal documentation of the system source code.
- Install from Source guides through setup from git download to running system.
- If you want to contribute take a look at Contributing
- R to DML walks through the basics of converting a script from R to dml.