class StreamingLinearRegressionWithSGD extends StreamingLinearAlgorithm[LinearRegressionModel, LinearRegressionWithSGD] with Serializable
Train or predict a linear regression model on streaming data. Training uses
Stochastic Gradient Descent to update the model based on each new batch of
incoming data from a DStream (see LinearRegressionWithSGD for model equation)
Each batch of data is assumed to be an RDD of LabeledPoints. The number of data points per batch can vary, but the number of features must be constant. An initial weight vector must be provided.
Use a builder pattern to construct a streaming linear regression analysis in an application, like:
val model = new StreamingLinearRegressionWithSGD() .setStepSize(0.5) .setNumIterations(10) .setInitialWeights(Vectors.dense(...)) .trainOn(DStream)
- Annotations
- @Since("1.1.0")
- Alphabetic
- By Inheritance
- StreamingLinearRegressionWithSGD
- Serializable
- StreamingLinearAlgorithm
- Logging
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new StreamingLinearRegressionWithSGD()
Construct a StreamingLinearRegression object with default parameters: {stepSize: 0.1, numIterations: 50, miniBatchFraction: 1.0}.
Construct a StreamingLinearRegression object with default parameters: {stepSize: 0.1, numIterations: 50, miniBatchFraction: 1.0}. Initial weights must be set before using trainOn or predictOn (see
StreamingLinearAlgorithm)- Annotations
- @Since("1.1.0")
Type Members
- implicit class LogStringContext extends AnyRef
- Definition Classes
- Logging
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val algorithm: LinearRegressionWithSGD
The algorithm to use for updating.
The algorithm to use for updating.
- Definition Classes
- StreamingLinearRegressionWithSGD → StreamingLinearAlgorithm
- Annotations
- @Since("1.1.0")
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def initializeLogIfNecessary(isInterpreter: Boolean): Unit
- Attributes
- protected
- Definition Classes
- Logging
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isTraceEnabled(): Boolean
- Attributes
- protected
- Definition Classes
- Logging
- def latestModel(): LinearRegressionModel
Return the latest model.
Return the latest model.
- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.1.0")
- def log: Logger
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logDebug(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logError(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logInfo(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logName: String
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logTrace(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry, throwable: Throwable): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(entry: LogEntry): Unit
- Attributes
- protected
- Definition Classes
- Logging
- def logWarning(msg: => String): Unit
- Attributes
- protected
- Definition Classes
- Logging
- var model: Option[LinearRegressionModel]
The model to be updated and used for prediction.
The model to be updated and used for prediction.
- Attributes
- protected
- Definition Classes
- StreamingLinearRegressionWithSGD → StreamingLinearAlgorithm
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def predictOn(data: JavaDStream[Vector]): JavaDStream[Double]
Java-friendly version of
predictOn.Java-friendly version of
predictOn.- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.3.0")
- def predictOn(data: DStream[Vector]): DStream[Double]
Use the model to make predictions on batches of data from a DStream
Use the model to make predictions on batches of data from a DStream
- data
DStream containing feature vectors
- returns
DStream containing predictions
- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.1.0")
- def predictOnValues[K](data: JavaPairDStream[K, Vector]): JavaPairDStream[K, Double]
Java-friendly version of
predictOnValues.Java-friendly version of
predictOnValues.- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.3.0")
- def predictOnValues[K](data: DStream[(K, Vector)])(implicit arg0: ClassTag[K]): DStream[(K, Double)]
Use the model to make predictions on the values of a DStream and carry over its keys.
Use the model to make predictions on the values of a DStream and carry over its keys.
- K
key type
- data
DStream containing feature vectors
- returns
DStream containing the input keys and the predictions as values
- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.1.0")
- def setConvergenceTol(tolerance: Double): StreamingLinearRegressionWithSGD.this.type
Set the convergence tolerance.
Set the convergence tolerance. Default: 0.001.
- Annotations
- @Since("1.5.0")
- def setInitialWeights(initialWeights: Vector): StreamingLinearRegressionWithSGD.this.type
Set the initial weights.
Set the initial weights.
- Annotations
- @Since("1.1.0")
- def setMiniBatchFraction(miniBatchFraction: Double): StreamingLinearRegressionWithSGD.this.type
Set the fraction of each batch to use for updates.
Set the fraction of each batch to use for updates. Default: 1.0.
- Annotations
- @Since("1.1.0")
- def setNumIterations(numIterations: Int): StreamingLinearRegressionWithSGD.this.type
Set the number of iterations of gradient descent to run per update.
Set the number of iterations of gradient descent to run per update. Default: 50.
- Annotations
- @Since("1.1.0")
- def setRegParam(regParam: Double): StreamingLinearRegressionWithSGD.this.type
Set the regularization parameter.
Set the regularization parameter. Default: 0.0.
- Annotations
- @Since("2.0.0")
- def setStepSize(stepSize: Double): StreamingLinearRegressionWithSGD.this.type
Set the step size for gradient descent.
Set the step size for gradient descent. Default: 0.1.
- Annotations
- @Since("1.1.0")
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def trainOn(data: JavaDStream[LabeledPoint]): Unit
Java-friendly version of
trainOn.Java-friendly version of
trainOn.- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.3.0")
- def trainOn(data: DStream[LabeledPoint]): Unit
Update the model by training on batches of data from a DStream.
Update the model by training on batches of data from a DStream. This operation registers a DStream for training the model, and updates the model based on every subsequent batch of data from the stream.
- data
DStream containing labeled data
- Definition Classes
- StreamingLinearAlgorithm
- Annotations
- @Since("1.1.0")
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- def withLogContext(context: HashMap[String, String])(body: => Unit): Unit
- Attributes
- protected
- Definition Classes
- Logging
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)