Stochastic Gradient Descent (SGD)

Stochastic gradient descent samples the gradient.

$∆w=\alpha (v_{\pi }(S)-\hat{v}(S,w)){\nabla }_w\hat{v}(S,w)$

SGD

• Stochastic Gradient Descent Randomly select a subset of training data.
• Gradient depends only on selected subset

Helps avoid getting stuck in local minimum.

Stochastic gradient descent is a specific case of Mini-Batch Gradient Descent.

CS294

SGD minimizes expectations, for a differentiable function $f$ of $\theta$, SGD solves $\arg {\min }_{\theta }\mathbb{E}[f(\theta )]$ We can use this with Maximum Likelihood Estimation