Stochastic Gradient Descent

Stochastic gradient descent samples the gradient.

$∆w=\alpha (v_{\pi }(S)-\hat{v}(S,w)){\nabla }_w\hat{v}(S,w)$

Stochastic gradient descent is a specific case of Mini-Batch Gradient Descent.

CS294

SGD minimizes expectations, for a differentiable function $f$ of $\theta$, SGD solves $\arg {\min }_{\theta }\mathbb{E}[f(\theta )]$ We can use this with Maximum Likelihood Estimation