Policy Gradient Methods

Monte-Carlo Policy Gradient (REINFORCE)

This is just a specific Vanilla Policy Gradient, see that page for notes on how its implemented.

It uses the log-likelihood trick to estimate gradients of the expected return. $J(\theta )=\mathbb{E}[\nabla \log {\pi }_{\theta }(a\mid s)G_t]$

Resources

• https://lilianweng.github.io/posts/2018-04-08-policy-gradient/#reinforce

Related

• PPO