Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems

Good survey of the open problems.

off-policy policy gradient is a thing via Importance Sampling.

Citation

(Levine et al., 2020)

Levine, S., Kumar, A., Tucker, G., & Fu, J. (2020). Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. https://arxiv.org/pdf/2005.01643