A Minimalist Approach to Offline Reinforcement Learning (TD3 + BC)

This paper’s core contribution is just add the BC loss to DDPG, and shows great improvement, i.e.

$\pi ={\text{argmax}}_{\pi }{\mathbb{E}}_{(s,a)\sim D}[\lambda Q(s,\pi (s))-(\pi (s)-a{)}^2]$

Shown in the Is Value Learning Really the Main Bottleneck in Offline RL.