Hindsight Experience Replay (HER)
This is a pretty fundamental paper with a pretty basic idea. Jason Ma explained it to me and ian.
The agent relabels trajectories with achieved states as pseudo-goals, turning failures into successes for training purposes. This provides learning signals even from trajectories that didn’t reach the original goal.
Goal-Conditioned RL essentially builds upon this idea.
Citation
(Andrychowicz et al., 2017)
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., & Zaremba, W. (2017). Hindsight Experience Replay (HER). https://arxiv.org/pdf/1707.01495