Prioritized Experience Replay
In this paper we develop a framework for prioritizing experience, so as to replay important transitions more frequently, and therefore learn more efficiently.
Thereās another paper that tries to take a generative angle, Prioritized Generative Replay.