IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures

First saw this when I was playing around with OGBench (but i think it was just the impala encoder?). Am realizing that it has much more implications? Cited by Learning to Drive from a World Model for example.

Multiple actors act in parallel and generate experiences that are sent to a central learner.

For some reason, I was confusing this with DAgger.

Citation

(Espeholt et al., 2018)

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., & Kavukcuoglu, K. (2018). IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures. https://arxiv.org/pdf/1802.01561