Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions

“While in principle simply replacing existing architectures (e.g., ResNets [15] or smaller convolutional neural networks [11, 14]) with a Transformer is conceptually straightforward, devising a methodology that effectively makes use of such architectures is considerably more challenging. High-capacity models only make sense when we train on large and diverse datasets – small, narrow datasets simply do not require this much capacity and do not benefit from it.”

Citation

(Chebotar et al., 2023)

Chebotar, Y., Vuong, Q., Irpan, A., Hausman, K., Xia, F., Lu, Y., Kumar, A., Yu, T., Herzog, A., Pertsch, K., Gopalakrishnan, K., Ibarz, J., Nachum, O., Sontakke, S., Salazar, G., Tran, H. T., Peralta, J., Tan, C., Manjunath, D., … Levine, S. (2023). Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions. https://arxiv.org/abs/2309.10150