Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning

Builds on top of Conservative Q-Learning.

I watched Sergey’s talk at RLC2024, where he talks about there’s a big gap with offline-RL and online-RL, and Cal-QL tries to close this gap.

Citation

(Nakamoto et al., 2023)

Nakamoto, M., Zhai, Y., Singh, A., Mark, M. S., Ma, Y., Finn, C., Kumar, A., & Levine, S. (2023). Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning. https://arxiv.org/pdf/2303.05479