Horizon Reduction Makes RL Scalable

Pretty important paper that addresses some of the scaling limitations. They found that everything (including model architecture) doesn’t scale as well as reducing the horizon.

minimal set of techniques (e.g., flow behavioral cloning and SARSA) that address the horizon issue in a scalable manner (see Section 6.1 for further discussions).