Greedy in the Limit with Infinite Exploration

Greedy in the Limit with Infinite Exploration (GLIE)

All state-action pairs are explored infinitely many times, ${\lim }_{k\to \infty }{N}_k(s,a)=\infty$

The policy converges on a greedy policy, ${\lim }_{n\to \infty }{\pi }_k(a|s)=1(a=\underset{a^{\prime }\in A}{\mathrm{argmax}}{Q}_k(s,a^{\prime }))$

I initially undersold how important this is, but this is EXTREMELY important to understand.

We use this GLIE idea for Monte-Carlo Control.