Eligibility Trace

? Eligibility traces are one of the basic mechanisms of reinforcement learning. They unity and generalize TD and Monte Carlo methods.

Frequency heurisitic: assign credit to most frequent states Recency heurisitic: assign credit to most recent states

Eligibility traces combine both herusitics.

$E_0(s)=0$ $E_t(s)=\gamma \lambda E_{t-1}(s)+1(S_t=s)$