TD-Lambda

Eligibility Trace

Eligibility traces are one of the basic mechanisms of reinforcement learning. They unity and generalize TD and Monte Carlo methods.

Frequency heurisitic: assign credit to most frequent states Recency heurisitic: assign credit to most recent states

Eligibility traces combine both herusitics.

$E_0(s)=0$ $E_t(s)=\gamma \lambda E_{t-1}(s)+1(S_t=s)$

https://www.youtube.com/watch?v=PnHCvfgC_ZA&list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-&index=4