Exponential Moving Average (EMA)

https://medium.com/analytics-vidhya/understanding-exponential-moving-averages-e3f020d9d13b

$y_{t+1}=\alpha y_t+(1-\alpha )y_{t-1}$

An EMA model (Exponential Moving Average model) is typically used in machine learning to maintain a smoothed version of a model’s parameters over time.

Polyak Averaging

I was introduced to this when learning DDPG.

${\varphi }_{\text{targ}}\leftarrow \rho {\varphi }_{\text{targ}}+(1-\rho )\varphi$ It’s not quite the exact same as EMA above, because here we haev sort of two targets. We are moving our target in the direction of $\varphi$, but $\varphi$ is also continuously moving. ${\varphi }_{targ}$ is just always lagging behind o give smoother Q-Targets to reduce variance and help stability TD-updates.