Off-Policy Methods

Importance Sampling

Almost all off-policy methods utilize Importance Sampling.

Importance Sampling is a general technique for estimating expected values under one distribution given samples from another. We can use the following approximation

${\mathbb{E}}_{x-q}[f(x)]={\int }_{x}q(x)f(x)dx$ $\approx \sum _{i=1}^n\frac{q(x_i)}{p(x_i)}f(x_i)$

F1TENTH

Learning this under the Particle Filter lecture. Basically, if the distribution you want to sample from is normal, you know what values to use. However, how would you do it for any distribution?

We can use the importance sampling principle.