Off-Policy Methods

Importance Sampling

Almost all off-policy methods utilize Importance Sampling.

Importance Sampling is a general technique for estimating expected values under one distribution given samples from another. We can use the following approximation


Learning this under the Particle Filter lecture. Basically, if the distribution you want to sample from is normal, you know what values to use. However, how would you do it for any distribution?

We can use the importance sampling principle.