Reparametrization Trick

https://lilianweng.github.io/posts/2018-08-12-vae/#reparameterization-trick

In Variational Autoencoders (VAEs), we want to backpropagate through a sampling step. To make this possible, we reparameterize the random variable $z$:

Instead of sampling $z\sim \mathcal{N}(\mu ,{\sigma }^2)$ directly, we do:

$$z=\mu +\sigma \cdot ϵ,ϵ\sim \mathcal{N}(0,1)$$

• This makes $z$ a deterministic function of $\mu$, $\sigma$, and a random variable $ϵ$, which allows gradients to flow through $\mu$ and $\sigma$ during training.