Generative Model

Autoregressive Model

-Model joint probability as a product of conditionals: $p(x)={\prod }_{t=1}^{T}p(x_t\mid x_{<t})$

Saw this from the Self-Driving Car Companies AI Day.

• Where?

Examples:

• PixelRNN / PixelCNN (image)
• GPT, Transformer decoders (text)
• WaveNet (audio)

And it’s a term that seems to pop around. But I think that all it does is really allow you to predict (regress) future data based on past data. So isn’t this just a simple RNN?

They also mention this in the Neural Autoregressive Flow.

CS294

Assuming a fully expressive Bayes net structure, any joint distribution can be written as a product of conditionals $\log p(x)={\sum }_{i=1}^d\log p(x_i|x_{1:i-1})$